1
Mod Zone / Re: Custom cooler mount
« on: October 15, 2023, 06:57:06 am »
I haven’t seen that but I definitely will try. A quick look at the source and I see it’s handling thread affinity, a requirement for Threadrippers on Windows.
Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
2
Mod Zone / Re: Custom cooler mount
« on: October 14, 2023, 10:53:27 am »
Thanks for the info, I know little about how the IBM WOF works.
The workload I'm testing with is from a graphics tool I've been working on for a while, which is interesting because I'd tuned the threading for 64C/128T Threadrippers and then started using it to compare other systems (and it's a nice workload for many cores). Send me a PM if you're interested in building it from source, I'm not ready to announce it yet so it's under wraps (GCC/Clang and CMake on most systems build out of the box).
The workload I'm testing with is from a graphics tool I've been working on for a while, which is interesting because I'd tuned the threading for 64C/128T Threadrippers and then started using it to compare other systems (and it's a nice workload for many cores). Send me a PM if you're interested in building it from source, I'm not ready to announce it yet so it's under wraps (GCC/Clang and CMake on most systems build out of the box).
3
Mod Zone / Re: Custom cooler mount
« on: October 05, 2023, 09:47:52 am »
Yes, I have the dual 18 core, and with the water cooling temps are around 40 degrees (same as the simpler solution from Vikings) and the machine is running silently (a quiet machine was the original aim). The same tests with a Noctua air cooler the temps are mid-50 and the fans are quiet (compared with the IBM HSF, anything is quiet).
We water cool racks of machines at work, with the big benefit being keeping the temps down keeps the clocks high and the power consumption down. With Threadrippers, for example, they run at boost clocks all the time, vs down-clocking to half the speed to stay within TDP, effectively doubling the throughput. Same with GPUs (which is where we do most of the water cooling, CPUs was just a curiosity).
Real life took over for a while but I will work on a jig to test the mounting pressure.
We water cool racks of machines at work, with the big benefit being keeping the temps down keeps the clocks high and the power consumption down. With Threadrippers, for example, they run at boost clocks all the time, vs down-clocking to half the speed to stay within TDP, effectively doubling the throughput. Same with GPUs (which is where we do most of the water cooling, CPUs was just a curiosity).
Real life took over for a while but I will work on a jig to test the mounting pressure.
4
Mod Zone / Re: Custom cooler mount
« on: September 12, 2023, 05:06:04 am »
I don't, but when comparing the exact same test with the IBM 3U HSF the temperatures were around 70 degrees.
I have a replacement mobo now (RCS were really helpful here) and I'm waiting on some CNC'd parts back with amends to make installing this easier. The design files are here for anyone interested:
https://github.com/cwoffenden/talosmods/tree/main/am5-sf2-cooler
(I'll be updating this and adding docs along the way)
I have a replacement mobo now (RCS were really helpful here) and I'm waiting on some CNC'd parts back with amends to make installing this easier. The design files are here for anyone interested:
https://github.com/cwoffenden/talosmods/tree/main/am5-sf2-cooler
(I'll be updating this and adding docs along the way)
5
Mod Zone / Re: Custom cooler mount
« on: August 23, 2023, 10:12:13 am »Do you have any idea how much force you are applying down through the module to the socket pins?I'm going to build a jig to measure this, for both the Noctua and EK options, then I can tailor the springs to make sure the force is similar to the IBM part. It'll keep me busy waiting for my replacement mobo to be delivered.
6
Mod Zone / Re: Custom cooler mount
« on: August 21, 2023, 07:49:06 am »
I found the problem with my second CPU, I appear to have dropped something onto the socket and bent a bunch of pins:
https://wip.numfum.com/cw/2023-08-21/cpu2-cu.jpeg
I have a replacement mobo coming from Raptor so will get back on with this in a few weeks.
https://wip.numfum.com/cw/2023-08-21/cpu2-cu.jpeg
I have a replacement mobo coming from Raptor so will get back on with this in a few weeks.
7
Mod Zone / Re: Custom cooler mount
« on: August 17, 2023, 11:45:30 am »
I've no idea of the amount of force. For the design I went with the same mounted hight (via trial and error and shims) as the Noctua cooler and the EK water block on an AM5 mobo.
What I found was, for the Noctua, until I got this height and the stiffness right then the system wasn't stable. Once I got this correct (measured at the spring hight, how far they were tensioned, how the cooler sits on the CPU including getting its eccentricity right) the system was a stable and I could wobble and shake the cooler. After the Noctua's SecuFirm2 mounts I did the same for the EK Pro water block. AM5 is a 1781 pin LGA, the POWER9 Sforza is a 2601 pin LGA, and have 40x40 vs 48x48 IHS sizes (but the coolers usually have 50x50 cold plates), so they're not like for like.
The socket I used for mounting and testing the various coolers had maybe 20 mount and test cycles, trying to keep the number down (since I'd read the socket was good for very few cycles). The second CPU socket, used for all the measuring and verification, seems to have been trashed in the process (I must have eventually squashed the pins, I'll need to get my SMT stereo microscope unpacked to look). The lack of independent loading mechanism (like on the AM5) appears to have been the downfall.
For stability testing I ran this machine hard for multi-day cycles (one of my own tools that 100% loads the CPU cores with lots of maths).
Since I trashed my second CPU socket I've not been able to test both CPUs yet, I need to solve that next.
What I found was, for the Noctua, until I got this height and the stiffness right then the system wasn't stable. Once I got this correct (measured at the spring hight, how far they were tensioned, how the cooler sits on the CPU including getting its eccentricity right) the system was a stable and I could wobble and shake the cooler. After the Noctua's SecuFirm2 mounts I did the same for the EK Pro water block. AM5 is a 1781 pin LGA, the POWER9 Sforza is a 2601 pin LGA, and have 40x40 vs 48x48 IHS sizes (but the coolers usually have 50x50 cold plates), so they're not like for like.
The socket I used for mounting and testing the various coolers had maybe 20 mount and test cycles, trying to keep the number down (since I'd read the socket was good for very few cycles). The second CPU socket, used for all the measuring and verification, seems to have been trashed in the process (I must have eventually squashed the pins, I'll need to get my SMT stereo microscope unpacked to look). The lack of independent loading mechanism (like on the AM5) appears to have been the downfall.
For stability testing I ran this machine hard for multi-day cycles (one of my own tools that 100% loads the CPU cores with lots of maths).
Since I trashed my second CPU socket I've not been able to test both CPUs yet, I need to solve that next.
8
Mod Zone / Re: Custom cooler mount
« on: August 14, 2023, 03:03:34 am »
With such good contact with the CPU, for regular use the radiator fans aren't even running, so it's just the sound of the pump.
9
Mod Zone / Re: Custom cooler mount
« on: August 12, 2023, 11:38:24 am »
Well I did what I said I wouldn't do and went all-in with custom mounts for the EK cooler to eliminate the copper heat spreader/shim:
https://wip.numfum.com/cw/2023-08-12/custom-mounts.jpeg
These attach to the earlier bracket and allow the AM5 water block to mount directly on the POWER9, all properly loaded and torqued:
https://wip.numfum.com/cw/2023-08-12/ek-am5-mounted.jpeg
The temps are amazing:
https://wip.numfum.com/cw/2023-08-12/bc2etc-gen-waku-ek-am5-custom.png
This is running flat-out and it's around 40 degrees. I'm using a graphite pad instead of paste, so it's not entirely comparable to the first tests I did with the Alphacool bracket from Vikings, and at a guess it would go even lower with the Thermal Grizzly paste (but I just couldn't bring myself to clean off any more paste!). I'll say I'm 100% happy with the result and will re-add the second CPU. The pad will never dry out and should be good for life.
I'm going to document and write this all up in the next weeks, plus make the CAD files available. What I will say, though, is if you want a quick and easy way to water cool, the option from Vikings is probably for you. My solution gets a few more degrees for a much greater cost (especially since this was all one-off custom machining, although with a group buy the price would come way down).
I do think the Noctua fan is a good solution for most people wanting a quieter T2 or Blackbird. Vikings Thum will hopefully publish his findings with a 3D printable version of the same bracket.
https://wip.numfum.com/cw/2023-08-12/custom-mounts.jpeg
These attach to the earlier bracket and allow the AM5 water block to mount directly on the POWER9, all properly loaded and torqued:
https://wip.numfum.com/cw/2023-08-12/ek-am5-mounted.jpeg
The temps are amazing:
https://wip.numfum.com/cw/2023-08-12/bc2etc-gen-waku-ek-am5-custom.png
This is running flat-out and it's around 40 degrees. I'm using a graphite pad instead of paste, so it's not entirely comparable to the first tests I did with the Alphacool bracket from Vikings, and at a guess it would go even lower with the Thermal Grizzly paste (but I just couldn't bring myself to clean off any more paste!). I'll say I'm 100% happy with the result and will re-add the second CPU. The pad will never dry out and should be good for life.
I'm going to document and write this all up in the next weeks, plus make the CAD files available. What I will say, though, is if you want a quick and easy way to water cool, the option from Vikings is probably for you. My solution gets a few more degrees for a much greater cost (especially since this was all one-off custom machining, although with a group buy the price would come way down).
I do think the Noctua fan is a good solution for most people wanting a quieter T2 or Blackbird. Vikings Thum will hopefully publish his findings with a 3D printable version of the same bracket.
10
Mod Zone / Re: Custom cooler mount
« on: July 14, 2023, 10:15:41 am »
Today I tested the mount with an EK AM5 water block. This needed a compromise in the design (which I could work around with custom threaded fittings, but each cooler/water block would need its own design, whereas what I have here works with any AM5 cooler) adding a 4mm copper heat spreader on top of the CPU:
https://wip.numfum.com/cw/2023-07-14/am5-copper-hs.jpeg
It's a smooth machined lump of copper with a pocket to grip the top of the CPU. I used a Thermal Grizzly Kryosheet pad instead of thermal paste (which was frustrating to cut to size), which I used to aid the potential repeatedly dismantling. I mounted the water block on that:
https://wip.numfum.com/cw/2023-07-14/am5-ekwb-1.jpeg
https://wip.numfum.com/cw/2023-07-14/am5-ekwb-2.jpeg
The whole thing is designed so the screws go all the way in (hand tight) to get the maximum out of the springs. Tightening was done in a cross pattern, a few turns at a time. Then the same Alphacool radiator hooked up I use for tests:
https://wip.numfum.com/cw/2023-07-14/am5-with-rad.jpeg
The mount is solid, as it is with the Noctua fan, and can be pushed and prodded with no play. I ran the same tests as I did with the Noctua and earlier Alphacool water block, with these numbers:
https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-waku-ek-am5.png
The temperatures are higher than the with the Alphacool:
https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-waku-acool.png
My solution runs nearly 10 degrees hotter, which is probably a combination of thermal pads vs paste (I used Kryonaut Extreme as the paste, at around $95 USD per pot over here, which we use at work for GPU water cooling) plus the compromise of that lump of copper. But a maximum of 57 degrees is cool running!
I've decided against custom fittings and will keep the compromise of copper heat spreader, but I will switch over to thermal paste and re-run the tests next week.
It's quiet, with the fans stalling and shutting off, and with temps better than the Noctua fan:
https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-noctua.png
And certainly better than the IBM HSF:
https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-hsf.png
More tests next week!
https://wip.numfum.com/cw/2023-07-14/am5-copper-hs.jpeg
It's a smooth machined lump of copper with a pocket to grip the top of the CPU. I used a Thermal Grizzly Kryosheet pad instead of thermal paste (which was frustrating to cut to size), which I used to aid the potential repeatedly dismantling. I mounted the water block on that:
https://wip.numfum.com/cw/2023-07-14/am5-ekwb-1.jpeg
https://wip.numfum.com/cw/2023-07-14/am5-ekwb-2.jpeg
The whole thing is designed so the screws go all the way in (hand tight) to get the maximum out of the springs. Tightening was done in a cross pattern, a few turns at a time. Then the same Alphacool radiator hooked up I use for tests:
https://wip.numfum.com/cw/2023-07-14/am5-with-rad.jpeg
The mount is solid, as it is with the Noctua fan, and can be pushed and prodded with no play. I ran the same tests as I did with the Noctua and earlier Alphacool water block, with these numbers:
https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-waku-ek-am5.png
The temperatures are higher than the with the Alphacool:
https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-waku-acool.png
My solution runs nearly 10 degrees hotter, which is probably a combination of thermal pads vs paste (I used Kryonaut Extreme as the paste, at around $95 USD per pot over here, which we use at work for GPU water cooling) plus the compromise of that lump of copper. But a maximum of 57 degrees is cool running!
I've decided against custom fittings and will keep the compromise of copper heat spreader, but I will switch over to thermal paste and re-run the tests next week.
It's quiet, with the fans stalling and shutting off, and with temps better than the Noctua fan:
https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-noctua.png
And certainly better than the IBM HSF:
https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-hsf.png
More tests next week!
11
Mod Zone / Re: Custom cooler mount
« on: July 13, 2023, 09:16:58 am »
Woo-hoo - the changes to strengthen the Noctua mount resulted in a stable system! I can wobble the cooler now and I have no crashes, so next will be long term testing (I have two earlier spares without amends for the AM5 mounts which I'll be sending to @vikings.thum). Some photos:
https://wip.numfum.com/cw/2023-07-13/noctua-1.jpeg
https://wip.numfum.com/cw/2023-07-13/noctua-2.jpeg
https://wip.numfum.com/cw/2023-07-13/noctua-3.jpeg
https://wip.numfum.com/cw/2023-07-13/secufirm2.jpeg
Tomorrow I'll test with a water block.
https://wip.numfum.com/cw/2023-07-13/noctua-1.jpeg
https://wip.numfum.com/cw/2023-07-13/noctua-2.jpeg
https://wip.numfum.com/cw/2023-07-13/noctua-3.jpeg
https://wip.numfum.com/cw/2023-07-13/secufirm2.jpeg
Tomorrow I'll test with a water block.
12
Talos II / Re: Arctic Tern bracket?
« on: July 12, 2023, 08:34:59 am »
I got my metal brackets back today, photo attached. I need to test and measure to qualify it, but on a once-over it looks as expected (I have powder coated black and bead blasted stainless steel variants).
https://wip.numfum.com/cw/2023-07-12/arctic-tern-pcie.jpeg
The micro-HDMI is the only worry, with the connectors being too far back from the bracket (and with nothing I could really do to correct it).
(The forum won't take any more uploads for me?)
https://wip.numfum.com/cw/2023-07-12/arctic-tern-pcie.jpeg
The micro-HDMI is the only worry, with the connectors being too far back from the bracket (and with nothing I could really do to correct it).
(The forum won't take any more uploads for me?)
13
Mod Zone / Re: Custom cooler mount
« on: June 25, 2023, 02:14:50 pm »
Thanks! I'll be back on it in a few weeks with more results.
14
Mod Zone / Re: Custom cooler mount
« on: June 24, 2023, 01:48:26 pm »
Thanks!
The version tested here is quite compromised. I'd been trying to get it thin enough for AM5 mounts that the Noctua mount got neglected, then I made things worse by needing to manually drill out a counterbore, so there's about 2mm of thread here (which is AFAIK where a lot of the instability comes from). I threadlocked the screws (otherwise the Noctua mount unscrews them as it tightens) but it gave on the last turn on one side, so it's not tightened all the way. Aaaand, I'm using Carbonaut instead of thermal paste, which is extremely slippy (which I think adds to the movement).
I've attached an STL of the latest version, which I'm hoping is even 3D printable with the right plastic (though maybe only viable with the AM5 mount; I 3D printed the test versions for the dimensions and fit, but never powered up with one). As you can see it's much thicker, which meant a compromise for the water block (more of when I get to test it!).
The version tested here is quite compromised. I'd been trying to get it thin enough for AM5 mounts that the Noctua mount got neglected, then I made things worse by needing to manually drill out a counterbore, so there's about 2mm of thread here (which is AFAIK where a lot of the instability comes from). I threadlocked the screws (otherwise the Noctua mount unscrews them as it tightens) but it gave on the last turn on one side, so it's not tightened all the way. Aaaand, I'm using Carbonaut instead of thermal paste, which is extremely slippy (which I think adds to the movement).
I've attached an STL of the latest version, which I'm hoping is even 3D printable with the right plastic (though maybe only viable with the AM5 mount; I 3D printed the test versions for the dimensions and fit, but never powered up with one). As you can see it's much thicker, which meant a compromise for the water block (more of when I get to test it!).
15
Mod Zone / Custom cooler mount
« on: June 24, 2023, 07:29:37 am »
I thought I'd share some work-in-progress on this. It's not yet finished, and I'm waiting on the next iteration to be CNC'd.
After a lot of prompting from @Vikings, I designed a bracket that allows other coolers to be fitted (Noctua SecuFirm2 and AMD AM4/5). I'm testing here with an NH-U12A, and the temps are better than the stock cooler (and it's quieter!). This version shown is flawed, and I don't know if the changes will fix it: the Noctua 78mm mount is too flexible, so the cooler can move, and since there's no independent loading mechanism on the T2 this movement transfers to the CPU (which then crashes the system).
In a few weeks when I get the next version back I'll retry with the Noctua cooler, and I'll also compare with an EK AM5 water block.
(I'll share the design files when it's done, whether it works or not.)
After a lot of prompting from @Vikings, I designed a bracket that allows other coolers to be fitted (Noctua SecuFirm2 and AMD AM4/5). I'm testing here with an NH-U12A, and the temps are better than the stock cooler (and it's quieter!). This version shown is flawed, and I don't know if the changes will fix it: the Noctua 78mm mount is too flexible, so the cooler can move, and since there's no independent loading mechanism on the T2 this movement transfers to the CPU (which then crashes the system).
In a few weeks when I get the next version back I'll retry with the Noctua cooler, and I'll also compare with an EK AM5 water block.
(I'll share the design files when it's done, whether it works or not.)
