Messages

1

Mod Zone / Re: Custom cooler mount

« on: September 30, 2024, 04:20:05 am »

It's complicated...! I initially designed for any AM5, but as I started testing, and having put a cooler mounting screw into my motherboard (because they're too long) I changed the design.

For the first change I made a thicker bracket, which fixed the screws going into the mobo but needed a copper shim (a 'hat') for the CPU. I tested this for a while but was never happy with it: the design worked but the copper part was difficult to get produced to the required accuracy, so wasn't anything anyone else was going to get made (it took a lot of attempts, and quotes from some machining companies were thousands for a one-off).

Eventually I opted for custom mounting posts for the cooler, adopting the AM5 bolt pattern but replacing the cooler's fittings completely, and since I no longer used the cooler's own screws I changed the pitch for something more appropriate, losing compatibility with any AM5 cooler.

Short version: it works with any SecuFirm2 Noctua air cooler, as long as the cold plate is large enough, tested with a U12S (which works perfectly); for water cooling an EK-Pro AM5 water block fits, anything else hasn't been tested. Both these solutions have large cold plates that correctly cover the POWER9.

A lot of text... all that said, I have a pile of spares and test parts, most of which will fit any AM5 cooler, along with the custom copper part to adapt any cold plate. PM me, I've been offering to send these to anyone who asks, they're only getting recycled otherwise.

2

Mod Zone / Re: Custom cooler mount

« on: September 29, 2024, 03:26:51 pm »

For mounting a Noctua air cooler, a nylon or carbon fibre print is probably good enough. I had a company we were trialing a laser sintering printer with at work do me some samples, but I never tried them (the prints were okay, a Bambu Lab for 1/20th of the cost was just as good though).

For an AM5 water cooler you need aluminium with the stainless steel mounting parts.

I’d add photos but the upload is broken.

3

Mod Zone / Re: Custom cooler mount

« on: September 28, 2024, 06:30:13 am »

"What material should I get it done with?"

I used bead-blasted anodised aluminium 7075 for the large cooler mount, then stainless steel 304 for the mounting posts and spring retention parts.

For the aluminium part the instructions were: "Tolerances should be ISO 2768 fine. Threads are required and are marked on the drawing (take note of the 3.5x0.9mm counterbore in the M3.5 threads, which has been added to the model). Threads are all the way through. The 'Text' layer in the DWG contains the laser engraving. Racking preference is either inside the unthreaded holes or on the opposite side of the laser engraving (with all of the round features and steps, not the flat side). The finish should be clean and smooth."

I should really write up the instructions, and set-up a click-to-buy option on PCBWay or something, otherwise you're at the whim of the machine shop. Give me a few days, I'll look into this.

I recently finished the second part of this project, custom brackets for the radiator, pump and reservoir to fit into the smallest space possible (I'd add photos but the upload still doesn't work).

4

GPU Compute / Accelerators / Re: Radeon RDNA3 support?

« on: September 25, 2024, 04:36:25 pm »

If anyone near Basel wants to give it a try, we’ve a selection of W7x00 Pro cards at work, I just don’t have a full running P9 machine right now.

5

Mod Zone / Re: Custom cooler mount

« on: September 25, 2024, 04:30:28 pm »

For some reason I didn’t get the forum updates, hmm.

Anyway, it’s probably a Git LFS issue and the downloaded files are only the text files with the URL and metadata. It’ll need a one-time LFS init of the repo.

The files to send for manufacture are the IGES files in the out directory.

6

Mod Zone / Re: Custom cooler mount

« on: October 15, 2023, 06:57:06 am »

I haven’t seen that but I definitely will try. A quick look at the source and I see it’s handling thread affinity, a requirement for Threadrippers on Windows.

7

Mod Zone / Re: Custom cooler mount

« on: October 14, 2023, 10:53:27 am »

Thanks for the info, I know little about how the IBM WOF works.

The workload I'm testing with is from a graphics tool I've been working on for a while, which is interesting because I'd tuned the threading for 64C/128T Threadrippers and then started using it to compare other systems (and it's a nice workload for many cores). Send me a PM if you're interested in building it from source, I'm not ready to announce it yet so it's under wraps (GCC/Clang and CMake on most systems build out of the box).

8

Mod Zone / Re: Custom cooler mount

« on: October 05, 2023, 09:47:52 am »

Yes, I have the dual 18 core, and with the water cooling temps are around 40 degrees (same as the simpler solution from Vikings) and the machine is running silently (a quiet machine was the original aim). The same tests with a Noctua air cooler the temps are mid-50 and the fans are quiet (compared with the IBM HSF, anything is quiet).

We water cool racks of machines at work, with the big benefit being keeping the temps down keeps the clocks high and the power consumption down. With Threadrippers, for example, they run at boost clocks all the time, vs down-clocking to half the speed to stay within TDP, effectively doubling the throughput. Same with GPUs (which is where we do most of the water cooling, CPUs was just a curiosity).

Real life took over for a while but I will work on a jig to test the mounting pressure.

9

Mod Zone / Re: Custom cooler mount

« on: September 12, 2023, 05:06:04 am »

I don't, but when comparing the exact same test with the IBM 3U HSF the temperatures were around 70 degrees.

I have a replacement mobo now (RCS were really helpful here) and I'm waiting on some CNC'd parts back with amends to make installing this easier. The design files are here for anyone interested:

    https://github.com/cwoffenden/talosmods/tree/main/am5-sf2-cooler

(I'll be updating this and adding docs along the way)

10

Mod Zone / Re: Custom cooler mount

« on: August 23, 2023, 10:12:13 am »

Do you have any idea how much force you are applying down through the module to the socket pins?

I'm going to build a jig to measure this, for both the Noctua and EK options, then I can tailor the springs to make sure the force is similar to the IBM part. It'll keep me busy waiting for my replacement mobo to be delivered.

11

Mod Zone / Re: Custom cooler mount

« on: August 21, 2023, 07:49:06 am »

I found the problem with my second CPU, I appear to have dropped something onto the socket and bent a bunch of pins:

    https://wip.numfum.com/cw/2023-08-21/cpu2-cu.jpeg

I have a replacement mobo coming from Raptor so will get back on with this in a few weeks.

12

Mod Zone / Re: Custom cooler mount

« on: August 17, 2023, 11:45:30 am »

I've no idea of the amount of force. For the design I went with the same mounted hight (via trial and error and shims) as the Noctua cooler and the EK water block on an AM5 mobo.

What I found was, for the Noctua, until I got this height and the stiffness right then the system wasn't stable. Once I got this correct (measured at the spring hight, how far they were tensioned, how the cooler sits on the CPU including getting its eccentricity right) the system was a stable and I could wobble and shake the cooler. After the Noctua's SecuFirm2 mounts I did the same for the EK Pro water block. AM5 is a 1781 pin LGA, the POWER9 Sforza is a 2601 pin LGA, and have 40x40 vs 48x48 IHS sizes (but the coolers usually have 50x50 cold plates), so they're not like for like.

The socket I used for mounting and testing the various coolers had maybe 20 mount and test cycles, trying to keep the number down (since I'd read the socket was good for very few cycles). The second CPU socket, used for all the measuring and verification, seems to have been trashed in the process (I must have eventually squashed the pins, I'll need to get my SMT stereo microscope unpacked to look). The lack of independent loading mechanism (like on the AM5) appears to have been the downfall.

For stability testing I ran this machine hard for multi-day cycles (one of my own tools that 100% loads the CPU cores with lots of maths).

Since I trashed my second CPU socket I've not been able to test both CPUs yet, I need to solve that next.

13

Mod Zone / Re: Custom cooler mount

« on: August 14, 2023, 03:03:34 am »

With such good contact with the CPU, for regular use the radiator fans aren't even running, so it's just the sound of the pump.

14

Mod Zone / Re: Custom cooler mount

« on: August 12, 2023, 11:38:24 am »

Well I did what I said I wouldn't do and went all-in with custom mounts for the EK cooler to eliminate the copper heat spreader/shim:

    https://wip.numfum.com/cw/2023-08-12/custom-mounts.jpeg

These attach to the earlier bracket and allow the AM5 water block to mount directly on the POWER9, all properly loaded and torqued:

    https://wip.numfum.com/cw/2023-08-12/ek-am5-mounted.jpeg

The temps are amazing:

    https://wip.numfum.com/cw/2023-08-12/bc2etc-gen-waku-ek-am5-custom.png

This is running flat-out and it's around 40 degrees. I'm using a graphite pad instead of paste, so it's not entirely comparable to the first tests I did with the Alphacool bracket from Vikings, and at a guess it would go even lower with the Thermal Grizzly paste (but I just couldn't bring myself to clean off any more paste!). I'll say I'm 100% happy with the result and will re-add the second CPU. The pad will never dry out and should be good for life.

I'm going to document and write this all up in the next weeks, plus make the CAD files available. What I will say, though, is if you want a quick and easy way to water cool, the option from Vikings is probably for you. My solution gets a few more degrees for a much greater cost (especially since this was all one-off custom machining, although with a group buy the price would come way down).

I do think the Noctua fan is a good solution for most people wanting a quieter T2 or Blackbird. Vikings Thum will hopefully publish his findings with a 3D printable version of the same bracket.

15

Mod Zone / Re: Custom cooler mount

« on: July 14, 2023, 10:15:41 am »

Today I tested the mount with an EK AM5 water block. This needed a compromise in the design (which I could work around with custom threaded fittings, but each cooler/water block would need its own design, whereas what I have here works with any AM5 cooler) adding a 4mm copper heat spreader on top of the CPU:

    https://wip.numfum.com/cw/2023-07-14/am5-copper-hs.jpeg

It's a smooth machined lump of copper with a pocket to grip the top of the CPU. I used a Thermal Grizzly Kryosheet pad instead of thermal paste (which was frustrating to cut to size), which I used to aid the potential repeatedly dismantling. I mounted the water block on that:

    https://wip.numfum.com/cw/2023-07-14/am5-ekwb-1.jpeg
    https://wip.numfum.com/cw/2023-07-14/am5-ekwb-2.jpeg

The whole thing is designed so the screws go all the way in (hand tight) to get the maximum out of the springs. Tightening was done in a cross pattern, a few turns at a time. Then the same Alphacool radiator hooked up I use for tests:

    https://wip.numfum.com/cw/2023-07-14/am5-with-rad.jpeg

The mount is solid, as it is with the Noctua fan, and can be pushed and prodded with no play. I ran the same tests as I did with the Noctua and earlier Alphacool water block, with these numbers:

    https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-waku-ek-am5.png

The temperatures are higher than the with the Alphacool:

    https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-waku-acool.png

My solution runs nearly 10 degrees hotter, which is probably a combination of thermal pads vs paste (I used Kryonaut Extreme as the paste, at around $95 USD per pot over here, which we use at work for GPU water cooling) plus the compromise of that lump of copper. But a maximum of 57 degrees is cool running!

I've decided against custom fittings and will keep the compromise of copper heat spreader, but I will switch over to thermal paste and re-run the tests next week.

It's quiet, with the fans stalling and shutting off, and with temps better than the Noctua fan:

    https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-noctua.png

And certainly better than the IBM HSF:

    https://wip.numfum.com/cw/2023-07-14/bc2etc-gen-hsf.png

More tests next week!