Author Topic: POWER9 water cooling kit  (Read 3723 times)

pocock

  • Full Member
  • ***
  • Posts: 236
  • Karma: +22/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #15 on: October 08, 2020, 03:44:06 pm »

At risk of going off-topic, I'd like to emphasize that water-cooling is only 50% of the problem when house-training the POWER9

The thread about powering-down unused cores is equally important

The CPU cores, especially in a dual-CPU configuration, using over 100W when idle, will waste electricity and generate unwanted heat.

Electric heating is particularly inefficient, 60-70% of electricity is lost in the energy network.  The only exception to this is the use of a local generator (renewable, cogeneration, etc), as it is generated locally, very little is lost in transmission lines.

In my own case, I'm using solar PV and the excess heat is therefore useful and efficient for 7-8 months of the year, it simply offsets the underfloor heating.

But for most people this could be expensive, wasteful and uncomfortable.

Solving both of these issues together (the water cooling and making some cores sleep when idle) will eliminate most concerns from buyers.
Debian Developer
https://danielpocock.com

MPC7500

  • Sr. Member
  • ****
  • Posts: 328
  • Karma: +18/-1
    • View Profile
    • Twitter
Re: POWER9 water cooling kit
« Reply #16 on: November 19, 2020, 08:26:00 am »
The lack of STR is a big one, IMO.

vikings.thum

  • Newbie
  • *
  • Posts: 13
  • Karma: +5/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #17 on: June 22, 2021, 10:18:28 am »
Hello there,

We were finally able to give this a quick run today and prepare some comparisons :)

Specs: Talos 2 with 1x 02CY296 (22-core, 88 threads)
Setup 1: HSF, 3U IBM HSF
Setup 2: Water cooling system heatsink, 120mm / 4.7″ radiator, 120mm / 4,7″ PWM fan, Laing DC pump, compression fittings and TPV (EPDM/PP) tubes.
Setup 3: Identical to setup 2, with MX5 thermal paste
Stressing the CPU: $ sudo sysbench --test=cpu --cpu-max-prime=990000000 --threads=90 run

We stressed each setup for 15 minutes / 9e+11 nanoseconds, both fans (HSF and the fan attached to the radiator) are PWM fans, controlled by Talos 2.

Findings:
Compared to setup 1, setup 2 showed similar core temperatures, though quieter. Which makes sense since this mechanism can't deliver the high pressure of 90 kg (200 lbs) the IBM HSF can.
Setup 3 showed the best results which were roughly 10 °C / 50 °F below the present temperatures at the 15 minute / / 9e+11 nanoseconds mark and roughly 20 °C / 68 °F below the "highest" temperature of setup 1 and 2. Quite honestly this is as positive surprise.

I'd expect even better results with a larger 240mm / 9.45″ radiator.

Unfortunately, the upper plate of the mechanism had warped during the first attempts (which ist a quality issue and shouldn't have happened) and the mechanism as a whole is too tight against the compression fittings (perhaps that's not an issue with angled fittings). It's easy to overtighten the screw of the mechanism because it's unclear how much force is necessary to hold the heat sink in place and provide enough pressure for a sufficient cooling result. 
All in all this seems to be very usable in principle, but there are some improvements to be made before we're happy enough with the quality and if we want to use standard components (esp. a standard water cooled heat sink) that doesn't break the bank.








From left to right: watercooled with thermal paste / watercooled without thermal paste / IBM HSF
« Last Edit: June 22, 2021, 10:28:19 am by vikings.thum »

ClassicHasClass

  • Full Member
  • ***
  • Posts: 211
  • Karma: +13/-0
  • Talospace Earth Orbit
    • View Profile
    • Floodgap
Re: POWER9 water cooling kit
« Reply #18 on: June 22, 2021, 12:56:29 pm »
Nice temperature reduction! I'm pretty confident all those problems are solvable.

ClassicHasClass

  • Full Member
  • ***
  • Posts: 211
  • Karma: +13/-0
  • Talospace Earth Orbit
    • View Profile
    • Floodgap
Re: POWER9 water cooling kit
« Reply #19 on: June 22, 2021, 01:07:17 pm »
Also, what fluid did you use?

vikings.thum

  • Newbie
  • *
  • Posts: 13
  • Karma: +5/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #20 on: June 22, 2021, 02:45:14 pm »
Nice temperature reduction! I'm pretty confident all those problems are solvable.
I'm pretty confident as well, the manufacturer has been helpful so far.

Also, what fluid did you use?
We use DM or osmosis water (better avoid distilled water from the supermarket, it's often a bit on the acidic side) and a glycol without additives somewhere around a ratio of 1:20 (or even less), I've never ran into problems with that but this also depends on what tubing you use (PVC tubing can be problematic with very high glycol contents). There are also ready-to-use solutions available for PC watercooling enthusiasts, AFAIK most of those also use glycol/water but are more expensive, some contain glutaral (even though very little) which I would be a bit scared of to be honest. Correct me if I'm wrong :)

Cooling solutions that contains glycol must be disposed of properly. In Germany for example municipalities accept it free of charge as hazardous waste in small quantities.