Author Topic: POWER9 water cooling kit  (Read 19586 times)

pocock

  • Sr. Member
  • ****
  • Posts: 296
  • Karma: +32/-0
    • View Profile
POWER9 water cooling kit
« on: May 08, 2020, 03:06:30 am »

This article has a photo of a POWER9 with a water cooling kit

https://www.forbes.com/sites/tiriasresearch/2017/12/08/why-ibm-launched-instead-of-power9/

Water cooling could be a useful solution for people who are trying to minimize noise levels.

It can also be useful for moving heat further away from the place where equipment is used, if sufficient pipework can be installed.
Debian Developer
https://danielpocock.com

MPC7500

  • Hero Member
  • *****
  • Posts: 587
  • Karma: +41/-1
    • View Profile
    • Twitter
Re: POWER9 water cooling kit
« Reply #1 on: May 08, 2020, 07:42:09 am »

surf

  • Newbie
  • *
  • Posts: 25
  • Karma: +1/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #2 on: May 08, 2020, 04:05:28 pm »
Thanks for the link.  I dug through the junk on there and found the cheese:



That's neat!  I wonder where/how we could get those water blocks shown?


vikings.thum

  • Newbie
  • *
  • Posts: 43
  • Karma: +17/-0
    • View Profile
    • Vikings
Re: POWER9 water cooling kit
« Reply #3 on: August 10, 2020, 08:39:19 am »
Vikings is working on a silent water cooling solution for POWER systems. We should have something presentable in ~one to ~two month(s) from now.
https://shop.vikings.net
XMPP: thum@jabber.vikings.net
Libera.Chat IRC: #vikings (handle: 'thum')

ClassicHasClass

  • Sr. Member
  • ****
  • Posts: 467
  • Karma: +35/-0
  • Talospace Earth Orbit
    • View Profile
    • Floodgap
Re: POWER9 water cooling kit
« Reply #4 on: August 10, 2020, 11:15:46 am »
Outstanding!

q66

  • Guest
Re: POWER9 water cooling kit
« Reply #5 on: August 10, 2020, 01:44:16 pm »
nice, but a 4U air cooler with 120mm fan(s) would likely be both easier to design and more useful for most users...

MPC7500

  • Hero Member
  • *****
  • Posts: 587
  • Karma: +41/-1
    • View Profile
    • Twitter
Re: POWER9 water cooling kit
« Reply #6 on: August 10, 2020, 05:46:05 pm »
This is really, really, really great. Is it a complete new development or an adapter for existing cooling solutions for e.g. LGA-2066 water cooling sets?

Anyway, I really appreciate your work.

I am looking forward to your OpenPOWER store.

pocock

  • Sr. Member
  • ****
  • Posts: 296
  • Karma: +32/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #7 on: August 10, 2020, 05:49:34 pm »
Instead of using radiator fans on the water cooling circuit, are there any external chiller or heat pump options that could reduce overall heat emission, even if they don't reduce net noise emissions a lot?  This type of option requires a way to dump the heat, much the same way a split-system air conditioner moves heat to the outdoor component.
Debian Developer
https://danielpocock.com

vikings.thum

  • Newbie
  • *
  • Posts: 43
  • Karma: +17/-0
    • View Profile
    • Vikings
Re: POWER9 water cooling kit
« Reply #8 on: September 07, 2020, 07:03:31 am »
Here's a draft for a POWER9 water cooler. The base plate is made of nickel-plated copper.
https://shop.vikings.net
XMPP: thum@jabber.vikings.net
Libera.Chat IRC: #vikings (handle: 'thum')

pocock

  • Sr. Member
  • ****
  • Posts: 296
  • Karma: +32/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #9 on: September 07, 2020, 08:11:41 am »
Looks really interesting.

Will the connections rotate?  It could be helpful for getting the best pipe routing.

I had been looking for companies that do this, I found this company, Optimus Water Cooling are close to Raptor in Chicago.

Then there is Ice Giant in Texas, maybe not far from the location where the motherboards are manufactured.
Debian Developer
https://danielpocock.com

vikings.thum

  • Newbie
  • *
  • Posts: 43
  • Karma: +17/-0
    • View Profile
    • Vikings
Re: POWER9 water cooling kit
« Reply #10 on: September 07, 2020, 08:56:22 am »
Looks really interesting.

Will the connections rotate?  It could be helpful for getting the best pipe routing.

Generally that's a feature of the fitting installed, not so much the cooler. Since standard dimensions are used, the cooler would be interoperable with other standard cooling systems. The top fittings in the draft are an example of rotating fittings.
We're also checking if this would work for 1U machines - would be a huge advantage for anyone looking for hosting these machines in a data center where rack space is expensive.

I had been looking for companies that do this, I found this company, Optimus Water Cooling are close to Raptor in Chicago.
Then there is Ice Giant in Texas, maybe not far from the location where the motherboards are manufactured.

While it is generally interesting to check all options available, we're based in Germany so it makes more sense to look for a partner on this side of the pond, also keeping indirect costs in check and potential legal issues manageable.
https://shop.vikings.net
XMPP: thum@jabber.vikings.net
Libera.Chat IRC: #vikings (handle: 'thum')

pocock

  • Sr. Member
  • ****
  • Posts: 296
  • Karma: +32/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #11 on: September 07, 2020, 10:55:45 am »

I use a lot of German products without any hesitation and given my location, I'd probably try your product first.

Some of the US users may prefer a supplier local to them, it could also be good publicity for the platform if some of these other vendors list Raptor and POWER9 on their web sites.

Personally, I've found that the dual POWER9 system is so fast that my projects are finished compiling in less than 30 seconds.  In that time period, the CPU doesn't get hot enough for the fans to make a crazy noise.

That said, as I put more jobs on the workstation, I suspect I'll need the watercooling.

I went looking for more adventurous solutions too, for example, external chillers and heat exchangers.  This could be interesting for people who can dump the heat into a thermal store.  For example, you can basically rack a couple of Talos II servers in your basement and 90% of the electricity will heat water for your shower.

https://koolance.com/cooling-systems-liquid-to-liquid

https://www.performance-pcs.com/water-cooling/water-chillers/koolance-exc-450-ultra-compact-450w-recirculating-liquid-chiller-exc-450.html

Debian Developer
https://danielpocock.com

vikings.thum

  • Newbie
  • *
  • Posts: 43
  • Karma: +17/-0
    • View Profile
    • Vikings
Re: POWER9 water cooling kit
« Reply #12 on: October 07, 2020, 04:17:03 am »
Some of the US users may prefer a supplier local to them, it could also be good publicity for the platform if some of these other vendors list Raptor and POWER9 on their web sites.

We can wholesale this system to an U.S. location on request. Not sure if RaptorCS would be interested.
We really chose the low-cost route here to check out demand. We're using an existing high-quality cooler (with different connectors where needed) with a custom-made high-pressure mount. If there is a market big enough, we'll invest in making a completely new cooler for 1U solutions (this only would fit 2U).

Personally, I've found that the dual POWER9 system is so fast that my projects are finished compiling in less than 30 seconds.  In that time period, the CPU doesn't get hot enough for the fans to make a crazy noise.

There are use-cases that require a more quiet setup, of course if you don't need it like with your use-case there's no need to spend extra $ for a water cooler.
I found the IBM HSFs quiet enough when idle as well, for everything else, like office or living room situations, those HSFs are unusable IMO.
The reason for using water cooling for POWER in a data center, at least for us, is space and using a water/glycol heat exchanger for heating nearby offices, especially with energy being very expensive in our area.

I went looking for more adventurous solutions too, for example, external chillers and heat exchangers.  This could be interesting for people who can dump the heat into a thermal store.  For example, you can basically rack a couple of Talos II servers in your basement and 90% of the electricity will heat water for your shower.

This could be a fun side project indeed! In order for this to be economically feasible I'd think you'd have to have more than one computer though :)
https://shop.vikings.net
XMPP: thum@jabber.vikings.net
Libera.Chat IRC: #vikings (handle: 'thum')

pocock

  • Sr. Member
  • ****
  • Posts: 296
  • Karma: +32/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #13 on: October 07, 2020, 08:47:43 am »
I found the IBM HSFs quiet enough when idle as well, for everything else, like office or living room situations, those HSFs are unusable IMO.

I agree a lot of people will have trouble with the HSFs

Another thing that comes to my mind is the performance of the case fans: if somebody has some very large and quiet case fans, e.g. 140mm and they move a lot of air across the HSF will the motherboard fan logic be able to reduce the HSF speeds?

Looking at the Noctua fans, for example, they are offering fans with different maximum speeds, for example, 800, 1200, 1500, I even saw one with 3000.  They will be noisy as well but will the overall noise be lower?
Debian Developer
https://danielpocock.com

helen

  • Newbie
  • *
  • Posts: 6
  • Karma: +1/-0
    • View Profile
Re: POWER9 water cooling kit
« Reply #14 on: October 07, 2020, 07:04:28 pm »
...
We're using an existing high-quality cooler (with different connectors where needed) with a custom-made high-pressure mount.
...

Hi, can I by one of these blocks with a custom mount  now?

Do you ship to Norway?

I am more or less going "all out" with this build. And custom water cooling loops is on the list of what I want to do :-)