Messages

1

General Hardware Discussion / Re: Lekuo SouthBridge Max PB65MX3

« on: September 02, 2026, 02:24:09 pm »

I actually just put one of these in my Blackbird system, works fine on... whatever version of Ubuntu I have on it (22.04 or 24.04, I forget).  Very nice card for these systems which are really short on pcie slots.  I'm using the m.2 slots (booting from one) and the SATA, haven't tried the USB.

2

Applications and Porting / Re: [DEV] applying patches to old version of gcc

« on: September 12, 2025, 11:08:26 am »

It's unlikely anyone has tried to manually build this outside of the full openpower build environment in forever.  Grab the talos or blackbird openpower repository to see what it's doing (https://scm.raptorcs.com/scm/git/blackbird-op-build).  There are probably tons of extra build flags and maybe patches being applied, along with a version specific version of the compiler.  I'm not very familiar with buildroot but it seems like there's some stuff in openpower/package/ppc42-gcc

If you don't really absolutely need to build it in isolation from the rest of the system, I'd recommend sticking with the full build and modifying just the parts you care about inside of it.

Please excuse me if this is a patronizing or X-Y problem response!

3

Firmware / Re: Messing with WOF Tables

« on: February 01, 2025, 10:59:32 am »

I wouldn't use any of the other ones, but modifying the matching one would be easy enough.  Ultimately, they're just spreadsheets with a bunch of values.

I'm probably dropping mine back down to the defaults anyway.  The extra heat and power usage doesn't seem worth it, especially since the blackbird is so limited on memory bandwidth.

4

Talos II / Re: Trying to overclock, what is raptor-aggressive

« on: December 17, 2024, 10:34:08 pm »

I'm not sure 100%, but based on what I'm reading, I think "folding" is just disabling cores temporarily, and re-enabling them when more work needs to be done.  Normally linux likes to spread workloads across all the cores and shuffle them around, so hypothetically you could get power savings by leaving most of the cores constantly in the minimum running power state.  In practice I think this has little to no effect on Raptor machines.  Linux calls the lowest power state "snooze" which I would guess is the same as what IBM is calling "nap", but it's hard to tell for sure.

How much power is yours using?  My blackbird never reports CPU power usage of less than 30 watts (self reported, not measured with an external meter).  Considering these are seven year old server chips, I think that's pretty good.

Underclocking may be worth trying, but I don't think IBM has any published WOF tables running the CPU at less than 1.9 GHz.  I'm sure there's some real minimum due to hardware constraints, but maybe it's a little lower.

5

General CPU Discussion / Re: 3U HSF End of Life'd

« on: October 04, 2024, 08:35:16 am »

If anyone is desparate for heatsinks, I understand power8 heatsinks are physically compatible.  There are usually some on ebay in various heights.  Some part numbers may be 00RR234, 00RR232, 00E8918, 01KL499.  I think the 3u with fan was only used by Raptor, so there probably aren't many out there.  3D printing a fan shroud is probably not too hard.

Some news would be nice, but I'll admit I'm feeling a little pessimistic.

6

Third Party CPU Discussion / Re: Why haven't we seen boards with Power9 / MicroWatt?

« on: August 17, 2024, 10:21:40 am »

I think everyone interested in processor development went towards RISCV instead, which has way more community energy.  FPGA development is also very niche skill and it's way harder to do anything compared to pure software, so the general pool of people able to work on projects like microwatt is much smaller.

Open source projects like this only get done if a company decides to do them (and it seems like no companies think they could make money on it), or someone has the skills AND time AND interest (which is all a very large investment for an individual).

Maybe someone could develop a minimal carrier board for the Arctic Tern module, which does use microwatt.  It has a enough peripherals to use as a standalone linux system.  $600+ for that would be a tough sell given the limited performance, though.

7

General CPU Discussion / Re: HPLinpack benchmarking

« on: June 13, 2024, 08:03:45 pm »

After messing with the parameters a bit, I can get my 16 core blackbird up to 2.1683e+02, I wonder if it's limited by RAM bandwidth (only the two channels).

8

Firmware / Re: Firmware 2.10 for Talos-II and Blackbird available

« on: February 28, 2024, 11:03:17 am »

When I built the firmware (prior to this release) I used an ubuntu 18.04 VM.

9

Blackbird / Re: Blackbird Cooling

« on: August 05, 2023, 08:01:03 pm »

sorry to necro this thread, but where did you get those heatsinks from @cy384? I plan on trying a 16 core cpu in my blackbird and those heatsinks look like they'd work pretty well with my fan setup

I used this double sided thermal tape and these 6.5mm x 6.5mm heatsinks, but they're both just chinese generic parts, probably easy to find many different "brands" on ebay or amazon or whatever.  I used a flathead screwdriver to spread the tines of the heatsinks a bit.

I changed around my fan setup so they get more air flow, but the power area still gets really hot and I don't think the power/thermal management stuff is being run optimally.  Fortunately/unfortunately it works well enough that I don't want to spend more time messing with it!

10

Blackbird / Re: OpenBMC reporting critical error Proc.FSI.Error.MasterDetectionFailure

« on: May 04, 2023, 07:15:15 pm »

I got this error in my Blackbird's BMC log when there was a brief (like one second) power outage (brownout maybe?) last week.

11

Blackbird / Re: Blackbird Cooling

« on: October 20, 2022, 03:24:26 pm »

It looks like there are 7 of the voltage regulator chips, no idea how they're used/wired up though.

Reading about the fan control code and it seems kinda crude, very on/off in design, based more on events rather than target temperatures and dynamic control.  Under max load my CPU fan oscillates between high and medium speeds very noticeably.  I assume this is not an upstream concern, since nobody cares if a server is slightly louder, but it's unpleasant for a workstation.  Raptor spliced in some PID control but I don't think the tuning is ideal.

AFAICT the regulator temperature is considered for fan speed but only after it hits 85C.  Ref https://git.raptorcs.com/git/blackbird-openbmc/tree/meta-rcs/meta-blackbird/recipes-phosphor/fans/phosphor-fan-control-events-config-native/events.yaml

12

Blackbird / Re: Blackbird Cooling

« on: October 13, 2022, 07:01:38 pm »

Got these nice little copper heatsinks installed and did some quick temperature testing... not a huge difference, unfortunately!  Need to redirect more airflow over them.

Edit: also, the voltage regulators seem to be TDA21472, with a maximum recommended temperature of 125C and thermal shutdown at 140C.  I guess running them hot is only moderately concerning.

13

Blackbird / Re: Blackbird Cooling

« on: October 08, 2022, 06:09:00 pm »

just to share some numbers, here's what "sensors" reports at idle:

Code: [Select]

nvme-pci-0100
Adapter: PCI adapter
Composite:    +39.9°C  (low  = -273.1°C, high = +76.8°C)
                       (crit = +79.8°C)
Sensor 1:     +39.9°C  (low  = -273.1°C, high = +65261.8°C)

ibmpowernv-isa-0000
Adapter: ISA adapter
Chip 0 Vdd Remote Sense: 683.00 mV (lowest =  +0.67 V, highest =  +1.01 V)
Chip 0 Vdn Remote Sense: 674.00 mV (lowest =  +0.67 V, highest =  +0.67 V)
Chip 0 Vdd:              685.00 mV (lowest =  +0.68 V, highest =  +1.02 V)
Chip 0 Vdn:              675.00 mV (lowest =  +0.68 V, highest =  +0.68 V)
Chip 0 Core 0:            +44.0°C  (lowest = +22.0°C, highest = +65.0°C)
Chip 0 Core 4:            +44.0°C  (lowest = +22.0°C, highest = +65.0°C)
Chip 0 Core 8:            +44.0°C  (lowest = +23.0°C, highest = +65.0°C)
Chip 0 Core 12:           +44.0°C  (lowest = +23.0°C, highest = +65.0°C)
Chip 0 Core 16:           +45.0°C  (lowest = +21.0°C, highest = +65.0°C)
Chip 0 Core 20:           +45.0°C  (lowest = +23.0°C, highest = +67.0°C)
Chip 0 Core 24:           +45.0°C  (lowest = +23.0°C, highest = +66.0°C)
Chip 0 Core 28:           +44.0°C  (lowest = +24.0°C, highest = +68.0°C)
Chip 0 Core 32:           +44.0°C  (lowest = +22.0°C, highest = +67.0°C)
Chip 0 Core 36:           +44.0°C  (lowest = +22.0°C, highest = +67.0°C)
Chip 0 Core 40:           +44.0°C  (lowest = +22.0°C, highest = +66.0°C)
Chip 0 Core 44:           +44.0°C  (lowest = +22.0°C, highest = +67.0°C)
Chip 0 Core 48:           +45.0°C  (lowest = +23.0°C, highest = +66.0°C)
Chip 0 Core 52:           +45.0°C  (lowest = +23.0°C, highest = +67.0°C)
Chip 0 Core 56:           +45.0°C  (lowest = +23.0°C, highest = +68.0°C)
Chip 0 Core 60:           +45.0°C  (lowest = +24.0°C, highest = +68.0°C)
Chip 0 DIMM 0 :           +49.0°C  (lowest = +30.0°C, highest = +51.0°C)
Chip 0 DIMM 1 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 2 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 3 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 4 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 5 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 6 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 7 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 8 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 9 :            +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 10 :           +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 11 :           +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 12 :          +50.0°C  (lowest = +30.0°C, highest = +53.0°C)
Chip 0 DIMM 13 :           +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 14 :           +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 DIMM 15 :           +0.0°C  (lowest =  +0.0°C, highest =  +0.0°C)
Chip 0 Nest:              +45.0°C  (lowest = +23.0°C, highest = +63.0°C)
Chip 0 VRM VDD:           +53.0°C  (lowest = +35.0°C, highest = +90.0°C)
Chip 0 :                  32.00 W  (lowest =  28.00 W, highest = 156.00 W)
Chip 0 Vdd:                4.00 W  (lowest =   2.00 W, highest = 127.00 W)
Chip 0 Vdn:                9.00 W  (lowest =   7.00 W, highest =  11.00 W)
Chip 0 :                 326.27 kJ
Chip 0 Vdd:               47.31 kJ
Chip 0 Vdn:               89.58 kJ
Chip 0 Vdd:                6.38 A  (lowest =  +4.00 A, highest = +129.75 A)
Chip 0 Vdn:               14.38 A  (lowest = +11.50 A, highest = +17.38 A)

and I've attached an image of what the openbmc web interface reports.

Now that I look at it again, the "Temperature Pcie" might just be the NVMe drive.

14

Blackbird / Blackbird Cooling

« on: October 08, 2022, 01:35:46 pm »

I've been struggling a bit to figure out how to best cool my Blackbird.  Possibly this is made worse by my choice of case (smallest mATX I could find, a SAMA IM01) and the CPU (160W 16 core).  I am using the 3U heatsink module with the provided fan.  Main complaints:

* CPU fan is oriented up/down, and blows downward, like an inch away from whatever's in the first PCIe slot, so I flipped the fan around to blow upwards, which means hot air going over the voltage regulators.
* No heatsink on the voltage regulators, while the Talos and Talos Lite both have them.  The BB doesn't have holes to mount a heatsink there, either.
* I'm not sure where various heat sensors are, physically, on the board.  Where's the ambient temperature sensor?  The PCIe sensor?  The CPU ambient sensor?
* RAM slots are as close as physically possible to the CPU (good for signal integrity, bad for airflow).
* Voltage regulator temps do not seem to be considered in setting fan speed.
* Changing the cooling parameters requires recompiling firmware.

I don't really care about any of these except that the voltage regulators hit 90C within a minute under heavy load.  I've ordered some tiny little heatsinks that can be stuck directly to the chips but I'm wondering if anyone else has a nice solution here.  They're really low on the board and in an awkward spot.

I do have a 3D printer and will design some ducts/shrouds if I can't get temps low enough otherwise.

Attaching some pics for the curious.

15

Firmware / Re: Messing with WOF Tables

« on: October 07, 2022, 02:53:44 pm »

Can you read out the processor frequencies and tell if it is boosting properly?    Do you see ~160W when at idle?   What about when running max threads off some heavy workload (e.g. Mersenne primes is a good one)?

I haven't looked closely at the frequencies, it idles around 30W and only approaches 160W under heavy load.