Topics

61

Talos II / Using the μPCIe connector (J10108)

« on: June 17, 2020, 06:04:47 am »

The manual:
https://wiki.raptorcs.com/w/images/e/e3/T2P9D01_users_guide_version_1_0.pdf

mentions a μPCIe connector (J10108)

How many lanes does it support?

Can anybody give any example of products that can be connected with this?  For example, can it be used with an M.2 or U.2 adapter for SSDs?

62

GPU Compute / Accelerators / nVidia official proprietary and open source code for POWER

« on: June 16, 2020, 11:09:25 am »

These are some of the links that appear most relevant:

NVIDIA forum, describes support for P100 on POWER8 and V100 on POWER9.

NVIDIA media release about collaboration with IBM on open source machine learning with Tesla V100

IBM pages about the same topic

The V100 products are rather expensive.

Nonetheless, some people have suggested that bits of the proprietary driver can be used to run other NVIDIA GPUs in a POWER system.  This thread might be a useful place to collect comments on topics like that.

63

Applications and Porting / Debian packaging and backport requests for OpenPOWER, Talos II, Blackbird?

« on: June 10, 2020, 09:58:22 am »

Are there any packages that don't exist at all in Debian but would be essential or highly desirable on Debian systems?

Are there packages in unstable that are either not in the stable release or they require something from a newer version to be backported to stable to make them work on ppc64el?

Maybe these could be entered as RFP or Request-for-backport bugs in bugs.debian.org and then a usertag can be added to all of those so they can be found quickly.  This is an example of how arm64 bugs and package requests are tagged.

I'm willing to build packages or backports of the most important things that I will use myself but it would be helpful if people can point them out or enter any hints or concerns through the BTS.

64

General CPU Discussion / Practical hardware and operating systems on Microwatt?

« on: June 09, 2020, 09:15:06 am »

Search engines return a lot of publicity about Microwatt but I couldn't see any practical examples of hardware and operating systems

Which FPGAs are people aiming for or testing right now?

Is it at the stage where somebody could build something vaguely similar to a Raspberry Pi, even if it lacks some of the ports?

Is it stable enough to run any GNU/Linux or BSD OS right now?

65

GPU Compute / Accelerators / Radeon Pro vs regular Radeon cards - worth the price premium for POWER users?

« on: June 05, 2020, 01:05:50 pm »

The RX 5700 came out last year and it was followed 6 months later by the Pro version, specifically the Radeon Pro W5700.

The W5700 is basically double the price of the RX 5700.  From the perspective of a POWER user, is this worthwhile?

Summarizing some of the key differences in the Pro version:

- AMD is testing the hardware and drivers more thoroughly: but do they test on any POWER9 systems?

- AMD is releasing driver updates for the Pro cards on a regular schedule: do these bug fixes appear in the amdgpu release for Linux users just as quickly?

- the marketing material describes various features, such as the AMD Remote Workstation (use your GPU remotely from a laptop) but is that relevant for a Linux user?  The software they offer is proprietary, so there are a large percentage of people in this space who would not use it anyway and we also have free software alternatives

- the last significant benefit I could see: the overall design is less aggressive, slightly less power consumption and lower clockrates than other cards so even ignoring the questions about drivers, maybe it will last longer and be more stable

- some people justified the purchase of Radeon Pro products when they included ECC RAM but in the W5700, it is not ECC, it is the same as the RX 5700

I've got an open mind about this: for example, an OEM built RX 5700 that has liquid cooling and isn't overclocked may be more relevant to some people than the W5700.  But if AMD is regularly testing amdgpu with W5700 on POWER9 then that alone would make me feel they are investing in this architecture.

66

Applications and Porting / VP9 benchmarks: have they improved?

« on: June 05, 2020, 12:12:38 pm »

One of the top search results for VP9 benchmarks is this OpenBenchmarking.org site where POWER9 is the slowest with 9.37 frames per second (less than real time).  Even the Intel i3 achieves 28.38 fps (better than real time).

Has this code been fixed and does anybody know how to get fresh results in that site?  I think it is very unfair to the platform when search engines show something like that if it is no longer valid.

On the other hand, if it is not fixed, has anybody proposed a bounty for working on it?  IBM is asking people to suggest issues that they will fund

67

Talos II / CPU hotplug / completely power down one CPU in a multi-CPU system

« on: June 05, 2020, 05:42:47 am »

Linux provides a mechanism to disable individual cores.  This can be useful to reduce peak power consumption or to simulate a smaller environment, for example, if a developer with a Talos II wants to know how their application would perform on a Blackbird with a 4-core CPU, they can turn off all but 4 cores.

echo 0 | sudo tee /sys/devices/system/cpu/cpu1/online

Spoiler: If you put all the cores of one CPU offline with that command then you won't be able to access the RAM and PCI slots connected to that CPU and you might observe strange behaviour.

Is it possible to go one step further and completely power down a CPU socket and maybe the associated RAM banks too, almost as if they were removed from the board?

There is some documentation about Linux kernel hotplug and it suggests x86 only.  Maybe this would be good for another bounty but first it is important to understand whether the Raptor and POWER9 hardware supports this and whether it would lead to energy savings or other benefits.

Problems that would be solved with this:

- reducing heat output from Talos II workstations during summer heatwaves

- extending runtime for a system on UPS batteries

68

Applications and Porting / gstreamer performance, lessons from ffmpeg efforts?

« on: June 02, 2020, 03:01:38 pm »

A bounty recently provided fixes for ffmpeg

Are similar fixes needed for gstreamer or any of its dependencies?

Could lessons from the ffmpeg porting be applied to any other similar multimedia frameworks, for example, gstreamer, the WebRTC media stacks in Firefox and Chrome?

69

General CPU Discussion / equivalent to a HP Z cooler?

« on: June 01, 2020, 04:29:53 pm »

Is there any possibility of somebody marketing a fanless POWER9 cooler similar to the HP Z coolers?

Maybe this would not be feasible for the 22 core chips but it would be very interesting for 4-core.  Not only is it silent but it also eliminates one possible type of mechanical failure.

70

Blackbird / power consumption figures?

« on: June 01, 2020, 04:25:21 pm »

The wiki has some rough power figures for Talos II

Could anybody provide similar figures for basic Blackbird builds, with bare minimum spec, no GPU?

E.g.
Blackbird + 32GB + basic SSD + CPU (4 core v2 idle) = ? Watts
Blackbird + 32GB + basic SSD + CPU (4 core v2 100% utilization) = ? Watts
Blackbird + 32GB + basic SSD + CPU (8 core v2 idle) = ? Watts
Blackbird + 32GB + basic SSD + CPU (8 core v2 100% utilization) = ? Watts

This type of detail is really useful for anybody trying to estimate PSU requirements, runtime on a UPS and heat output of the box

In comparison, HP provides a couple of example builds for each of their Z workstations in quickspecs documents like this (see page 26).  Most people can extrapolate from one of the examples to their own use case.

71

General Discussion / LSI HBAs on ordering page - blobs?

« on: June 01, 2020, 04:03:11 pm »

The Raptor ordering page includes some LSI HBAs for SATA / SAS:

• LSI 9300-8i 8-port Internal SAS 3.0 HBA
• LSI 9300-8e 8-port External SAS 3.0 HBA
• MegaRAID 9361 8-port Internal SAS HBA w/ BBU
• MegaRAID 9361 16-port Internal SAS HBA w/ BBU

The non-RAID cards have a PowerPC 476 running firmware from LSI.  The RAID cards have some kind of RAID SoC, I haven't looked at the details but it also includes firmware.

Is there any particular reason these are recommended in the ordering process or is source code available now?

Do these cards allow people to replace the firmware or do they expect cryptographically signed blobs from the vendor?

72

Talos II / OCuLink ambiguity

« on: June 01, 2020, 02:05:08 am »

When I look at the ordering pages on the Raptor site, it offers various OCuLink products:

• 4x OCuLink PCIe Expander (Gen 3)
• OCuLink to U.2 Cable

The manual:
https://wiki.raptorcs.com/w/images/e/e3/T2P9D01_users_guide_version_1_0.pdf

includes errata stating that

ER0001
μPCIe connector (J10108) is incorrectly identified as OCuLink (SFF-8621) due to errors that were discovered post-production in IBM OpenPOWER reference designs.  While the physical connector is the same as OCuLink ports, the wiring is different, notably in the presence detect / clocking pins.  As such, while an OCuLink peripheral is unlikely to be damaged by being plugged in to the μPCIe port, it will not function while attached.


If that is the case, buyers selecting the OCuLink products will not be able to use them - or has the board design been changed somehow to make OCuLink work?

73

General Discussion / noise levels, sound dampening, tempered glass cases

« on: May 19, 2020, 09:07:06 am »

Some cases (e.g. the Fractal Define 7 XL) offer metal side panels with acoustic dampening or tempered glass as an alternative.

For systems built with the various Raptor motherboards, Talos II and Blackbird, what are the real world observations about noise levels?

Is anybody going to notice the difference in acoustic emissions if they choose tempered glass cases over the fully sound dampened cases?

74

General OpenPOWER Discussion / European logistics / delivery, warranty returns, etc

« on: May 11, 2020, 09:19:52 am »

There are a few comments about this topic in other threads (search for Europe to find them) but I thought it is worth having its own thread.

To summarize the problems:

- delivery from the US means longer delay, higher shipping costs and uncertain administrative charges at customs clearance

- if a product has to be returned under warranty, there are extra taxes in each direction, on top of the extra shipping cost and delay

- if a product being returned is completely dead on arrival or subsequently failed beyond repair, it is a horrendous waste of money to pay more taxes for a round-trip through customs

- overall, the type of person who buys this product is probably an experienced user who is aware of all these risks and may be deterred from purchasing

There are other products with similar challenges, for example, the LimeSDR.  In their case, it is even worse, because they are based in the UK, they asked Europeans to support them in crowdfunding but they decided to outsource all shipping to a US company:
https://discourse.myriadrf.org/t/shipping-from-europe/310

Here is why the issue is really important for OpenPOWER:

- most EU countries have a single market and there is a possibility the UK will participate in the single market after Brexit.

- Switzerland is not in the single market but some logistics companies have been very effective at setting up customs-bonded warehouses near the border and they can deliver into either the EU or Swiss markets without extra burden.  Example: Basel (CH) airport is actually on French territory, in the EU.

- a lot of the developers on related projects are in the European countries.  Looking at the Raptor wiki, the Firefox and Tor issues are on the front page, both of these groups have a lot of people in Europe.

What can be done about this, for the OpenPOWER products but also for other products like LimeSDR?

Please feel free to share any ideas

75

General OpenPOWER Discussion / next generation Blackbird ideas

« on: May 08, 2020, 06:04:11 pm »

I had a look over the various specs and blogs that people wrote about experiences with Blackbird and Talos II boards.

Here are some observations, maybe a wish list for future board designs:

Blackbird only supports 2 DIMMs / 2 memory channels, but for the price of these boards and CPUs I feel that it is very compelling to have 4 channels, as in the Talos II Lite, which is only a little bit more expensive than Blackbird anyway.

With only two PCIe slots on the Blackbird and Talos II Lite and no support for PCIe bifurcation, it is difficult for people to do NVMe RAID.  It would be really useful to have a couple of M.2 slots on the board so that people can have up to two NVMe SSDs and still keep the PCIe slots free.

4K monitors are much more common today than they were when POWER9 was launched.  It would be nice to have basic 2D support for a 4K display on the built-in HDMI port.  From practical experience running Debian and GNOME, I found that small 4K graphics cards, like the NVIDIA K420 with 2GB were unstable but cards with 4GB or more have been trouble-free for a development desktop.

On size, it could be bigger than Blackboard but it really needs to be smaller than Talos II (Lite) to open up a wider choice of cases.

A board with all of the above would probably be somewhere between Blackbird and Talos II Lite.