Messages

166

Operating Systems and Porting / Re: kernel config: page size 4k vs 64k

« on: September 16, 2020, 02:17:02 am »

Today I made some tests comparing Firefox WebRTC on 64k and 4k page size

With a Logitech webcam, I found both 64k and 4k worked

With Elgato Camlink 4k, v4l2-loopback and my gstreamer script, I found it only worked on 4k.  With 64k, Firefox displayed the local monitor / self-view from the Camlink but the remote host wasn't receiving the picture.  Firefox was receiving the picture from the remote host, so this was a one-way video problem.

The gstreamer script is a very simple chroma-key example, it is not OBS but then it is less than 10 lines of code.

167

Operating Systems and Porting / Re: kernel config: page size 4k vs 64k

« on: September 15, 2020, 05:22:39 pm »

(I modified the code last night, now it builds 64k and 4k kernels as two different flavours so you can install both on the same system and choose between them in the petitboot menu, the 4k kernel file has 4k in the filename and ABI string)

Here is my patch for building a Debian kernel package with the 4k page size, like other architectures

To use it:

Code: [Select]

mkdir -p ~/ws/kernel
cd ~/ws/kernel
wget http://deb.debian.org/debian/pool/main/l/linux/linux_4.19.132.orig.tar.xz
xzcat linux_4.19.132.orig.tar.xz | tar xf -
git clone https://gitlab.com/dpocock/linux-kernel-debian
cd linux-kernel-debian
git checkout pocock/buster-ppc64el-4k
cd ../linux-4.19.132
ln -s ../linux-kernel-debian/debian .
dpkg-buildpackage -rfakeroot -i.* --no-sign -b -j`grep -c ^processor /proc/cpuinfo`
cd ..

You should find the packages *.deb in ~/ws/kernel

You can just install it with dpkg and reboot into the kernel with 4k page size

Beware: if you have a btrfs root filesystem with sectorsize=64, the kernel with 4k page size can't mount it.  If your root filesystem is ext4 it will just work.

If people are comfortable with this change then it will be a good idea to open bug reports for the kernel packaging teams in each of the relevant distributions.

168

Talos II / CPU alignment / slots not working

« on: September 15, 2020, 03:12:48 pm »

After my success with the NVIDIA card, I decided to try it in each slot

I found that it wouldn't work in slot 3, that is the middle 16x slot.

I tried removing and re-inserting it a couple of times and I inspected the slot with a torch and couldn't see any problem

Next I tried removing and re-seating CPU #2.  After doing this, the NVIDIA card in slot 3 is working.

Are there any diagnostics to confirm CPUs are correctly inserted?  Everything else I did on the CPU appeared normal, for example, I had exercised all cores and RAM without problems.

Is there any technique to follow when inserting the CPUs to avoid issues like this?

169

User Zone / Re: Working xorg.conf for nouveau driver for a NVIDIA graphics card?

« on: September 15, 2020, 03:08:13 pm »

I took this a bit further

On a positive note, Quadro K420 also worked immediately

On the other hand, when I put the system under heavy load, Nouveau crashed.  I was still able to ssh into the box, kill Xorg and start Xorg again.  This may not be a problem specific to this platform, it may be Nouveau.  amdgpu hasn't given me any similar problems when I have the system under load.

170

User Zone / Re: Working xorg.conf for nouveau driver for a NVIDIA graphics card?

« on: September 15, 2020, 01:00:01 pm »

After compiling the Debian buster kernel with a 4k page size, the Nouveau driver and Quadro K2200 immediately worked

171

General OpenPOWER Discussion / Re: News?

« on: September 15, 2020, 09:05:22 am »

I don't have an answer about the Blackbird

From my personal analysis, I feel that many people considering a Blackbird might get more advantages from a Talos II Lite, it is only about 10% more expensive but there are benefits as outlined on the comparison page here.

In particular, using more than 2 memory channels or more than 8 cores both give you significant performance benefits if that is necessary for your workload.  It can also take a GPU wider than 2 slots because there is nothing else beyond the GPU slot.

172

Operating Systems and Porting / Re: kernel config: page size 4k vs 64k

« on: September 15, 2020, 06:40:26 am »

Some emulation and virtualization tools have dependencies on page size. We had some page size issues in Firefox initially, though I think we smoked most of those out.

Maybe that is why Thunderbird is not working at all either.

Simply running a 4k kernel doesn't always fix the issues.  Sometimes it is necessary to rebuild each broken application on the host running a 4k kernel.  This is because the build scripts of some applications look at the page size during compile time and assume it will be the same for run-time and they hardcode it into the binary.

I will probably try building both the Firefox and Thunderbird packages from source on a Debian buster host running with the 4k page size.

I tried Firefox for a WebRTC call yesterday and the video was missing, I'll see if that works with the 4k page size too, with and without recompiling Firefox itself.

The two biggest concerns I heard from people about this platform are not being able to use Firefox reliably and the noise issues.  Developers are reluctant to embrace any platform that might sidetrack us from the things we are supposed to be working on.  If the noise issues can be fixed with Vikings' water cooling solution and the Firefox issues can be reined in by a 4k page size then it could make a crucial difference to the success of the platform.  It is therefore quite important to look at how to engage distributions in that solution.

173

Operating Systems and Porting / Re: kernel config: page size 4k vs 64k

« on: September 15, 2020, 06:29:43 am »

Yes, that is correct, the sectorsize parameter for a btrfs filesystem must be the same as PAGESIZE for the kernel

Therefore, btrfs filesystems created on systems with a 4k page size can only be used on systems with a 4k page size

btrfs filesystems created on systems with a 64k page size can only be used on systems with a 64k page size

174

User Zone / Re: Working xorg.conf for nouveau driver for a NVIDIA graphics card?

« on: September 14, 2020, 02:56:20 pm »

Yes, I only discovered that by chance while investigating the btrfs issues.

I compiled a kernel package with 4k page size and I have another SSD with Debian in an ext4 filesystem that should boot with 4k page size.  I'll probably try it in the next couple of days to see if it works with Nouveau and the Quadro K2200

175

Operating Systems and Porting / kernel config: page size 4k vs 64k

« on: September 14, 2020, 05:19:55 am »

Most distributions have chosen the 4k page size for their kernels on Intel architectures.

On powerpc64le, Debian (since 2014) and Fedora are using 64k

This may be good for HPC environments but troublesome for workstation users

Here are some specific observations:

Nouveau driver apparaently won't work at all on anything other than 4k

btrfs filesystems have a sectorsize that corresponds to the page size on the host where the filesystem was created.  The filesystems can't be mounted on any host with a different page size.

Example problems:

• you have a 6TB drive with btrfs from an x86 workstation, you want to move it to a Blackbird running the default Debian or Fedora kernel but it won't mount.
• you install Fedora 33 with the default kernel (64k page size) and default filesystem (btrfs from Fedora 33) and later you want to recompile your kernel for 4k.  Now you can't mount your root filesystem because mkfs was run for 64k

As Fedora is going to use btrfs by default now, btrfs volumes will be more common and users are more likely to encounter frustration from time to time.

The btrfs developers recently started a patch that allows the systems with 64k page size to read a btrfs volume with 4k sectorsize.  It only works in one direction and it is read-only.

Does anybody feel that distributions should offer different permutations of their kernel and installer based on the 4k page size?

From the perspective of workstation users, are there any other strong reasons, other than Nouveau and btrfs, for distributions to consider this?

176

General Discussion / Re: noise levels, sound dampening, tempered glass cases

« on: September 12, 2020, 03:58:08 pm »

My supplier still doesn't have the regular case (both sides dampened panels) so I took the case with dark tinted glass

As the case is so large and the case fans are big, 140mm, they can run at low speed and the overall build is very quiet when CPUs are idle

After hearing the noise levels at full load, I don't believe the dampened panel would make much difference so it is more a matter of personal preference choosing between glass or the regular panel on this case.

As there are some flashing leds on the motherboard, the glass panel might be annoying if the machine is installed in a location where somebody sleeps.

177

Operating Systems and Porting / Re: petitboot doesn't show the OpenBSD partition

« on: September 12, 2020, 03:52:53 pm »

Thanks to feedback from the OpenBSD developers, I found that it is necessary to select the "whole disk" option while partitioning

If that is selected, petitboot will show OpenBSD in the boot menu and it is possible to boot the installed OpenBSD system

This was tested with OpenBSD installed on an SSD in a USB 3 dock

178

Applications and Porting / Re: Building software, a bit of a newb question

« on: September 12, 2020, 02:18:28 pm »

OK, it looks like they only distribute the package for amd64

Something like this will help you build a local package on your architecture:

Code: [Select]

sudo apt build-dep obs-studio
git clone https://salsa.debian.org/multimedia-team/obs-studio.git
cd obs-studio
sed -i -e 's/any-amd64/any/g' debian/control
dpkg-buildpackage -rfakeroot -i.git -b

Something like that will create *.deb package files for you in ../

You can try asking in #debian-devel on IRC for any issues

I will eventually get around to running OBS myself, sometime this month and then I will share more details if the above is not sufficient.

179

Applications and Porting / Re: RStudio desktop working on Raptor's power computer (ppc64le)?

« on: September 12, 2020, 01:50:03 pm »

Did you have any luck with RStudio?

I also use both R and RStudio and I'm likely to try it in on this platform very shortly

180

Talos II / power consumption observations

« on: September 12, 2020, 01:48:31 pm »

I made the following observations using the meter in my UPS, it measures units of approximately 0.1A

Measurements were made with Debian buster in the idle state, at the GNOME login screen.  It actually uses less power there than at the petitboot menu.

Voltage = 240V



Unplugged: 0

BMC powered: ~ 0.1A

Single 8 core CPU installed with 4 memory chips: add 0.3A - 0.4A above the BMC power draw (total, 0.4A = 96W)

Second 8 core CPU installed with 4 more memory chips: add 0.2A - 0.3A above the single 8 core (total, 0.6A = 144W)

These are bare minimum figures in an idle system with RX 580, LSI 9207-4i4e and one SATA SSD

The system doesn't have the SAS option on the board, that would use more watts if present.

HP suggests a comparable single CPU Z8 G4 workstations can use 74W at idle:
https://www8.hp.com/h20195/v2/GetPDF.aspx/c05527763.pdf (page 31)

Their Z4 workstations can go down to about 40W at idle although the spec is more like a Blackbird with 4 core:
https://www8.hp.com/h20195/v2/getpdf.aspx/c05527757.pdf

I feel that for people who are mostly idle and just need an occasional boost of power to compile something from time to time, it may be better to stay on a single CPU and Talos II Lite but for people who need the power regularly for different types of work, the HP and Talos II would end up using similar amounts of power.

For people who leave their workstation on all the time or use it as a file server, the second CPU is wasting a lot of power when idle.  As discussed in another thread, it would be useful to have options to power down some cores