Raptor Computing Systems Community Forums (BETA)

Software => User Zone => Topic started by: q66 on November 27, 2019, 06:51:06 pm

Title: Void Linux thread
Post by: q66 on November 27, 2019, 06:51:06 pm
So I got this idea we could possibly start a thread per distro/OS and share things like a general overview, hardware support and current status and news as a part of it. That would help get things organized and provide some kind of entry point. I wasn't entirely sure which is the best section for this; but the operating system section appears to be more about porting, while I'd like to focus more on what it means for actual users, so User Zone seemed like the right place.

And of course, while at it, as a maintainer of the PowerPC support, shameless self-promotion is never a bad thing for an establishing project, so without further ado, here's a Void Linux thread. I suggest other distribution threads follow a similar layout.

Summary

- Website: https://voidlinux.org, https://voidlinux-ppc.org (PowerPC port)
- IRC: #voidlinux and #voidlinux-ppc on irc.freenode.net
- Package manager: xbps
- Packaging: binary (with ports-style source build support), >11000 packages, >7000 unique software projects
- Release: rolling
- C library: glibc or musl
- Userland: GNU
- Init system: runit
- FSF certified: no
- Linux-libre: no (ships firmware)
- Non-free software: separate repository

PowerPC support

Source support upstream; binary support provided via an unofficial staging fork (production ready)

- 64-bit little endian (glibc + musl) - POWER8+
- 64-bit big endian (glibc + musl) - 970/G5+ (AltiVec required)
- 32-bit (glibc + musl) - generic

What is Void Linux?

Void is a general purpose Linux distribution with the aim of being small, simple and minimalistic, while inheriting lots of its philosophy from the BSDs; it's also independent (not forked from anything). It's based on its own xbps package manager, is rolling release and binary based (while allowing simple custom source builds based on a similar system to a ports tree; this system also allows for cross-compilation of nearly every package to different architectures). You get a choice of two C standard library implementations (glibc and musl) and it notably uses runit instead of systemd for its init system (but is not an anti-systemd distro per se) as well as libressl instead of openssl. Officially, it's available for x86 (i686 and x86_64) and ARM (armv6l, armv7l, aarch64). Unofficially, it's available for PowerPC (ppc32, ppc64, ppc64le). Source build profiles are also available for MIPS (mips, mipsel).

PowerPC fork

Started in December 2018. Void now provides support for 6 flavors (ppc64le/ppc64/ppc glibc/musl) total, with ppc64le having the greatest support for now, but all of them growing (https://repo.voidlinux-ppc.org/stats.html). I'd consider it production ready at this point. Live images are shipped with support for everything from old Macs to most modern POWER9 machines (for both LE and BE). Installer is provided.

Modern ELFv2 ABI is used for both LE and BE (kernel and userland). 4kB page size is used for kernel (as opposed to 64kB used by most other distros), as it makes much more sense for any use case other than a single-purpose server with large amounts of RAM, which is nowhere near the primary target (it's desktop/workstation).

Most desktops and other important software are already supplied. There's at least one modern browser with JavaScript support on every target (Firefox Quantum on all 64-bit with 32-bit being WiP, webkit2gtk and qt5-webkit based ones (Epiphany, Midori, Surf...) on all, and Chromium-based qt5-webengine and browsers based on that (Falkon, qutebrowser...) on ppc64le and experimentally ppc64 - runs, but has color and likely other issues)

Build infrastructure is provided by a Talos 2 Lite, with an 18-core POWER9 CPU, 128GB RAM and a 4x1TB (2+2 mirror) SSD setup with ZFS. A VM (KVM) with half the resources of the machine handles BE and 32-bit builds (no cross-compiling for any of the packages). Both the host and guest systems run Void, of course. The final packages are hosted in Chicago, IL on a server provided by a Void and Talos community member Zach Dykstra.

There's also documentation in a handbook format and FAQ provided on the website.

Blackbird:
(https://i.imgur.com/AF8JZVJ.png)

(https://ftp.octaforge.org/q66/random/desktop.png)

PowerBook G4:
(https://i.imgur.com/ZYcgspm.jpg)

Power Mac G5:
(https://i.imgur.com/xrxJXq7_d.jpg?maxwidth=9999)

(https://i.imgur.com/lo9ZhNp.png)
Title: Re: Void Linux thread
Post by: Jubadub on November 27, 2019, 09:02:54 pm
Agreed on selecting the "User Zone" section, it does seem more appropriate, according to their descriptions. And this is an excellent idea! I had considered something like this, but I had zero drive to actually get it done, let alone with this level of superb quality. Nice job! This certainly serves as a perfect entry point.

If I may make a suggestion, what about an extra line in the summary to state whether or not the distro/flavor contains binary blobs, and its FSF status? This will be an important point for some, especially now that the Talos line has been officially certified with FSF's RYF. Libre software enthusiasts are certainly a considerable end-user demographic for RaptorCS. Afterall, we are talking about the most powerful computer lines of all time among those which FULLY give control back to the user.
Title: Re: Void Linux thread
Post by: q66 on November 27, 2019, 11:55:27 pm
I guess that's a good point. Though, for Talos specifically, it won't try to load any blobs by default in the first place, as they're not required (unless you add something that needs them like a GPU)
Title: Re: Void Linux thread
Post by: Jubadub on November 28, 2019, 06:42:22 am
Interesting, I didn't know that. But would blobs be required even for the AMD Radeon Pro WX7100 and AMD Vega 64? On the official purchase page (https://secure.raptorcs.com/content/TL2WK2/purchase.html), unlike the other 4 GPUs listed there (all NVidia), those 2 AMD ones aren't marked as "proprietary".
Title: Re: Void Linux thread
Post by: q66 on November 28, 2019, 06:57:40 am
Obviously, as all GPUs require firmware, and specifically anything using amdgpu needs *loadable* firmware (i.e. it's not onboard anymore) in form of signed firmware blobs. The driver may not be proprietary, but the firmware always is.
Title: Re: Void Linux thread
Post by: SiteAdmin on November 28, 2019, 03:31:05 pm
Obviously, as all GPUs require firmware, and specifically anything using amdgpu needs *loadable* firmware (i.e. it's not onboard anymore) in form of signed firmware blobs. The driver may not be proprietary, but the firmware always is.

We mark anything requiring kernel/userspace binaries as Proprietary.  Card side firmware we don't explicitly call out, since it's IOMMU isolated.  If we see any libre GPU options start showing up, we'll start calling out cards that require vendor firmware somehow, but for now there's no real point considering the state of the GPU market.
Title: Re: Void Linux thread
Post by: Jubadub on November 29, 2019, 07:54:25 am
Obviously, as all GPUs require firmware, and specifically anything using amdgpu needs *loadable* firmware (i.e. it's not onboard anymore) in form of signed firmware blobs. The driver may not be proprietary, but the firmware always is.

Actually, what is obvious is that what you said is not obvious (and is technically even wrong in the end when you said "the firmware always is [proprietary]"), for 2 simple, factual reasons:

- Just like drivers, firmware source code exists for various devices, and GPU firmware is no different (whether they are already out there or not is another matter). In fact, even the Talos computers contain firmware you can recompile at will;

- The purchase page marks the NVidia GPUs as "proprietary", which, for people not familiar with the availability or lack thereof of libre GPU firmware, can easily and reasonably be interpreted it refers to the firmware. (The driver situation being proprietary or not is OS-dependant, not Talos-dependant, and the purchase page is for Talos alone).

There's also the fact it has been previously discussed on Raptor's Twitter (not sure if by them or a 3rd party) that ALL firmware has been "liberated" (as in, made libre) for the RaptorCS computers (or something along those lines), except one: HDD / SSD firmware. But whether that is true or not, I don't know. But people checking out Raptor's two Twitter accounts can easily come across the statement.

We mark anything requiring kernel/userspace binaries as Proprietary.  Card side firmware we don't explicitly call out, since it's IOMMU isolated.  If we see any libre GPU options start showing up, we'll start calling out cards that require vendor firmware somehow, but for now there's no real point considering the state of the GPU market.

Now this clears it up, many thanks.
Title: Re: Void Linux thread
Post by: q66 on November 29, 2019, 09:08:08 am
Actually, what is obvious is that what you said is not obvious (and is technically even wrong in the end when you said "the firmware always is [proprietary]"), for 2 simple, factual reasons:

- Just like drivers, firmware source code exists for various devices, and GPU firmware is no different (whether they are already out there or not is another matter). In fact, even the Talos computers contain firmware you can recompile at will;

What are you talking about? There is no libre GPU firmware. It literally doesn't exist. Raptor hardware doesn't contain a GPU, so it doesn't need to concern itself with that out of box, but any GPU you add, it will always have proprietary firmware.
Title: Re: Void Linux thread
Post by: MPC7500 on November 29, 2019, 09:47:13 am
That's why Raptor is unable to build a notebook at the moment.

Quote
...and an open-firmware DRM-free GPU.  That's the primary hold-up for us right now, not the CPU.

Twitter (https://twitter.com/RaptorCompSys/status/1190770997412143109?s=20)
Title: Re: Void Linux thread
Post by: nglevin on November 30, 2019, 11:04:43 pm
q66, thanks for all the work to get Void working on POWER. I really enjoyed your talk on the challenges of porting the distro (https://www.invidio.us/watch?v=e4ooNubBFJw), along with exactly what pieces work and why.

I'd like to give it a try with WindowMaker (http://www.windowmaker.org/) and dmenu (https://tools.suckless.org/dmenu/), after being somewhat convinced of dmenu's seaworthiness with this Void Linux setup guide (http://www.troubleshooters.com/linux/void/quickinst.htm). With that, it should resemble an old FreeBSD desktop I used to have.

Also, props for having bpftrace (https://github.com/iovisor/bpftrace) on all platforms. That will easily make me miss dtrace less.


Since it's mentioned as a distinguishing factor (https://docs.voidlinux-ppc.org/) in this port, are there benchmarks showing the impact of ELFv2 on ppc64be or ppc32?
Title: Re: Void Linux thread
Post by: q66 on November 30, 2019, 11:12:55 pm
Since it's mentioned as a distinguishing factor (https://docs.voidlinux-ppc.org/) in this port, are there benchmarks showing the impact of ELFv2 on ppc64be or ppc32?

ELFv1/ELFv2 are 64-bit ABIs, 32-bit uses the SVR4 ABI, so it's not relevant there. ELFv2 on 64-bit supposedly has faster library calls and stuff, but I haven't benchmarked it. It's nicer/simpler/less annoying from programming perspective though, and there is just no reason to go with a 30 years old legacy ABI when there's a choice not to, if people have some legacy binaries or something, they can spin a debian chroot or whatever legacy environment for it, there's nothing preventing that.

I don't even see how I'd go about benchmarking that as it would mean having to have a reasonably large ELFv1 testing environment in the first place (i.e. compiled with otherwise the same flags etc., so another distro won't do) which I never had (ELFv1 was never even attempted).
Title: Re: Void Linux thread
Post by: nglevin on November 30, 2019, 11:48:50 pm
That's fair. Benchmarks can be deceiving as they are further isolated from their original context.

POWER is one of those ABIs that I hadn't had much experience in. Based on your reference to SVR4, found this 2014 slide deck (https://llvm.org/devmtg/2014-04/PDFs/Talks/Euro-LLVM-2014-Weigand.pdf) introducing ELFv2 in the context of what came before and the default 64-bit Linux POWER ABIs (ELFv1 for be, ELFv2 for el) in the GCC compiler options (https://gcc.gnu.org/onlinedocs/gcc/RS_002f6000-and-PowerPC-Options.html).

This clears things up, thanks.
Title: Re: Void Linux thread
Post by: Raion on December 03, 2019, 01:09:11 am
Neat stuff you're doing q66 for Void Power.

Aikatsu's a cute anime, I saw that in your shot. Checking the docs you have, it's not clear - does this distro have qemu-kvm support?
Title: Re: Void Linux thread
Post by: q66 on December 03, 2019, 06:37:50 am
Neat stuff you're doing q66 for Void Power.

Aikatsu's a cute anime, I saw that in your shot. Checking the docs you have, it's not clear - does this distro have qemu-kvm support?

Sure, there's just one caveat when using it as a host system: when using -machine newer than pseries-2.x, qemu will complain about not having 64kB pages, as Void uses 4kB pages for its kernels; the workaround is either to further limit HPT to 4k (like -machine <whatever you want to use>,cap-hpt-max-page-size=4096) - this will work but also limit guests to 4k when using HPT (on POWER9 when using Radix MMU, which is default, 64k guests will work fine unless booted in HPT mode), alternatively use something like -machine pseries-2.11 which will boot 64kB guests regardless of other settings.

So basically mostly affects POWER8 users, or those who explicitly disable Radix (for KVM-PR or whatnot) and even most of those can just use some pseries-2.x machine type and it'll work alright. The only ones out of luck are those who have older hw, require 64kB guests, and also require some feature available only in newer machine types, but vast majority of users don't need to care.
Title: Re: Void Linux thread
Post by: q66 on December 10, 2019, 04:32:37 pm
Currently attempting a 64-bit long double transition to get rid of the messy IBM double-double format used on most distros and unify the representation across targets. Musl has been using it all along, now trying glibc.

I don't want to go with the 128-bit IEEE754 format as that requires VSX and would mean a different one on ppc64le and the rest (and also, when not targeting POWER9, it means basically reverting to soft-float).
Title: Re: Void Linux thread
Post by: madscientist159 on December 10, 2019, 06:22:45 pm
Currently attempting a 64-bit long double transition to get rid of the messy IBM double-double format used on most distros and unify the representation across targets. Musl has been using it all along, now trying glibc.

I don't want to go with the 128-bit IEEE754 format as that requires VSX and would mean a different one on ppc64le and the rest (and also, when not targeting POWER9, it means basically reverting to soft-float).

What kind of performance loss are we looking at though on POWER9 vs. 128-bit IEEE754?
Title: Re: Void Linux thread
Post by: q66 on December 10, 2019, 07:47:50 pm
Currently attempting a 64-bit long double transition to get rid of the messy IBM double-double format used on most distros and unify the representation across targets. Musl has been using it all along, now trying glibc.

I don't want to go with the 128-bit IEEE754 format as that requires VSX and would mean a different one on ppc64le and the rest (and also, when not targeting POWER9, it means basically reverting to soft-float).

What kind of performance loss are we looking at though on POWER9 vs. 128-bit IEEE754?

If you mean performance loss with 64-bit ldbl vs 128-bit IEEE754, 64-bit should be faster, because Void is compiled for POWER8 like other distros and therefore can't use POWER9 isns (other than in glibc math functions, which can be chosen at runtime). The actual difference, don't know, I haven't benchmarked it (and won't, as that would mean compiling a set of packages for that).

I kind of don't want to use the 128-bit stuff for long double type in principle, as the hard VSX requirement is there either way (gcc will not compile it without -mcpu set to at least power7) and I'd kind of like the little endian and big endian targets differ mostly in just endianness and baseline -mcpu setting.

I never liked the long double type anyway, as the guarantees you get with it are zero (it's only required to be at least as big/precise as double and that's it). So going with a 64-bit baseline that's covered well in hardware everywhere seems like the sane choice (plus musl won't be adopting 128bit IEEE754 as it would require introduction of a new subarch because musl doesn't version symbols - so that would make glibc little endian the *only* outlier).

People who want 128-bit IEEE754 floating point can easily use the __float128 type in gcc. That is a much better option as it has guaranteed properties (similarly, it works on modern x86 with at least 128-bit wide SIMD as well). People who want the IBM 128bit floating point can also use the __ibm128 type in gcc in a similar manner.

Adopting 64-bit long double in glibc is actually pretty easy right now - because it used to use it in the past (before IBM made them change the default to the silly 128-bit double-double). Therefore, all the compatibility symbols as well as redirects for when you use -mlong-double-64 are set up in glibc, and all it takes is recompiling libraries and programs that use 128-bit IBM long double. You can easily tell which do - they will be tagged like Tag_GNU_Power_ABI_FP: hard float, 128-bit IBM long double in the ELF attributes. This tag is only introduced when long doubles are used in the binary and it was compiled with those set to 128-bit IBM format.

So, I just went and checked every package containing ELF files for the tag, and compiled a list. It's a relatively small number, only about 330 packages out of 8000+. The rest of them can be left alone.
Title: Re: Void Linux thread
Post by: q66 on December 12, 2019, 12:50:00 pm
After some experimentation, I came to the conclusion that it is currently not practical to switch, as while things will mostly work, using compat symbols ultimately means that some configurations will break; in this case, things that manually declare stuff like math symbol prototypes (which is allowed) for long double math funcs will still use the ibm128 symbols and silently have wrong results... while this is rare, it is a concern.

I searched for potential workarounds but they mostly just introduced new issues elsewhere...

I'll wait until glibc has ironed out its IEEE754 128-bit support and some other distros have switched to it (i know Fedora is planning that) and then I'll reevaluate my options (e.g. possibly see if I could get in a 64-bit ldbl ABI *without* falling back to compat symbols, which would generally solve everything, besides compatibility with ibm128 and glibc is going to need to handle that for the ieee754 transition anyway). Since transition has proved to be less of a problem than expected by itself, it should be relatively easily doable later on.

So, sticking with what I have for now.
Title: Re: Void Linux thread
Post by: q66 on December 16, 2019, 03:51:14 am
New packages batch; bringing out-of-box support for the Navi GPUs in the 5.4 kernel, further BE expansion, and other cool stuff 8)
Title: Re: Void Linux thread
Post by: q66 on January 03, 2020, 11:07:10 am
We're now complete (as in the whole repo is built and everything not buildable is properly marked as such) on ppc64le and ppc64le-musl \o/
Title: Re: Void Linux thread
Post by: ClassicHasClass on January 04, 2020, 06:28:13 pm
Excellent!!
Title: Re: Void Linux thread
Post by: q66 on January 15, 2020, 02:08:50 am
The repos are now complete on big endian/32 bit targets as well. Final coverage as usual at https://repo.voidlinux-ppc.org/stats.html. Currently at ~100k packages, amounting to just under 450GB total.
Title: Re: Void Linux thread
Post by: xilinder on January 18, 2020, 10:08:04 am
I installed Void Linux as a base system only and added 'mc''links' and 'gpm'.
I'm not getting a mouse. How do I start it?
Title: Re: Void Linux thread
Post by: q66 on January 18, 2020, 10:12:10 am
presumably you need to enable the appropriate service...

something like


ln -s /etc/sv/gpm /var/service/
Title: Re: Void Linux thread
Post by: xilinder on January 18, 2020, 10:47:17 am
Yup! That did it. Thanks
Title: Re: Void Linux thread
Post by: q66 on April 11, 2020, 11:34:24 am
Fresh installation/live media now available: https://voidlinux-ppc.org/news/2020/04/new-images.html
Title: Re: Void Linux thread
Post by: q66 on May 01, 2020, 09:19:02 pm
Latest packages now fix Navi GPUs on kernel 5.6 (5.5 and 5.4 had it working all along). Once again we're the first to have it, but necessary patch was submitted upstream as well and in the meantime made it into drm-next, and will trickle down to other distros later.