Running FreeBSD 12.2 as KVM Guest on Ubuntu 20.04 on Talos II.

[bloudraak]

I'm curious whether anyone has been able to run FreeBSD 12.2 PPC64 as a guest in KVM, running on Ubuntu 20.04 ppc64le (or anything relevant), and how they managed to get it to work. I'm hoping I looked at this problem too long and I'm missing something really simple.

I'm building out a lab of different operating systems and configurations in order to test some software. My Talos II is running Ubuntu 20.04 ppc64le, with KVM installed and configured. I'm already running a few ppc64le and ppc64 virtual machines on the host, including FreeBSD 13 (both ppc64le and ppc64). The issue at hand is specific to FreeBSD 12.02 ppc64. It has 256GB RAM and 24TB storage, which I think is sufficient to build a lab to test various PPC64 and PPC64LE configurations.

I downloaded the FreeBSD 12.2 ppc64 ISO and tried to create the virtual machine as follows:

Code: [Select]

virt-install \
    --name freebsd12-ppc64-01 \
    --memory 2048 \
    --virt-type=kvm \
    --disk pool=default,size=64,format=qcow2 \
    --vcpus=1 \
    --network network=br0 \
    --os-variant freebsd12.2 \
    --graphics none \
    --cdrom /var/lib/libvirt/boot/FreeBSD-12.2-RELEASE-powerpc-powerpc64-dvd1.iso \
    --debug \
    --force

which failed with

Code: [Select]

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...               
Kernel entry at 0x102390 ...
panic: Standard large pages (SLB[L] = 1, PTE[LP] = 0) not supported by this system. Please enable huge page backing if running under PowerKVM.
cpuid = 0
time = 1
KDB: stack backtrace:
#0 0xc000000000729964 at ??+0
#1 0xc0000000006c1254 at ??+0
#2 0xc0000000006c1320 at ??+0
#3 0xc000000000b1a764 at ??+0
#4 0xc000000000b0b2ac at ??+0
#5 0xc000000000b04a0c at ??+0
#6 0xc000000000102440 at ??+0
Uptime: 1s

I configured huge pages on Ubuntu using  https://help.ubuntu.com/community/KVM%20-%20Using%20Hugepages. And the output of

Code: [Select]

cat /proc/meminfo | grep Huge
Looks something as follows:

Code: [Select]

AnonHugePages:   2904064 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:   16384
HugePages_Free:    15360
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:        33554432 kB
I then tried to create the VM using the following after adding memorybacking argument.

Code: [Select]

virt-install \
    --name freebsd12-ppc64-01 \
    --memory 2048 \
    --virt-type=kvm \
    --disk pool=default,size=64,format=qcow2 \
    --vcpus=1 \
    --memorybacking hugepages=yes \
    --network network=br0 \
    --os-variant freebsd12.2 \
    --graphics none \
    --cdrom /var/lib/libvirt/boot/FreeBSD-12.2-RELEASE-powerpc-powerpc64-dvd1.iso \
    --debug \
    --force
It still failed with the same error.  I requested some help from the FreeBSD forums https://forums.freebsd.org/threads/installing-freebsd-12-2-ppc64-in-kvm-on-power9.83253/, on which there's an interesting comment:

freebsd kernel reads available page sizes from openfirmware device tree @ibm,segment-page-sizes
if the key is present but it does not report 16MB pages it bombs

The default huge page size is 2M, and no matter what I have tried (kernel args and whatnot), that remains the default. Hugeadm lists 2M and 1G page sizes.

Code: [Select]

$ hugeadm --page-sizes-all
2097152
1073741824
and if I try to create a 16M pool, it barfs:

Code: [Select]

$hugeadm --pool-pages-min 16M:64
hugeadm:ERROR: 16M: unknown page size
What am I missing?