Guide: Run FreeBSD 13.1-RELEASE for ARM64 in QEMU on Apple Silicon Mac (MacBook Pro M1, etc) with HVF acceleration (Hypervisor.framework)

README.md

Guide: Run FreeBSD 13.1-RELEASE for ARM64 in QEMU on Apple Silicon Mac (MacBook Pro M1, etc) with HVF acceleration (Hypervisor.framework)

This guide was adapted from https://gist.github.com/niw/e4313b9c14e968764a52375da41b4278#running-ubuntu-server-for-arm64

Running FreeBSD 13.1-RELEASE for ARM64

1. Install Xcode from App Store or install Command Line Tools on your Mac running on Apple Silicon.

   xcode-select --install
   

2. Install Homebrew and QEMU.

   https://brew.sh/

   brew install qemu

   rehash

   qemu-system-aarch64 --version

   QEMU emulator version 7.0.0
   Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
   

3. Download pre-build EDK II OVMF EFI image for QEMU.

   This EFI image is built from stable202011 tag with additional resolutions in QemuRamfb.c.

   https://gist.github.com/niw/4f1f9bb572f40d406866f23b3127919b/raw/f546faea68f4149c06cca88fa67ace07a3758268/QEMU_EFI-cb438b9-edk2-stable202011-with-extra-resolutions.tar.gz

   To build it from the source code for adding more resolutions, see the following section.

4. Prepare pflash for non-volatile variable store, such as screen resolution.

   mkdir ~/qemu-vm/
   cd ~/qemu-vm/
   tar xvf ~/Downloads/QEMU_EFI-cb438b9-edk2-stable202011-with-extra-resolutions.tar.gz
   dd if=/dev/zero of=pflash0.img bs=1m count=64
   dd if=/dev/zero of=pflash1.img bs=1m count=64
   dd if=QEMU_EFI.fd of=pflash0.img conv=notrunc
   dd if=QEMU_VARS.fd of=pflash1.img conv=notrunc
   

   • This step is optional, you can use -bios QEMU_EFI.fd instead of -drive ...if=pflash lines in the next step, but in that case, any changes in EFI will not be persistent.

5. Download FreeBSD 13.1-RELEASE for ARM64 raw VM image xz-compressed file

   https://ftp2.uk.freebsd.org/pub/FreeBSD/releases/VM-IMAGES/13.1-RELEASE/aarch64/Latest/FreeBSD-13.1-RELEASE-arm64-aarch64.raw.xz

   (See https://docs.freebsd.org/en/books/handbook/mirrors/ for a list of mirrors to choose from.)

6. Decompress xz-compressed file, keeping a copy of the original compressed file

   Keeping a copy of the original file is convenient because then you can use it if you want to create additional VMs later. Just be careful not to overwrite the image of your first VM when you want to make a second VM though :P

   mv ~/Downloads/FreeBSD-13.1-RELEASE-arm64-aarch64.raw.xz .
   unxz -k FreeBSD-13.1-RELEASE-arm64-aarch64.raw.xz
   

7. Grow the disk image

   After you've decompressed the disk image, it'll be about 5 GiB in size. Depending on what you plan to do the amount of available space may be a bit on the low side. Let's grow the disk image by another 30 GiB.

   qemu-img resize -f raw FreeBSD-13.1-RELEASE-arm64-aarch64.raw +30G
   

   By performing this resize before you boot the VM for the first time, FreeBSD will automatically adjust the partition size during first boot.

8. Run your FreeBSD 13.1-RELEASE for ARM64 VM

   qemu-system-aarch64 \
     -M virt \
     -accel hvf \
     -cpu host \
     -smp 4 \
     -m 4096 \
     -drive file=pflash0.img,format=raw,if=pflash,readonly=on \
     -drive file=pflash1.img,format=raw,if=pflash \
     -device virtio-gpu-pci \
     -display default,show-cursor=on \
     -device qemu-xhci \
     -device usb-kbd \
     -device usb-tablet \
     -device intel-hda \
     -device hda-duplex \
     -drive file=FreeBSD-13.1-RELEASE-arm64-aarch64.raw,format=raw,if=virtio,cache=writethrough \
     -nographic \
     -serial mon:stdio
   

sysbench CPU benchmark comparison

Running natively in macOS Monterey Version 12.5 (21G72) on MacBook Pro (13-inch, M1, 2020)

(Installed via Homebrew.)

Events per second: 25,445,903.16 or ~25.4 Mio events/sec

sysbench cpu --threads=2 run

sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 2
Initializing random number generator from current time


Prime numbers limit: 10000

Initializing worker threads...

Threads started!

CPU speed:
    events per second: 25445903.16

General statistics:
    total time:                          10.0000s
    total number of events:              254468472

Latency (ms):
         min:                                    0.00
         avg:                                    0.00
         max:                                    0.12
         95th percentile:                        0.00
         sum:                                 8916.15

Threads fairness:
    events (avg/stddev):           127234236.0000/15768.00
    execution time (avg/stddev):   4.4581/0.00

Running in FreeBSD 13.1-RELEASE for ARM64 in QEMU on MacBook Pro (13-inch, M1, 2020)

(Installed via FreeBSD ports.)

Events per second: 15,467,914.47 or ~15.5 Mio events/sec

sysbench cpu --threads=2 run

sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 2
Initializing random number generator from current time


Prime numbers limit: 10000

Initializing worker threads...

Threads started!

CPU speed:
    events per second: 15467914.47

General statistics:
    total time:                          10.0003s
    total number of events:              154689213

Latency (ms):
         min:                                    0.00
         avg:                                    0.00
         max:                                    0.21
         95th percentile:                        0.00
         sum:                                 5684.03

Threads fairness:
    events (avg/stddev):           77344606.5000/169564.50
    execution time (avg/stddev):   2.8420/0.00

Tested on MacBook Air M2, works fine.
I just tested OpenBSD too:
https://gist.github.com/astreknet/4860e3362ad98e1116f0a970b99e2afc

This guide will not work on M1 macs. Qemu 7.0 has a regression (6.2 worked just fine) and you have to use another build:

# Install QEmu for Mac M1:
#
brew tap uenob/qemu-hvf
brew install --head qemu-hvf

# and then you may want to start headless vm like this:

/opt/homebrew/opt/qemu-hvf/bin/qemu-system-aarch64 \
    -M virt,accel=hvf,highmem=off \
    -m 12288 \
    -smp cores=4 \
    -cpu cortex-a72 \
    -drive file=edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on \
    -drive file=freebsd13.qcow2,format=qcow2 \
    -nographic \
    -device virtio-net-device,netdev=en0 \
    -netdev user,id=en0,hostfwd=tcp::4445-:22 \
    -serial tcp::4444,server,telnet,nowait \
    -pidfile freebsd13.pid

With QEMU 7.2 it works perfectly fine but probably it's a bit outdated.

Ran exactly like explained in the guide, but only on qemu 7.2. While idle, FreeBSD creates load ~200% CPU on my host machine. Is it supposed to be like this?

Update:
I found a solution - simply add this line: kern.hz=100 in /boot/loader.conf
Source: https://gitlab.com/qemu-project/qemu/-/issues/959

Ran exactly like explained in the guide, but only on qemu 7.2. While idle, FreeBSD creates load ~200% CPU on my host machine. Is it supposed to be like this?

Update: I found a solution - simply add this line: kern.hz=100 in /boot/loader.conf Source: https://gitlab.com/qemu-project/qemu/-/issues/959

That's inside FreeBSD root filesystems isn't it?

That's inside FreeBSD root filesystems isn't it?

Yes, on the guest machine (inside qemu) that runs FreeBSD. So this is a boot file config for the guest FreeBSD.

For some reason, I had to do
root@freebsd:~ # efibootmgr -a -c -l /boot/efi/EFI/BOOT/bootaa64.efi -L FreeBSD-14
to get a boot entry for FreeBSD. Not sure why this wasn't the default after the first boot...

Has anyone gotten X running with or without acceleration?

On a M1 Mac with 14.1 absolut perfect