XSystem252/RaspberryPi4Archlinux64EncryptionUSBBootBtrfsGuide.md

## RaspberryPi4Archlinux64EncryptionUSBBootBtrfsGuide.md

      
    Raw
  

              RaspberryPi4Archlinux64EncryptionUSBBootBtrfsGuide.md
            
          
    How To Set Up a Raspberry Pi 4 with Archlinux 64-bit (AArch64) and Full Disk Encryption (+SSH unlock), USB Boot (No SD-Card) and btrfs

Written by: XSystem

First published on: 20 Dec 2020

Last updated on: 20 Dec 2020
[0] Introduction

Overview

In this guide you will learn how to set up a Raspberry Pi 4 Model B with the following features:

64 bit Archlinux ARM (AArch64)
Full USB Boot

This allows you to ditch the SD card entirely and boot from a thumb drive or SSD connected through a SATA-to-USB adapter.


Full Disk Encryption + SSH Unlock

This applies to the root filesystem, not the boot partition.
You will be able to unlock the system locally using monitor and keyboard or remotely by connecting to the Pi through an SSH connection we will set up.


btrfs as root filesystem

Of course, you can omit certain steps in this guide if you're only interested in parts of the setup.
Maybe you're content with ext4 as the root filesystem, or would like to set up Full Disk Encryption on the SD card and not a USB drive.

All steps in this guide are explained, so you should be able to discern what is necessary for your specific use case.
Caveats

As the guide is written right now, it is assumed you are using the ethernet port for networking. With the way the initramfs will be set up, the onboard WiFi interface is not detected anymore. I believe this can be fixed, but I haven't been able to do that yet.
Motivation and Special Thanks

I decided to write this guide because I was able to find resources online documenting parts of the process, but getting everything to work together took some effort, and I wanted to share my results.
Lots of information presented in this guide is based on already existing guides / resources. Specifically, I'd like to thank and reference:

gea0's guide on setting up Full Disk Encryption + SSH Unlock on a Raspberry Pi 3:

https://gist.github.com/gea0/4fc2be0cb7a74d0e7cc4322aed710d38


lightirius' post over on the ArchlinuxARM forums on how to set up full USB boot by manually updating u-boot:

https://archlinuxarm.org/forum/viewtopic.php?t=14672#p64313


What You Will Need


An SD Card in order to get Archlinux running on the Pi. We will use this installation to set everything up and copy it over to the USB drive once we're ready.
Your USB boot device. This can be a thumb drive, SSD or HDD, connected via USB or a USB-to-SATA bridge.

The Raspberry Pi can be finnicky when it comes to the USB-to-SATA adapters it supports. Do some research to find out which ones are worth picking up.


A way to flash OS images to an SD card.
Some third host from which you would like to remotely unlock the Pi. We will set up an SSH key + config there.
A Raspberry Pi 4 Model B, of course.

[1] Set up the Raspberry Pi's EEPROM to support USB booting.

You need to update the Pi's EEPROM in order for it to support full USB booting.
Since the EEPROM is a piece of memory directly integrated on the Pi's SoC this change will persist even if you swap out all storage media attached to the Pi.
If you have already done this for your Pi, you can skip this section altogether.
There are already a number of guides out on the web on how to accomplish this, so I will keep this section brief.
Install the latest version of Raspberry Pi OS (Yes, Raspberry Pi OS, not Archlinux) onto your SD card.

Refer to the official guides for more information on this: https://www.raspberrypi.org/software/
Note that Raspberry Pi OS Lite is sufficient, a desktop environment is not required.
Remember to place an empty file called 'ssh' into the boot partition if you'd like to use a headless setup. More info here: https://www.raspberrypi.org/documentation/remote-access/ssh/README.md

Once you're up and running, update your system:
sudo apt update
sudo apt full-upgrade
Edit the configuration file governing what firmware updates you receive:
sudo nano /etc/default/rpi-eeprom-update
In this file the FIRMWARE_RELEASE_STATUS variable should currently be set to critical.

Change it to stable in order to receive the latest updates. The file should then look like this:
FIRMWARE_RELEASE_STATUS="stable"

Update the firmware of the Pi:
sudo rpi-eeprom-update -d -a
If required, perform a system reboot now.
At the time of writing (20 Dec 2020), the updated VL805 EEPROM version reads '000138a1'.
Now that the EEPROM has been updated, you can shut down the system. The installation of Raspberry Pi OS is no longer needed and will be replaced with Arch in the next step.
[2] Install 64 bit Archlinux onto the SD card.

We will set up a basic Arch installation on the SD card without any of the special features (no btrfs, no encryption) which will be cloned to the USB drive once everything has been prepared.
Therefore, use ext4 for the root filesystem at this point, we will set up btrfs later.
Simply follow the guide on the Archlinux ARM website on how to do this: https://archlinuxarm.org/platforms/armv8/broadcom/raspberry-pi-4
Don't forget to perform the steps listed in the AArch64 section, i.e. don't forget to pull the 64 bit image and don't forget to update /etc/fstab so that it points to the right block device.
Additionally, you may want to choose a larger size for the boot partition. 200 MiB are sufficient for our setup, but a little more headroom can't hurt.
Once setup, boot into the system and perform basic installation steps.

Please refer to the Archwiki's Installation page for guidance: https://wiki.archlinux.org/index.php/installation_guide
Note that all software you install at this point will carry over to the final, encrypted system.

I'd recommend at least performing the following steps:

As described on the Archlinux ARM installation page, set up the package signing keys with pacman-key --init and pacman-key --populate archlinuxarm.
Perform a full system update with pacman -Syu.
Set up networking so that the machine retains internet access across reboots without manual setup on each boot.
Configure and generate locales.
Set up (a new) user account and change the passwords, sudo is recommended as well.
Install a text editor of your choice.

[3] Install packages required for the special setup.

The following packages are required:


Package
Description


dosfstools
Required to set up a vfat partition on the USB drive.


btrfs-progs
Required to maintain btrfs filesystems. Not necessary if you're sticking with ext4.


rsync
Will be used to clone the prepared system to the USB drive and to transfer files between the Pi and your third system from which you will remotely unlock the Pi.


unzip
What it says on the tin, really. Used to decompress zip-archives.


base-devel
Required to build user packages.


uboot-tools
Required to update the U-Boot boot script.


mkinitcpio-utils
See below.


mkinitcpio-netconf
See below.


mkinitcpio-dropbear
These three packages set up networking and an SSH shell during boot to allow remotely unlocking the root filesystem.


To install all of them at once, run:
sudo pacman -S dosfstools btrfs-progs rsync unzip base-devel \
    uboot-tools mkinitcpio-utils mkinitcpio-netconf mkinitcpio-dropbear
[4] Swap out the stable U-Boot for the release candidate.

To boot the generic / mainline 64 bit Linux kernel, Archlinux uses a bootloader called Das U-Boot.
At the time of writing, the version of the bootloader supplied in the repositories does not support full USB booting. There is, however, a release candidate available that supports full USB booting.
We will now download the package files of the u-boot package installed on the Pi, modify them so that they install the release-candidate instead of the stable version, and then swap out the bootloader on the Pi.
First, acquire the package files for the package uboot-raspberrypi.

These can be found here: https://github.com/archlinuxarm/PKGBUILDs/tree/master/alarm/uboot-raspberrypi
You can use a tool like DownGit in order to download just the specific folder of the repository:

Visit https://downgit.github.io/
Paste the link to the specific folder given above.
Download the zip archive.

You now need to transfer the zip archive to the Pi. (Assuming you performed the steps above on a third system and not on the Pi itself.)
This can for instance be achieved using rsync over ssh:
# Perform this on the third host, assuming the Pi's username and hostname
# are still called 'alarm' and the Pi is connected to the same network.
rsync uboot-raspberrypi.zip alarm@alarm:/home/alarm/
Once the zip archive is available on the Pi, unzip it and change into the directory:
# Make sure you are not root, as root cannot / should not install user packages.
# Change into the home directory, if not already.
cd
# Unzip the archive. All files are already contained within a folder within the archive.
unzip uboot-raspberrypi.zip
# Change into the directory.
cd uboot-raspberrypi
Next, we need to perform some edits on the PKGBUILD in order to use the release candidate instead of the stable version.
vim PKGBUILD
Perform the following three replacements:


Variable
Before
After


pkgname
uboot-raspberrypi
uboot-raspberrypi-rc


pkgver
2020.07
2020.10rc2


First value in md5sums
86e51eeccd15e658ad1df943a0edf622
bae5280c7ce49961c3722fa9019535bf


The PKGBUILD should then look something like this:
# U-Boot: Raspberry Pi
# Maintainer: Kevin Mihelich <[email protected]>

buildarch=12

pkgname=uboot-raspberrypi-rc
pkgver=2020.10rc2
pkgrel=2
pkgdesc="U-Boot for Raspberry Pi"
arch=('armv7h' 'aarch64')
url='http://www.denx.de/wiki/U-Boot/WebHome'
license=('GPL')
backup=('boot/boot.txt' 'boot/boot.scr' 'boot/config.txt')
makedepends=('bc' 'dtc' 'git')
conflicts_armv7h=('linux-raspberrypi')
_commit=f4b58692fef0b9c16bd4564edb980fff73a758b3
source=("ftp://ftp.denx.de/pub/u-boot/u-boot-${pkgver/rc/-rc}.tar.bz2"
        "https://github.com/raspberrypi/firmware/raw/${_commit}/boot/bcm2710-rpi-3-b.dtb"
        "https://github.com/raspberrypi/firmware/raw/${_commit}/boot/bcm2710-rpi-3-b-plus.dtb"
        "https://github.com/raspberrypi/firmware/raw/${_commit}/boot/bcm2710-rpi-cm3.dtb"
        "https://github.com/raspberrypi/firmware/raw/${_commit}/boot/bcm2711-rpi-4-b.dtb"
        '0001-rpi-increase-space-for-kernel.patch'
        'boot.txt.v2'
        'boot.txt.v3'
        'mkscr')
md5sums=('bae5280c7ce49961c3722fa9019535bf'
         '0c56f6b8fde06be1415b3ff85b5b5370'
         'e4b819439961514c7441473d4733a1b4'
         '38cab92f98944f0492c5320cf8b36870'
         '04f2dd06c65cd7ad2932041cbe220a13'
         '728c4a0a542db702b8d88ffe1994660c'
         '69e883f0b8d1686b32bdf79684623f06'
         'be8abe44b86d63428d7ac3acc64ee3bf'
         '021623a04afd29ac3f368977140cfbfd')

# ...
Now, build the package but do not install it just yet:
# Make sure you are cd'd into the PKGBUILD's directory.
makepkg -s
This should install the necessary build-dependencies, download the release candidate and build the bootloader.
Now we're going to swap the stable bootloader with the release-candidate. Make sure to perform both steps in a single session (i.e. don't reboot the system inbetween), as your system is briefly left with no bootloader.
# Uninstall the stable bootloader.
sudo pacman -R uboot-raspberrypi
# Afterwards, while still in the uboot-raspberrypi directory,
# install our freshly built release-candidate.
makepkg -si
makepkg -si will inform you the package has already been built, and will simply proceed to install it.
Once this is done, reboot the system. This is to ensure that swapping out the bootloader worked on your system.
[5] Configure dropbear to allow early SSH access during boot.

These steps are simply adapted from gea0's guide for the Pi 3 which was already linked in the beginning:
https://gist.github.com/gea0/4fc2be0cb7a74d0e7cc4322aed710d38
Set up an RSA SSH key on the system from which you would like to remotely unlock your pi:
ssh-keygen -t rsa -b 4096 -a 100 -f ~/.ssh/pi_unlock_key
Transfer it to the pi:
rsync ~/.ssh/pi_unlock_key.pub alarm@alarm:/home/alarm/
Additionally, set up the configuration on this host system:
vim ~/.ssh/config
--------------------------------------------------------------------------------
Host pi-unlock
  HostName pi-locked
  User root
  IdentityFile ~/.ssh/pi_unlock_key

Of course, you can choose a different hostname here, we will set it up in a later section when we update the /boot/boot.txt file.
If you decide to go with a static IP setup, you can also replace pi-locked with the static IP.
The user has to be root because this is the user used to connect during the early unlock stage.
This doesn't mean that we enable actual root access over ssh for the booted system.
The next steps will now be performed on the Pi and not the host used for unlocking.
Make the copied key dropbear's root key.
sudo mv ~/pi_unlock_key.pub /etc/dropbear/root_key
With this setup, the generated key will only be used to unock the Pi during boot, and not for actually connecting with the Pi over ssh once it has booted.
Of course, feel free to set this up differently.
For instance, you could use one key for both unlocking and connecting with the booted system.
Finally, regenerate the RSA host key with the -m PEM option. This is due to a bug in dropbear which will cause errors in the next section when running the dropbear-hook during mkinitcpio.
cd /etc/ssh
# I found it sufficient to only regenerate the RSA key.
sudo rm ssh_host_rsa_key*
# Regenerate it with the '-m PEM' option.
sudo ssh-keygen -t rsa -b 4096 -f ssh_host_rsa_key -N "" -m PEM < /dev/null
Now that dropbear is set up, we can move on to the initramfs.
[6] Update the initramfs to support btrfs, full USB boot and full disk encryption.

We will now perform a number of changes to the system's initramfs in order to ensure support for the three essential features (btrfs, usb boot and disk encryption + SSH unlock) during boot.
Edit mkinitcpio.conf:
sudo vim /etc/mkinitcpio.conf
We now need to add three additional modules:


module
description


btrfs
Necessary to allow booting from a btrfs filesystem. Not necessary when using ext4.


pcie_brcmstb
Necessary to allow booting from a USB device.


broadcom
The netconf-hook would get stuck during boot if I didn't add this module. It's therefore a necessary module for remotely unlocking the machine.


Also add the following binary:


binary
description


/usr/lib/libgcc_s.so.1
Decryption would otherwise fail with the error that pthread_cancel is not available when using the encryptssh hook.


Finally, we need the following additional hooks:


hooks
insert after
description


keyboard keymap
autodetect
Loads the keyboard early in order to allow entry of the passphrase using monitor and keyboard.


sleep netconf dropbear encryptssh
block
The four hooks necessary to set up early networking, the ssh shell and encryption. sleep ensures all devices are online before setting up networking.


Your /etc/mkinitcpio.conf should then look something like this:
# ...
MODULES=(btrfs pcie_brcmstb broadcom)

# ...
BINARIES=(/usr/lib/libgcc_s.so.1)

# ...
FILES=()

# ...
HOOKS=(base udev autodetect keyboard keymap modconf block sleep netconf dropbear encryptssh filesystems fsck)

Now, rebuild the initramfs:
sudo mkinitcpio -P
[7] Prepare the USB device.

We will now set up the USB device using full-disk encryption.
Note that you should now perform some special steps on the USB drive to ensure full security, which often involves overriding either the entire drive or the second partition with random bytes or zero bytes.

It may also be advisable to perform a SATA Secure Erase on the drive before partitioning.
Instead of providing all the details here, you are strongly advised to consider the corresponding page on the Archwiki before proceeding: https://wiki.archlinux.org/index.php/Dm-crypt/Drive_preparation
Now, on to the actual partitioning of the drive.
Plug in the USB device to the pi and identify it with sudo fdisk -l.
It will probably have been assigned the /dev/sda-identifier.
Once identified, reformat it similarly to the installation guide:
sudo fdisk /dev/sda
# Create a new MBR table.
o
# Create a new primary boot partition.
n
[Enter] (picks the default 'p')
[Enter] (picks the default '1')
[Enter] (picks the default '2048')
+200M
# Set its type correctly.
t
c
# Create the second partition, which we will encrypt momentarily.
n
[Enter] (picks the default 'p')
[Enter] (picks the default '2')
[Enter] (picks the default first sector)
[Enter] (picks the default last sector)
# Print the pending changes and ensure everything looks good.
p
# Apply and exit.
w
Next, set up the appropriate filesystems and encryption:
# Set up the correct filesystem for the boot partition.
sudo mkfs.vfat /dev/sda1
# Set up encryption on the second partition.
sudo cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha512 --use-random -i 1000 /dev/sda2
sudo cryptsetup luksOpen /dev/sda2 root
# And format the btrfs filesystem for it.
sudo mkfs.btrfs /dev/mapper/root
# Alternatively, if you prefer ext4 for the root filesystem:
sudo mkfs.ext4 /dev/mapper/root
If you're using btrfs, now is the time to set up the subvolume structure.

How you do this is ultimately up to you, therefore the section below is just an example.
We will later define rootfs as the root subvolume when adjusting the kernel command line parameters.
Another option can be to set the default subvolume using btrfs subvolume set-default.
# Mount the btrfs subvolume.
sudo mount /dev/mapper/root /mnt
# Create a subvolume for the root filesystem.
sudo btrfs subvolume create /mnt/rootfs
# Unmount for now.
sudo umount /mnt
sudo cryptsetup close root
[8] Prepare the boot.txt with the correct kernel command line arguments.

For now, create a copy of the boot.txt and then edit this copy.
sudo cp /boot/boot.txt /boot/boot.txt.new
sudo vim /boot/boot.txt.new
In it, comment out the line that reads part uuid ... by placing a #-symbol in front of it.
Next, we will focus our attention on the setenv-line that defines the command line arguments of the kernel. Replace the section that reads root=PARTUUID=${uuid} with the following:
ip=::::pi-locked:eth0:dhcp cryptdevice=/dev/sda2:root root=/dev/mapper/root rootflags=subvol=rootfs

The first argument tells the netconf-hook how exactly it is supposed to set up networking.
This is the place where the hostname we defined earlier during ssh configuration comes into play.
Generally, the ip argument has the following pattern:
ip=<client-ip>:<server-ip>:<gateway-ip>:<netmask>:<hostname>:<device>:<autoconf>:<dns0-ip>:<dns1-ip>

For more information and further examples on how to set this up, refer to the corresponding section in the Archwiki: https://wiki.archlinux.org/index.php/Mkinitcpio#Using_net
In my example I have explicitly set the hostname to pi-locked and told netconf to autoconfigure the ethernet interface using DHCP.
The next two arguments set up full disk encryption, using /dev/sda2 as the encrypted blockdevice and mapping it to /dev/mapper/root.
Finally, rootflags=subvol=rootfs ensures that the btrfs subvolume we set up earlier is used as the root filesystem. If you're using ext4 you can omit this parameter.
The boot.txt.new should then look something like this:
# After modifying, run ./mkscr

# Set root partition to the second partition of boot device
#part uuid ${devtype} ${devnum}:2 uuid

setenv bootargs console=ttyS1,115200 console=tty0 ip=::::pi-locked:eth0:dhcp cryptdevice=/dev/sda2:root root=/dev/mapper/root rootflags=subvol=rootfs rw rootwait smsc95xx.macaddr="${usbethaddr}"

# ...

Note: We will run ./mkscr later, which will apply the changes. If applied now, you wouldn't be able to reboot the system anymore.
[9] Clone the system.

We will now clone our prepared system to the USB drive.
First, mount both partitions of the USB drive. One way to achieve this can look like this:
# Create a new folder in your home-directory for this.
mkdir ~/pi-setup
cd ~/pi-setup
# Create folders for the two partitions.
mkdir usb-boot
mkdir usb-root
# Identify the disks.
sudo fdisk -l
# Assuming your USB drive was detected as /dev/sda
# Open the encrypted partition
sudo cryptsetup luksOpen /dev/sda2 root
# Mount both USB partitions
sudo mount /dev/sda1 usb-boot/
# Make sure you mount the BTRFS subvolume for the root filesystem.
sudo mount -t btrfs -o subvol=rootfs /dev/mapper/root usb-root/
# Alternatively, when using ext4:
sudo mount /dev/mapper/root usb-root/
Note: Double check that you mounted the right partitions in the right folders. It is easy to make a mistake here, and you may corrupt the bootstrap-system we've set up so far with the next few commands if you're not careful.
Now, actually clone the system.
sudo rsync --info=progress2 -axHAX /boot/ usb-boot/
sudo rsync --info=progress2 -axHAX / usb-root/
# Ensure the cache is empty.
sudo sync
Now, we need to perform two final adjustments.
First, adjust the fstab of the cloned system by replacing /dev/mmcblk1p1 with /dev/sda1:
sudo vim usb-root/etc/fstab
--------------------------------------------------------------------------------
/dev/sda1 /boot vfat  defaults  0 0
Secondly, apply the changes we made to the bootloader.
cd ~/pi-setup/usb-boot
sudo mv boot.txt.new boot.txt
sudo ./mkscr
Finally, unmount the two partitions.
cd ~/pi-setup
sudo umount usb-boot usb-root
sudo cryptsetup close root
And that's it. Shutdown your system and remove the SD card.
[10] Finish!

Now, connect the USB drive and only the USB drive to the Pi and power it on.
You should now be able to unlock the Pi locally through both monitor and keyboard as well as remotely via ssh by connecting to the Pi from your other host with the host-config that we've set up:
ssh pi-unlock
Known Issues

This setup assumes you're using ethernet for your networking. The initramfs lacks the appropriate kernel modules to set up WiFi, which unfortunately makes it unavailable in the booted system. I'm sure this can be remedied by including the correct module in the initramfs, but I don't know which modules those are.
Moreover, the system consistently prints an error message in the audit-log about a failing hardware interrupt on mmc1. The system consistently scans for the SD card which, of course, isn't there.
I read that this can be turned off by passing an appropriate kernel parameter, but this appeared to have no effect on the problem.
Feedback is welcome!

If you have ideas on how to fix the known issues described above, noticed some other mistake in the guide or if everything worked out fine for you, feel free to let me know in the comments below.
This is the first time I've written a larger guide like this, I hope it will prove helpful.
Package	Description
dosfstools	Required to set up a `vfat` partition on the USB drive.
btrfs-progs	Required to maintain `btrfs` filesystems. Not necessary if you're sticking with `ext4`.
rsync	Will be used to clone the prepared system to the USB drive and to transfer files between the Pi and your third system from which you will remotely unlock the Pi.
unzip	What it says on the tin, really. Used to decompress zip-archives.
base-devel	Required to build user packages.
uboot-tools	Required to update the U-Boot boot script.
mkinitcpio-utils	See below.
mkinitcpio-netconf	See below.
mkinitcpio-dropbear	These three packages set up networking and an SSH shell during boot to allow remotely unlocking the root filesystem.
Variable	Before	After
`pkgname`	`uboot-raspberrypi`	`uboot-raspberrypi-rc`
`pkgver`	`2020.07`	`2020.10rc2`
First value in `md5sums`	`86e51eeccd15e658ad1df943a0edf622`	`bae5280c7ce49961c3722fa9019535bf`
module	description
`btrfs`	Necessary to allow booting from a `btrfs` filesystem. Not necessary when using `ext4`.
`pcie_brcmstb`	Necessary to allow booting from a USB device.
`broadcom`	The netconf-hook would get stuck during boot if I didn't add this module. It's therefore a necessary module for remotely unlocking the machine.
hooks	insert after	description
`keyboard keymap`	`autodetect`	Loads the keyboard early in order to allow entry of the passphrase using monitor and keyboard.
`sleep netconf dropbear encryptssh`	`block`	The four hooks necessary to set up early networking, the ssh shell and encryption. `sleep` ensures all devices are online before setting up networking.