Skip to content

Instantly share code, notes, and snippets.

@misuzu
Last active December 28, 2024 21:28
Show Gist options
  • Save misuzu/89fb064a2cc09c6a75dc9833bb3995bf to your computer and use it in GitHub Desktop.
Save misuzu/89fb064a2cc09c6a75dc9833bb3995bf to your computer and use it in GitHub Desktop.
Install NixOS on Oracle Cloud over Ubuntu 18.04

Install NixOS on Oracle Cloud over Ubuntu 18.04 (make sure to use Ubuntu 18.04 or this may not work)

# install useful tools
sudo apt-get update
sudo apt-get install --no-install-recommends -y nano mc git

# prepare /boot
sudo umount /boot/efi
sudo mv /boot /boot.bak
sudo mkdir /boot/
sudo mount /dev/sda15 /boot
sudo mv /boot/* /boot.bak/efi/

# use swap file
sudo dd if=/dev/zero of=/swapfile bs=1M count=1024 status=progress
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

# install nix
sh <(curl -L https://nixos.org/nix/install)
. $HOME/.nix-profile/etc/profile.d/nix.sh
nix-channel --add https://nixos.org/channels/nixos-21.11 nixpkgs
nix-channel --update

# install nixos-generate-config and nixos-install
nix-env -f '<nixpkgs>' -iA nixos-install-tools

# generate config
sudo `which nixos-generate-config` --root /

# remove lxc mounts
sudo nano /etc/nixos/hardware-configuration.nix
# set hostname, add users and ssh-keys, enable openssh
sudo nano /etc/nixos/configuration.nix

# build config
nix-env -p /nix/var/nix/profiles/system -f '<nixpkgs/nixos>' -I nixos-config=/etc/nixos/configuration.nix -iA system

# prepare target
sudo chown -R 0.0 /nix
sudo touch /etc/NIXOS
sudo touch /etc/NIXOS_LUSTRATE
echo etc/nixos | sudo tee -a /etc/NIXOS_LUSTRATE

# install NixOS
sudo NIXOS_INSTALL_BOOTLOADER=1 /nix/var/nix/profiles/system/bin/switch-to-configuration boot

sudo reboot

Recommended configuration options

{
  # Oracle Cloud uses EFI boot
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  # Kernel cmdline from Ubuntu config
  boot.kernelParams = [
    "console=ttyS0"
    "console=tty1"
    "nvme.shutdown_timeout=10"
    "libiscsi.debug_libiscsi_eh=1"
  ];

  # Load graphics driver in stage 1
  boot.initrd.kernelModules = [ "bochs_drm" ];

  # swap file is recommended
  swapDevices = [
    {
      device = "/swapfile";
      priority = 0;
    }
  ];
}

Repartitioning target system from kexec image

Create kexec.nix file with following contents (do not add any packages to environment.systemPackages or it won't boot on 1GB system):

{ config, pkgs, ... }:
{
  imports = [
    # this will work only under qemu, uncomment next line for full image
    # <nixpkgs/nixos/modules/installer/netboot/netboot-minimal.nix>
    <nixpkgs/nixos/modules/installer/netboot/netboot.nix>
    <nixpkgs/nixos/modules/profiles/qemu-guest.nix>
  ];

  # stripped down version of https://github.com/cleverca22/nix-tests/tree/master/kexec
  system.build = rec {
    image = pkgs.runCommand "image" { buildInputs = [ pkgs.nukeReferences ]; } ''
      mkdir $out
      cp ${config.system.build.kernel}/${config.system.boot.loader.kernelFile} $out/kernel
      cp ${config.system.build.netbootRamdisk}/initrd $out/initrd
      nuke-refs $out/kernel
    '';
    kexec_script = pkgs.writeTextFile {
      executable = true;
      name = "kexec-nixos";
      text = ''
        #!${pkgs.stdenv.shell}
        set -e
        ${pkgs.kexectools}/bin/kexec -l ${image}/kernel --initrd=${image}/initrd --append="init=${builtins.unsafeDiscardStringContext config.system.build.toplevel}/init ${toString config.boot.kernelParams}"
        sync
        echo "executing kernel, filesystems will be improperly umounted"
        ${pkgs.kexectools}/bin/kexec -e
      '';
    };
    kexec_tarball = pkgs.callPackage <nixpkgs/nixos/lib/make-system-tarball.nix> {
      storeContents = [
        {
          object = config.system.build.kexec_script;
          symlink = "/kexec_nixos";
        }
      ];
      contents = [ ];
      compressCommand = "cat";
      compressionExtension = "";
    };
    kexec_tarball_self_extract_script = pkgs.writeTextFile {
      executable = true;
      name = "kexec-nixos";
      text = ''
        #!/bin/sh
        set -eu
        ARCHIVE=`awk '/^__ARCHIVE_BELOW__/ { print NR + 1; exit 0; }' $0`
        tail -n+$ARCHIVE $0 | tar x -C /
        /kexec_nixos $@
        exit 1
        __ARCHIVE_BELOW__
      '';
    };
    kexec_bundle = pkgs.runCommand "kexec_bundle" { } ''
      cat \
        ${kexec_tarball_self_extract_script} \
        ${kexec_tarball}/tarball/nixos-system-${kexec_tarball.system}.tar \
        > $out
      chmod +x $out
    '';
  };

  boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd" ];
  boot.kernelParams = [
    "panic=30" "boot.panic_on_fail" # reboot the machine upon fatal boot issues
    "console=ttyS0" # enable serial console
    "console=tty1"
  ];
  boot.kernel.sysctl."vm.overcommit_memory" = "1";

  environment.systemPackages = with pkgs; [ cryptsetup ];
  environment.variables.GC_INITIAL_HEAP_SIZE = "1M";

  networking.hostName = "kexec";

  services.getty.autologinUser = "root";
  services.openssh = {
    enable = true;
    challengeResponseAuthentication = false;
    passwordAuthentication = false;
  };

  documentation.enable = false;
  documentation.nixos.enable = false;
  fonts.fontconfig.enable = false;
  programs.bash.enableCompletion = false;
  programs.command-not-found.enable = false;
  security.polkit.enable = false;
  security.rtkit.enable = pkgs.lib.mkForce false;
  services.udisks2.enable = false;
  i18n.supportedLocales = [ (config.i18n.defaultLocale + "/UTF-8") ];

  users.users.root.openssh.authorizedKeys.keys = [
    # add your ssh key here
    "ssh-ed25519 ...."
  ];
}

Build kexec image (you'll need nix/NixOS installed on your machine):

nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_bundle -I nixos-config=./kexec.nix

Copy tarball to remote machine using scp and reboot into kexec image:

scp ./result ubuntu@somehost:/tmp/kexec
ssh ubuntu@somehost -t sudo /tmp/kexec
# wait for machine to boot and then connect
ssh root@somehost

Repartition your drive, format, mount file systems, create swap(file) and activate it as soon as possible. Check manual for more info.

@Gowee
Copy link

Gowee commented Nov 5, 2021

Great thanks for sharing this script!

I manage to install NixOS for the first time on Oracle Cloud Ubuntu 20.04 following the first way. But I notice there is a little problem:
curl https://nixos.org/nix/install returns nothing as it is a 301 redirection to https://releases.nixos.org/nix/nix-2.4/install. So I have to add a -L option to let curl follow redirections: curl -L https://nixos.org/nix/install | sh.

@misuzu
Copy link
Author

misuzu commented Nov 5, 2021

@Gowee fixed, thanks!

@griffi-gh
Copy link

griffi-gh commented Nov 20, 2024

Anyone got any luck with latest Ubuntu and nixos unstable?
I wrote my own script that does basically the same thing as this one (but using flakes for building the nixos configuration) before finding it and was only able to get to stage 1. As soon as it switches to stage 2 it's unable to read any files and panics

Screenshot_20241120_205606

boot log (stage 2 only, stage 1 omitted)
https://pastebin.com/raw/gMD35ABZ

@misuzu
Copy link
Author

misuzu commented Nov 22, 2024

I'd recommend to install the minimal system without flakes like described in the document and then convert the system to flakes.

If you want to repartition the disk there's https://github.com/nix-community/nixos-images#kexec-tarballs
I've used it recently to install NixOS with ZFS on a 1GB free tier instance, but I had to eval and build the system closure elsewhere.

@griffi-gh
Copy link

turns out boot.initrd.systemd isnt supported by nixos lustrate and should be disabled for at least the first boot.

kexec worked for me but randomly OOM'd even with a remote rebuild leaving the machine unbootable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment