Alpine LXC Container with iGPU Ollama Server on Proxmox

proxmox_alpine_lxc_ollama_igpu.md

How to setup an LXC container with AMD iGPU (Ryzen 7 5800H) passthrougth for Ollama in Proxmox

Proxmox

First we need to install the Alpine LXC, the easiest way is to use Proxmox Helper scripts: https://tteck.github.io/Proxmox/

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/alpine.sh)"

I would install in "advance" mode and be generous with the resources (8-16 cores, 16G ram, 128GB disk)

Once its installed halt the container.
Then append this into the configuration file in Proxmox to passthrough (actually bin-mount) the iGPU:

# /etc/pve/lxc/<LXC_ID>.conf
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 234:0 rwm
lxc.mount.entry: /dev/kfd dev/kfd non bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file

Alpine LXC

First we need to setup docker:

apk update
apk add docker docker-compose
rc-update add docker
service docker start

Then just create a docker-compose.yml file with this content:

version: "3.8"

services:
  ollama:
    user: root
    container_name: ollama
    image: ollama/ollama:rocm
    healthcheck:
      test: ollama --version || exit 1
      interval: 10s
    ports:
      - "11434:11434"
    restart: unless-stopped
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/kfd:/dev/kfd
    volumes:
      - ./config:/root/.ollama
    environment:
      - HSA_OVERRIDE_GFX_VERSION='9.0.0'

  owui:
    ports:
      - 80:8080
    extra_hosts:
      - host.docker.internal:host-gateway
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama
    container_name: open-webui
    restart: unless-stopped
    image: ghcr.io/open-webui/open-webui:main

volumes:
  open-webui:

To run it docker-compose up -d.

You can now access ollama (to install new models throught CLI) with this command:

docker exec -it ollama /bin/ollama
# Alternatively you can create an alias for it.
alias ollama="docker exec -it ollama /bin/ollama"

I think a more straightforward approach for GPU (Intel Quicksync) passthrough would be to use the script provided by tteck on GitHub, specifically designed for Proxmox. The command bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/plex.sh)" can be used to set it up. This method has worked well for me with my Intel NUC, utilizing Intel Quicksync. Once the setup is complete, the service can be safely deleted.

@piyush97 you shared the command to install / update Plex Media Server - are you sure this is the command?

@ChrizZz90 , install plex lxc and remove plex service from it, this script has gpu passthrough enabled and that's the reason I've been using it, then install docker and ollama as mentioned above.

did you try the openwebui script: https://community-scripts.github.io/ProxmoxVE/scripts?id=openwebui
Alternatively there is also a "hidden" ollama script: community-scripts/ProxmoxVE#202
Both scripts should also include the passthrough - I couldn't try it out yet.