Guide: An Ubuntu WiFi Secure Captive Portal: Network Manager / DNSMasq / HA Proxy / Let's Encrypt

An Ubuntu WiFi Secure Captive Portal.md

An Ubuntu WiFi Secure Captive Portal

Improvements, suggestions & fixes are welcome!

Captive portals can be a pain. Here's an opinionated and no-doubt entirely imperfect guide to setting one up for a WiFi access point on Ubuntu (tested on 20+), utilising Network Manager, DNSMasq, HA Proxy and (optionally) Let's Encrypt for a secure, locally hosted landing page.

Note: This setup was originally designed for an offline WLAN, providing access to a small number of locally hosted domains ... think the WiFi media portal on a flight or boat. If you are looking to provide internet access behind a captive portal then this guide won't get you all the way there. That said, many routers have this capability built in, as do any number of open source router firmware solutions. So you probably don't need to roll your own. If you'd like to try anyway, Ha Proxy Stick Tables would probably come in handy. Very happy to update the guide with any provided working examples.

Before you start

• It's probably best to start with a clean Ubuntu install. This setup was originally implemented on 20.04. I'll leave that bit up to you.
• Security hardening is not covered, nor webservers (beyond a proof of concept), or any other basic system setup items like local networking or remote access. You should look into these.
• The domain in use across the guide is https://captivity.bytes ... make sure to replace this with your own.
• This guide uses a https endpoint for the landing page. For this to work you will need a registered domain (hint: sub domain) where you control the DNS (it leverages DNS auth for certificate installation). If you'd like to stick with the easier non-https option, just skip the final section "Securing our Portal with Let's Encrypt".

Some useful resources:

• Wireless Broadband Alliance: Captive Network Portal Behavior
• Apple: How to modernize your captive network

Part 1: Ubuntu Networking with Netplan / Network Manager / DNSMasq

1. First, let's ensure Network Manager is installed (e.g $ sudo apt install network-manager)
2. Modern Ubuntu uses netplan for network configurations .. we'll use it to set up a WiFi access point:

# /etc/netplan/wifi.yaml
network:
  version: 2
  renderer: NetworkManager
  wifis:
    wlp4s0:
      addresses:
        - 192.168.10.1/24
      access-points:
        "Captivity_Bytes":
          password: "freedom"
          mode: ap

3. Next is DNS, we'll tell Network Manager to use DNSMasq by adding dns=dnsmasq under [main] in the config e.g:

# /etc/NetworkManager/NetworkManager.conf
[main]
dns=dnsmasq

4. We want DNSMasq to forward all traffic via our local IP (caveats? tell me more). We also advertise our "modern" captive portal here:

# /etc/NetworkManager/dnsmasq-shared.d/00-redirects.conf

# some global options
domain-needed
bogus-priv
expand-hosts

# advertise modern captive portal (update the hostname, leaving the "/captive-api" endpoint)
dhcp-option-force=114,https://captivity.bytes/captive-api

# Wildcard redirect
address=/#/192.168.10.1

5. Finally we'll apply our netplan $ netplan --debug apply and, all ok, restart ubuntu
6. WiFi access point is a go-go! Jump on your slick apple or android device (others?), open WiFi and see if you can find your no-doubt radly-named SSID. Not there? Explode it all.

Part 2: HAProxy for the win

HAProxy is great - simple and flexible. We'll be using it for all the heavy lifting in our captive portal, to "reverse proxy" / carefully map out each weird and wonderful captive scenario to the appropriate portal-enabling response.

Note that this section utilises a bunch of files attached to the gist, which you're expected to copy into appropriate locations at appropriate times. You might also notice mentions of HTTPS in haproxy.cfg .. you can safely ignore these until you choose to follow the "Securing our Portal with Let's Encrypt" section (further down the guide).

1. Let's install HAProxy: $ sudo apt install haproxy
2. To be overly cautious we'll make a copy of the default config before we meddle: $ sudo cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.old
3. Insurance policy in-hand we can now update the original config with our own carefully curated haproxy.cfg (attached below). Have a close look and note how the OS-specific 'captive portal endpoint matching logic' works. So simple. So much fun. Does linux/canonical/samsung have their own endpoints? Why not.
4. Next we need to create a directory for our captive portal voodoo magic ($ sudo mkdir /etc/haproxy/captive/) and copy each of the attached captive-* files into it. That's captive-modern.http, captive-android.http, captive-android.lst, captive-apple.http and captive-apple.lst. The *.lst matches endpoints called by the OS, *.http is the required response. You'll see that all these files are referenced in the config.
5. Finally we can test our config $ /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c and then reload $ sudo service haproxy reload
6. Config passed the test? HAProxy running? Boom! No? Bin fire.

Part 3: Testing with a Simple Local Server over WiFi

A Simple Local Server

1. Ok so we'll keep this real simple. Let's make a directory ($ mkdir ~/www) and an (admittedly invalid) html file: $ echo "captivity bytes" >> ~/www/index.html
2. Next let's serve it with pre-installed Python (on port 8001, per the portal backend on line 78 of our HAProxy config) $ python3 -m http.server 8001 -d ~/www
3. You can check the server is working locally in another terminal session: $ curl 127.0.0.1:8001
4. Even better, use HAProxy's stats console. Browse to {NETWORK_IP}:8888 from another device on the same network (username and pass are on the last line in haproxy.cfg). The portal backend should be up in lights.

Over WiFi

1. Ok, now for the good stuff. Grab your slick device and hit connect on your new WiFi creation". All going to plan you should get a notification almost immediately stating that you "need to sign in" / "no internet" / somesuch. No notification? Try opening a browser. Still nothing? Try visiting a website. Still no banana? Throw your device on the floor, stamp on it with your heel, and see if that fixes things. >> "captivity bytes"

Part 4: Securing our Portal with Let's Encrypt

We have a working captive portal! To secure it we'll use Let's Encrypt / Certbot to obtain and automatically renew a certificate, authenticating via DNS, for a custom domain that we own. If you don't own a custom domain (or don't want a secure portal) you can skip this step like you might do breakfast, keeping in mind that breakfast is (arguably) the most important meal of the day.

Apart from security, HTTPS connections are great because they unlock a number of useful browser-level features for our portal (such as storage and service workers) and generally just calm suspicious minds.

We're using DNS authentication so we can prove ownership without a publicly accessible web server. This also gives us the option of creating multiple offline deployments with certificates issued for the same domain (or sub domain). Note that ongoing intermittent internet access (< 3 monthly) is necessary to ensure our certificates don't expire.

The following steps are a combination / mutation of more detailed guides available in any number of places on the web. For example Certbot's own guide (select the "wildcard" tab), Digital Ocean's: How To Acquire a Let's Encrypt Certificate Using DNS Validation with acme-dns-certbot on Ubuntu 18.04 and Skarlso's: How to HTTPS with Hugo LetsEncrypt and HAProxy. If you want more detail please check these out.

Certbot Preparation

1. We're using snap (because it's recommended by certbot) so we'll make sure it's current: $ sudo snap install core; sudo snap refresh core
2. ... next we'll install certbot with full system access (classic): $ sudo snap install --classic certbot
3. ... ensure it can be executed: $ sudo ln -s /snap/bin/certbot /usr/bin/certbot
4. ... and ensure our dns plugin (see next step) also has the full system access it needs: $ sudo snap set certbot trust-plugin-with-root=ok

DNS provider

Certbot has a bunch of DNS plugins for popular providers, find yours in the list then follow the provided steps to install and retrieve the appropriate credentials. The steps for Cloudflare are outlined below.

1. First we'll install our provider-specific dns plugin (full list here): $ sudo snap install certbot-dns-cloudflare
2. Next, we'll need to retrieve credentials from our DNS provider (Cloudflare in this case) and secure them somewhere: $ sudo chmod 600 ~/.secrets/certbot/cloudflare.ini

Cert installation & renewal

1. Before we can use a certificate in HAProxy, we need to combine two of the Certbot-issued certificate files into one, stored in a HAProxy-friendly location ($ sudo mkdir /etc/haproxy/certs/). We'll do this when the certificate is created and then each time it renews. Let's setup a special post-hook file to do just that (referenced in our Certbot command at step 2, below):

#!/usr/bin/env bash
# /etc/letsencrypt/renewal-hooks/post/haproxy.sh

# concatenate cert files
DOMAIN='captivity.bytes' sudo -E bash -c 'cat /etc/letsencrypt/live/$DOMAIN/fullchain.pem /etc/letsencrypt/live/$DOMAIN/privkey.pem > /etc/haproxy/certs/$DOMAIN.pem'

# reload  HAProxy
service haproxy reload

2. Almost there. Once we've made our post-hook executable ($ sudo chmod +x /etc/letsencrypt/renewal-hooks/post/haproxy.sh) it's time to generate our certificate (ensure you update the domain after -d below):

$ sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials ~/.secrets/certbot/cloudflare.ini \
  --post-hook /etc/letsencrypt/renewal-hooks/post/haproxy.sh \
  -d captivity.bytes

3. Bingo! Everything going to plan Certbot should have downloaded your new certificates into /etc/letsencrypt/live/ and run your post-hook to create a new, combined cert for HAProxy. Check it: sudo ls -l /etc/haproxy/certs/
4. Because Certbot is awesome our certificates will be set to automatically renew within 90 days (you can check this: $ systemctl list-timers '*certbot*' and $ sudo certbot renew --dry-run), at which point our post-hook will run again and update the combined cert. Winning.

Updating HAProxy for HTTPS

You might have noticed a number of commented lines in the HAProxy config that were specific to https. These basically instruct you to comment/uncomment specific lines in the context to enable a secure connection. Now all the ingredients are in place, this is all that's left to do.

1. Follow the instructions to edit our HAProxy configuration on lines 39, 55, 62 & 81 of /etc/haproxy/haproxy.cfg.
2. Edit /etc/haproxy/captive/captive-modern.http, switching the protocols under user-portal-url and venue-info-url from http => https (this may not be entirely necessary due to redirects in place ... but why not)
3. Reload HAProxy to use the updated config and resource: $ sudo service haproxy reload
4. Grab your device and re-connect to WiFi. All going to plan you should be captive and swimming in green 😎

That's it!

This guide has been hastily put together from the scraps of a larger project. Please let me know of any challenges / issues / oversights / criticisms in the comments. Improvements happily accepted. Thanks for reading.

captive-android.http

HTTP/1.1 204 No Content
Content-Length: 0

captive-android.lst

clients3.google.com
clients4.google.com
android.clients.google.com
connectivitycheck.android.com
connectivitycheck.gstatic.com
www.gstatic.com
www.google.com
www.androidbak.net

captive-apple.http

HTTP/1.0 200 OK
Cache-Control: no-cache
Connection: keep-alive
Content-Type: text/html

<HTML><HEAD><TITLE>Success</TITLE></HEAD><BODY>Success</BODY></HTML>

captive-apple.lst

captive.apple.com
www.apple.com
www.appleiphonecell.com
www.airport.us
www.ibook.info
www.itools.info
www.thinkdifferent.us
apple.com

captive-modern.http

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/captive+json

{
    "captive": true,
    "user-portal-url": "http://captivity.bytes",
    "venue-info-url": "http://captivity.bytes"
}

haproxy.cfg

# /etc/haproxy/haproxy.cfg
# HAProxy config with optional HTTPS (to enable follow steps 1-4 below)
global
  log /dev/log    local0
  log /dev/log    local1 notice
  chroot /var/lib/haproxy
  stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
  stats timeout 30s
  user haproxy
  group haproxy
  daemon

  # (these lines are likely out of date - you might consider updating them)
  # See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
  ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE->
  ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
  ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
  tune.ssl.default-dh-param 2048

defaults
  log     global
  mode    http
  option  httplog
  option  dontlognull
  timeout connect 5000
  timeout client  50000
  timeout server  50000
  errorfile 400 /etc/haproxy/errors/400.http
  errorfile 403 /etc/haproxy/errors/403.http
  errorfile 408 /etc/haproxy/errors/408.http
  errorfile 500 /etc/haproxy/errors/500.http
  errorfile 502 /etc/haproxy/errors/502.http
  errorfile 503 /etc/haproxy/errors/503.http
  errorfile 504 /etc/haproxy/errors/504.http

frontend localhost
  bind *:80
  ###
  # HTTPS STEP 1: uncomment line 42 below
  # - binds to port 443 with our combined certificate (don't forget to update the domain)
  ###
  # bind *:443 ssl crt /etc/haproxy/certs/captivity.bytes.pem
  mode http

  # match our primary domain
  acl is_portal hdr(host) -i captivity.bytes

  # match OS-specific captive portal checks
  acl is_captive_modern path_beg /captive-api
  acl is_captive_apple hdr(host) -f /etc/haproxy/captive/captive-apple.lst
  acl is_captive_android hdr(host) -f /etc/haproxy/captive/captive-android.lst
  # acl is_captive hdr(User-Agent) -i CaptiveNetworkSupport

  ###
  # HTTPS STEP 2: comment line 58 below and uncomment line 59
  # - redirects to our base domain if nothing matches
  ###
  http-request redirect code 302 location http://captivity.bytes if !is_portal !is_captive_apple !is_captive_android
  # http-request redirect code 302 location https://captivity.bytes if !is_portal !is_captive_apple !is_captive_android

  ###
  # HTTPS STEP 3: uncomment line 65 below
  # - redirects all requests to https
  ###
  # redirect scheme https if !{ ssl_fc } is_portal

  # send traffic to captive responses or portal
  use_backend captive_modern if is_captive_modern
  use_backend captive_apple if is_captive_apple
  use_backend captive_android if is_captive_android
  use_backend portal if is_portal

backend portal
  mode http
  balance roundrobin
  option forwardfor
  option httpchk HEAD / HTTP/1.1\r\nHost:localhost
  server portal 127.0.0.1:8001 check
  http-request set-header X-Forwarded-Port %[dst_port]
  ###
  # HTTPS STEP 4: uncomment line 84 below
  # - adds https redirect/forwarding header
  ###
  # http-request add-header X-Forwarded-Proto https if { ssl_fc }

backend captive_modern
  errorfile 503 /etc/haproxy/captive/captive-modern.http

backend captive_apple
  errorfile 503 /etc/haproxy/captive/captive-apple.http

backend captive_android
  errorfile 503 /etc/haproxy/captive/captive-android.http

# HAProxy Stats GUI - don't forget to update "user:pass" below (!)
listen stats
  bind              0.0.0.0:8888
  mode              http
  stats             enable
  option            httplog
  stats             show-legends
  stats             uri /
  stats             realm Haproxy\ Statistics
  stats             refresh 5s
  stats             auth user:pass

Trying to follow your guide, I am running into an issue with the HAProxy configuration. It looks like the captive-android.http file is invalid for some reason:

/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c -V
[ALERT] 186/073124 (1280) : config: backend 'captive_android': invalid message for HTTP return code 503: unabled to parse headers (error offset: -2).
[ALERT] 186/073124 (1280) : Fatal errors found in configuration.

This is using HA-Proxy version 2.0.31-0ubuntu0.1 2023/03/22 - https://haproxy.org/. I can "fix" it by replacing the contents of the captive-android.http file with those of the captive-apple.http file, but this will of course change the behavior on android devices...

Do you have any idea how to fix this properly?

Hey @lukas-lang thanks for trialing! Just to confirm, you're simply switching out the captive_android 503 to:

backend captive_android
  errorfile 503 /etc/haproxy/captive/captive-apple.http

Yes that should work to get HAProxy running, however you're right - older android devices expect a valid 204 response. You could try switching out the response in captive-android.http for google's default at curl --head http://clients3.google.com/generate_204:

HTTP/1.1 204 No Content
Content-Length: 0
Cross-Origin-Resource-Policy: cross-origin
Date: Thu, 06 Jul 2023 22:35:52 GMT

That said, modern devices (from Android 11 at least) will actually use the modern captive portal protocol and so will behave correctly via the captive_modern backend.

Let me know if you have any luck. I'll have a play with the latest version of HAProxy as soon as I get a chance and let you know what I find.