Instructions for installing and configuring Squid caching proxy server on Asuswrt-Merlin

asuswrt-merlin.md

Setup Entware on Asuswrt-merlin

Entware is a modern alternative to Optware.

For those unfamiliar with Optware: it's a software repository that offers various software programs that can be installed on your router. They allow you to add new functionality to your router (provided you have the know-how to properly configure them).

Entware system library is specially modified (patched) so that standard linux files that are normally located in /etc directory are now located in /opt/etc/ directory. To simplify things let's consider /etc/passwd file. On Asuswrt-merlin it normally looks something like:

admin:x:0:0:admin:/root:/bin/sh
nas:x:100:100:nas:/dev/null:/dev/null
nobody:x:65534:65534:nobody:/dev/null:/dev/null
tor:x:65533:65533:tor:/dev/null:/dev/null

We call standard installation Entware the case, when /opt/etc/passwd (shadow, group, gshadow, shells) files are symbolic links to the equivalent files in /etc directory. When Entware binaries need to access /opt/etc/passwd file they access /etc/passwd file.

We call alternative installation Entware the case when /opt/etc/passwd and /opt/etc/group are real files (not symlinks). Entware binaries will use user names, group names, passwords from these files. They can be different from Asuswrt-merlin firmware. The original Asuswrt-merlin firmware has admin as a superuser, alternative installation Entware always has root as a superuser.

Important: Please remove Optware, Entware, Entware.arm if it is currently installed.

Uninstall Optware (optional)

You cannot use both Optware and Entware at the same time. Entware cannot be used simultaneously with Optware.

Important: Asus's Download Master is based on Optware, and therefore is NOT compatible with Entware. You will have to uninstall Download Master.

After uninstalling, you should make sure asusware.arm or asusware. directory on mounted disk partition is deleted. Otherwise, Entware won't work properly.

Uninstall Entware (optional)

SSH to your router, change 192.168.1.1 with the IP address of your router

$ ssh [email protected]

Delete /tmp/opt symlink

$ rm /tmp/opt

Delete start scripts

$ rm /jffs/scripts/services-start
$ rm /jffs/scripts/services-stop
$ rm /jffs/scripts/post-mount

Disable custom scripts and configs from /jffs

$ nvram set jffs2_scripts=0
$ nvram commit

Reboot router

$ reboot

Unmount and format the USB drive

$ df -h
$ umount -l /dev/sda1
$ mkfs.ext2 /dev/sda1

Reboot router

$ reboot

Prerequisites

Important: Please check Linux kernel version of your router using uname -a command. Linux kernel version should not be less than 3.2!

$ uname -a
Linux RT-AC86U-E5F0 4.1.27 #2 SMP PREEMPT Sun Jun 28 16:02:00 EDT 2020 aarch64 ASUSWRT-Merlin

The goal is to mount /opt folder to some ext2|ext3|ext4 partition on a USB drive, which will be automatically mounted on every boot, so make sure to plug in a USB drive before installation.

In this example the target is a Asus RT-AC86U. The RT-AC86U contains a Broadcom BCM4906 SoC. It integrates two Cortex-A53 ARMv8-A cores that Broadcom has customized. Armv8-A has two execution states AArch32 and AArch64. AArch32 provides backwards compatibility with ARMv7.

This all means you can use ready made installation scripts for ARMv7.

SSH to your router, change 192.168.1.1 with the IP address of your router

$ ssh [email protected]

Change directory to /tmp/share

$ cd /tmp/share

Download entware-setup.sh from the Asuswrt-merlin NG (New Generation) code branch

$ wget https://raw.githubusercontent.com/RMerl/asuswrt-merlin.ng/a46283c8cbf2cdd62d8bda231c7a79f5a2d3b889/release/src/router/others/entware-setup.sh

For standard installation (armv7,k3.2,std)

$ sed -i 's~https://bin.entware.net/armv7sf-k2.6/installer/generic.sh~https://bin.entware.net/armv7sf-k3.2/installer/generic.sh~g' /tmp/share/entware-setup.sh

For alternative installation (armv7,k3.2,alt)

$ sed -i 's~https://bin.entware.net/armv7sf-k2.6/installer/generic.sh~https://bin.entware.net/armv7sf-k3.2/installer/alternative.sh~g' /tmp/share/entware-setup.sh

Correct permissions for /tmp/share/entware-setup.sh

$ chmod +x /tmp/share/entware-setup.sh

Unmount and format the USB drive and afterward physically reconnect the USB drive

$ df -h
$ umount -l /dev/sda1
$ mkfs.ext2 /dev/sda1

Install Entware

$ /tmp/share/entware-setup.sh

admin@RT-AC86U-E5F0:/tmp/share# /tmp/share/entware-setup.sh
 Info:  This script will guide you through the Entware installation.
 Info:  Script modifies "entware" folder only on the chosen drive,
 Info:  no other data will be changed. Existing installation will be
 Info:  replaced with this one. Also some start scripts will be installed,
 Info:  the old ones will be saved on Entware partition with name
 Info:  like /tmp/mnt/sda1/jffs_scripts_backup.tgz

 Info:  This platform supports both 64bit and 32bit Entware installations.
 Info:  64bit support is recommended, but 32bit support may be required
 Info:  if you are using other 32bit applications.
 Info:  The 64bit installation is also better optimized for newer kernels.
 =>  Do you wish to install the 64bit version? (y/n)

Choose 32bit version, in this case n

 Info:  Looking for available partitions...
[1] --> /tmp/mnt/sda1
 =>  Please enter partition number or 0 to exit

Choose a partition where Entware should be installed, in this case 1

[0-1]: 1
 Info:  /tmp/mnt/sda1 selected.

Output on a clean install should look like this

Info:  Creating /tmp/mnt/sda1/entware folder...
Info:  Creating /tmp/opt symlink...
Info:  Creating /jffs scripts backup...
Info:  Modifying start scripts...

Output on a successful install should look like this

Info: Congratulations!
Info: If there are no errors above then Entware was successfully initialized.
Info: Add /opt/bin & /opt/sbin to $PATH variable
Info: Add "/opt/etc/init.d/rc.unslung start" to startup script for Entware services to start
Info: Use ssh server from Entware for better compatibility.
Info: Found a Bug? Please report at https://github.com/Entware/Entware/issues

The script has created a new directory named entware

admin@RT-AC86U-E5F0:/tmp/share# cd /opt
admin@RT-AC86U-E5F0:/tmp/mnt/sda1/entware#

Setup (alternative installation)

If this is an alternative installation Entware, it is recommended to install and setup SSH server from Entware and use it instead of the firmware supplied one. There are two SSH servers in Entware - Dropbear and OpenSSH. You can install either Dropbear or OpenSSH.

Important: The superuser root has password 12345 that can be changed after the first login and is independent from Asuswrt-merlin firmware.

Install Dropbear

$ opkg update
$ opkg install dropbear

You need to edit /opt/etc/config/dropbear.conf configuration file and change the line PORT=22 to PORT=2222 to make Dropbear SSH port different from the system's one.

$ sed -i 's~PORT=22~PORT=2222~g' /opt/etc/config/dropbear.conf

Run Dropbear

$ /opt/etc/init.d/S51dropbear start

Check if Dropbear is currently running

$ /opt/etc/init.d/S51dropbear status
dropbear already running

SSH to your router, change 192.168.1.1 with the IP address of your router

$ ssh [email protected] -p 2222

If Dropbear disconnects immediately after successful auth, change root's shell and try again

Connection to 192.168.1.1 closed.
$ /opt/bin/sh
Segmentation fault
$ /bin/sh
$ sed -i 's~/opt/bin/sh~/bin/sh~g' /opt/etc/passwd

Notice that the PATH variable now has /opt/bin before /bin

$ echo $PATH

Entware (alternative installation) installs its own busybox copy

$ which busybox
/opt/bin/busybox

References

1. Entware – Alternative install vs standard
2. Asuswrt-merlin – Entware

entware.md

How to compile a package for Entware

For the case where Entware doesn't have the package on board that you wish to use, you have the option to compile an Entware package yourself.

Prerequisites

Setup Multipass

We are going to use Ubuntu 18.04 in Multipass.

• You can download Multipass here

Launch an instance

Note: Change the total number of CPU cores you would like to allocate.

multipass launch --name primary --cpus 4 --disk 16G --mem 4G 18.04

Open a shell inside the instance

multipass shell primary

Setup Entware on Ubuntu

Update the APT package index:

sudo apt update

Now you can install all the required packages.

Note: This list may change over time as new components are added to the firmware code.

sudo apt-get install g++ libncurses5-dev libssl-dev make python-dev unzip

Then setup Entware by cloning its repository:

$ cd $HOME
$ git clone https://github.com/Entware/Entware.git
$ cd Entware

Update Entware packages feed

$ make package/symlinks

Setup the configuration for the target device

In this example the target is a Asus RT-AC86U. The RT-AC86U contains a Broadcom BCM4906 SoC. It integrates two Cortex-A53 ARMv8-A cores that Broadcom has customized. Armv8-A has two execution states AArch32 and AArch64. AArch32 provides backwards compatibility with ARMv7.

This all means you can use a ready made configuration file that is located in the 'configs' directory with the name armv7-3.2.config:

$ cp -v configs/armv7-3.2.config .config

Prepare build environement

The full execution on OpenWrt Buildroot a.k.a. rebuild repo a.k.a. make is not necessary. You only need:

$ make tools/install
$ make toolchain/install

You can speed up compilation on multiprocessor systems by running several build threads simultaneously by adding -jN after the make command:

$ make -j4 tools/install
$ make -j4 toolchain/install

Error in tools or toolchain

In case there is an error in the tools/install or toolchain/install phase, and you change your configuration, remember to run a make dirclean instead of make clean.

Compile squid package

Now you should be ready to compile an individual package. For starters you can test make package/.../compile using a tiny package, for example squid:

$ make -j4 package/squid/compile

If something goes wrong, turn on verbose mode by adding V=s:

$ make -j1 package/squid/compile V=s

When everything succeeds your package ends up in a subdirectory of the Entware 'bin' directory. For this ARMv7 example, you should find the .ipk file at 'bin/targets/armv7-3.2/generic-glibc/packages':

$ ls bin/targets/armv7-3.2/generic-glibc/packages | grep squid
squid-mod-cachemgr_4.13-1_armv7-3.2.ipk
squid_4.13-1_armv7-3.2.ipk

Install compiled squid package

Change to the directory where you have the squid_4.13-1_armv7-3.2.ipk file

$ cd bin/targets/armv7-3.2/generic-glibc/packages

Copy the .ipk file to '/tmp/share' directory, change 192.168.1.1 with the ip address of your router

$ scp squid_4.13-1_armv7-3.2.ipk [email protected]:/tmp/share

SSH to your router, change 192.168.1.1 with the ip address of your router

$ ssh [email protected]
$ opkg remove squid (if previously installed)
$ opkg install /tmp/share/squid_4.13-1_armv7-3.2.ipk
$ opkg info squid

The output from the squid -v should start with something like:

Squid Cache: Version 4.13

Correct permissions to '/dev/shm' directory to fix error: squid: Ipc::Mem::Segment::create failed to shm_open(/squid-cf__metadata.shm): (13) Permission denied

$ sed -i '/PREARGS=""/a PRECMD="chmod 777 /dev/shm"' /opt/etc/init.d/S22squid

Run squid

$ /opt/etc/init.d/S22squid start
Starting squid...              done.

Check if squid is currently running

$ /opt/etc/init.d/S22squid check
Checking squid...              alive.

Run startup script (optional)

You might want to start your package automatically when the router starts. For this purpose the router does execute scripts located in the directory '/opt/etc/init.d/'.

Entware init system

Entware uses an 'init.d' directory (where init is short for initialization) containing scripts. Entware uses "rc.func" as a wrapper to provide its main and default functionality. Entware also uses "rc.unslung" as the helper script to start and stop all '/opt/etc/init.d/' scripts. To be more precise, rc.unslung will process all executable files that start with a capital S and will do that in sorted order. Except for stopping, restarting and killing, then the executable S* init script files will be processed in reverse sorted order.

rc.func

Learn yourself bash scripting and have a look inside "rc.func" to understand its functionality. By this rc.func template, the available commands for any init script are as follows:

start         Start the service
stop          Stop the service
restart       Restart the service
check         Check the service (dead or alive)
kill          Kill the service
reconfigure   Reload configuration files (sends SIGHUP signal)

All these arguments can be passed to the script when run. For example, to stop squid call it with stop:

# /opt/etc/init.d/S22squid stop

Entware init script

An example Entware init script, S22squid from the net package, looks like this:

#!/bin/sh

ENABLED=yes
PROCS=squid
ARGS=""
PREARGS=""
DESC=$PROCS
PATH=/opt/sbin:/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

. /opt/etc/init.d/rc.func

References

1. Entware – Compile packages from sources
2. Entware – How to add a new package

Useful links

1. Entware Packages list
2. Entware Packages list (archive)
3. Entware-ng Packages list (deprecated)
4. Entware-ng-3x Packages list (deprecated)
5. Squid Versions

setup.md

ssh

iptables

#iptables -A PREROUTING -t mangle -p tcp --dport 80 -s 192.168.1.1 ! -d `nvram get lan_ipaddr`/`nvram get lan_netmask` -j ACCEPT
#iptables -A PREROUTING -t mangle -p tcp --dport 80 -j MARK --set-mark 3
#ip rule add fwmark 3 table 2
#ip route add default via 192.168.1.1 dev br0 table 2
#iptables -A PREROUTING -t mangle -p tcp --dport 80 -s 192.168.1.xxx -j ACCEPT
#iptables -A PREROUTING -t nat -i eth0 -p --dport 80 -j REDIRECT --to-port 3128

squid

opkg install squid
sed -i '/PREARGS=""/a PRECMD="chmod 777 /dev/shm"' /opt/etc/init.d/S22squid
nano /opt/etc/squid/squid.conf
squid -k parse
/opt/etc/init.d/S22squid start
/opt/etc/init.d/S22squid check

config

mkdir /opt/var/log/squid/
nano /opt/etc/squid/squid.conf
squid -k reconfigure

commands

#squid -k rotate
#squid -k reconfigure
#squid -k kill

log

tail -f /opt/var/log/squid/access.log | grep -i 192.168.1.xxx

nvram

nvram get wan_ipaddr
nvram get lan_ipaddr
nvram get lan_netmask

opkg

opkg update
opkg list_installed
opkg list-upgradable

debug

cat /etc/resolv.conf
free -m
time nslookup -debug -d2 8.8.8.8
watch 'dmesg | tail -20'
ls -la /tmp/mnt
netstat -tuln
rm /opt/var/run/squid.pid

uninstall

opkg status squid
opkg remove --autoremove squid

performance

opkg install procps-ng-top
top -p `pgrep -d, -f squid`
top -p `pgrep -d, -f program1`, `pgrep -d, -f program2` (upper case P) makes top order by CPU

squid.conf

acl localnet src 192.168.1.0/24
acl ssl_ports port 443
acl safe_ports port 80
acl safe_ports port 443
acl CONNECT method CONNECT

http_access deny !safe_ports
http_access deny CONNECT !ssl_ports

http_access allow localhost manager
http_access deny manager

http_access deny to_localhost

http_access allow localnet
http_access allow localhost

http_access deny all

#http://www.squid-cache.org/Doc/config/logformat/
#logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt
#logformat common %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st %Ss:%Sh
#logformat combined %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
#logformat referrer %ts.%03tu %>a %{Referer}>h %ru
#logformat useragent %>a [%tl] "%{User-Agent}>h"

#logformat something %tl %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<A %mt
#access_log stdio:/opt/var/log/squid/access.log something

access_log none
cache_log /dev/null
logfile_daemon /dev/null
logfile_rotate 2

http_port 192.168.1.1:3128

buffered_logs on
cache_mem 64 MB
dns_nameservers 192.168.1.1
forwarded_for delete
ignore_unknown_nameservers off
pipeline_prefetch 6
shutdown_lifetime 1 seconds
strip_query_terms off
via off

# Request Headers Forcing

request_header_access Accept allow all
request_header_access Accept-Charset allow all
request_header_access Accept-Encoding allow all
request_header_access Accept-Language allow all
request_header_access Allow allow all
request_header_access Authorization allow all
request_header_access Cache-Control allow all
request_header_access Connection allow all
request_header_access Content-Encoding allow all
request_header_access Content-Language allow all
request_header_access Content-Length allow all
request_header_access Content-Range allow all
request_header_access Content-Type allow all
request_header_access Cookie allow all
request_header_access Date allow all
request_header_access Expires allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Last-Modified allow all
request_header_access Location allow all
request_header_access Mime-Version allow all
request_header_access Pragma allow all
request_header_access Proxy-Authenticate allow all
request_header_access Proxy-Authorization allow all
request_header_access Proxy-Connection allow all
request_header_access Range allow all
request_header_access Referer allow all
request_header_access Retry-After allow all
request_header_access Title allow all
request_header_access User-Agent allow all
request_header_access WWW-Authenticate allow all
request_header_access All deny all

# Response Headers Spoofing

reply_header_access Via deny all
reply_header_access X-Cache deny all
reply_header_access X-Cache-Lookup deny all

My USB needed to be formatted prior to install on PC as exFAT.

FAT gave the issue:
/dev/sda is apparently in use by the system; will not make a filesystem here!