Skip to content

Instantly share code, notes, and snippets.

@shigenobuokamoto
Last active November 14, 2024 12:14
Show Gist options
  • Save shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0 to your computer and use it in GitHub Desktop.
Save shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0 to your computer and use it in GitHub Desktop.
/etc/systemd/system/network-mirrored.service
[Unit]
Wants=network-pre.target
Before=network-pre.target shutdown.target
[Service]
User=root
ExecStart=/bin/sh -ec '\
[ -x /usr/bin/wslinfo ] && [ "$(/usr/bin/wslinfo --networking-mode)" = "mirrored" ] || exit 0;\
echo "\
add chain ip nat WSLPREROUTING { type nat hook prerouting priority dstnat - 1; policy accept; };\
insert rule ip nat WSLPREROUTING iif loopback0 ip daddr 127.0.0.1 counter dnat to 127.0.0.1 comment mirrored;\
"|nft -f -\
'
ExecStop=/bin/sh -ec '\
[ -x /usr/bin/wslinfo ] && [ "$(/usr/bin/wslinfo --networking-mode)" = "mirrored" ] || exit 0;\
for chain in "ip nat WSLPREROUTING";\
do\
handle=$(nft -a list chain $chain | sed -En "s/^.*comment \\"mirrored\\" # handle ([0-9]+)$/\\1/p");\
for n in $handle; do echo "delete rule $chain handle $n"; done;\
done|nft -f -\
'
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
@Adam-Jin
Copy link

I saw your description in the issue "adding this action breaks the prerouting hook and disables any Docker rules set in the PREROUTING chain". Can you explain why? The dnat of 127.0.0.1 to 127.0.0.1 seems redundant. thx

@shigenobuokamoto
Copy link
Author

@Adam-Jin
as you said, it is redundant.
nothing changes with this DNAT, except that it can terminate the prerouting hook.
PREROUTING is not processed by applying DNAT first.

@shigenobuokamoto
Copy link
Author

this is the reason why PERROUTING (Docker's rules) is handled poorly in mirrored networking.
microsoft/WSL#10494 (comment)

explanation about systemd service can be found here.
microsoft/WSL#10494 (comment)

@Adam-Jin
Copy link

thx

@KingOfGitH
Copy link

Everything looks fine, but it doesn't work. Access via lan is fine. Do you have any ideas?
image

@shigenobuokamoto
Copy link
Author

@KingOfGitH
could you please check with the script below?

echo "NETWORKINGMODE:" $(wslinfo --networking-mode)
echo "LOOPBACK:      " $(ip r list table 127)
ps ax | sed -En 's/^.*docker-proxy.*host-ip ([0-9.]+) -host-port ([0-9]+).*$/PROXY PORTS:    \1:\2/p'  
echo "NFT RULE:      " $(sudo nft list ruleset 2>/dev/null | grep mirrored)

@KingOfGitH
Copy link

KingOfGitH commented May 14, 2024

@KingOfGitH could you please check with the script below?

echo "NETWORKINGMODE:" $(wslinfo --networking-mode)
echo "LOOPBACK:      " $(ip r list table 127)
ps ax | sed -En 's/^.*docker-proxy.*host-ip ([0-9.]+) -host-port ([0-9]+).*$/PROXY PORTS:    \1:\2/p'  
echo "NFT RULE:      " $(sudo nft list ruleset 2>/dev/null | grep mirrored)

of course

output is

root@:host~# echo "NETWORKINGMODE:" $(wslinfo --networking-mode)
NETWORKINGMODE: mirrored
root@:host~# echo "LOOPBACK: " $(ip r list table 127)
LOOPBACK: 127.0.0.1 via 169.x.x.x dev loopback0 proto kernel src 127.0.0.1 onlink
root@:host~# ps ax | sed -En 's/^.*docker-proxy.host-ip ([0-9.]+) -host-port ([0-9]+).$/PROXY PORTS: \1:\2/p'
PROXY PORTS: 0.0.0.0:9443
PROXY PORTS: 0.0.0.0:5432
PROXY PORTS: 0.0.0.0:9000
PROXY PORTS: 0.0.0.0:8000
root@:host~# echo "NFT RULE: " $(sudo nft list ruleset 2>/dev/null | grep mirrored)
NFT RULE: iif "loopback0" ip daddr 127.0.0.1 counter packets 25 bytes 1300 dnat to 127.0.0.1 comment "mirrored"

@shigenobuokamoto
Copy link
Author

@KingOfGitH
thank you for reporting

there seems to be no problem with parts a, b, c, and d.

Windows Host --------> (virtual network) ----------> Ubuntu ------> container
                                                       a       b
                                              loopback0 ----> docker-proxy
                                                               c
                                    <------------------
                                169.254.73.152
                                    d (probably ok)

iif "loopback0" ip daddr 127.0.0.1 counter packets 25 bytes 1300 dnat to 127.0.0.1

25 packets are being processed, traffic from the Windows Host is being forwarded to the 127.0.0.1(= docker-proxy process) on the Ubuntu.

so, the purpose of netwrok-mirroed.service appears to be working correctly.
there may be a problem with the return traffic from the virtual network to Windows.

Windows Host <----------- (virtual network)
                  e?

could you also check this?

docker compose stop
python3 -m http.server 9000
/mnt/c/Windows/System32/curl.exe http://localhost:9000

if curl.exe cannot access to 9000, there is a problem with communication on localhost regardless of Docker.
in that case, the cause may be Windows security software.

i do not think there were any particular problems with WSL 2.1.5, but please check 2.2.4 just to be sure.

wsl --update --pre-release

@KingOfGitH
Copy link

image

The problem really has nothing to do with docker. I turn off the firewall, whether it is windows or wsl. The wsl version has also been upgraded. It's so sad that nothing has changed

@KingOfGitH
Copy link

KingOfGitH commented May 15, 2024

root@xxx:~# hostname -i
127.0.1.1
root@xxx:~# hostname -I
10.0.0.147 172.18.0.1 172.17.0.1
root@xxx:~# cat /etc/hosts
# This file was automatically generated by WSL. To stop automatic generation of this file, add the following entry to /etc/wsl.conf:
# [network]
# generateHosts = false
127.0.0.1       localhost
127.0.1.1       myhost.  myhost

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
root@xxx:~# cat /etc/resolv.conf
# This file was automatically generated by WSL. To stop automatic generation of this file, add the following entry to /etc/wsl.conf:
# [network]
# generateResolvConf = false
nameserver 10.255.255.254
search lan
root@earth:~# ifconfig
br-9154cb03c575: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255
        ether 02:42:8e:9e:16:1f  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2  bytes 180 (180.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:84:ec:9d:e5  txqueuelen 0  (Ethernet)
        RX packets 740  bytes 2556700 (2.5 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 544  bytes 40789 (40.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.147  netmask 255.255.255.0  broadcast 10.0.0.255
        inet6 fe80::23e5:f7e7:e75f:b020  prefixlen 64  scopeid 0x20<link>
        ether d8:43:ae:24:60:63  txqueuelen 1000  (Ethernet)
        RX packets 1018  bytes 290244 (290.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 900  bytes 2591782 (2.5 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 71  bytes 5170 (5.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 71  bytes 5170 (5.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

loopback0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:15:5d:f1:17:2c  txqueuelen 1000  (Ethernet)
        RX packets 140  bytes 10220 (10.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 109  bytes 6366 (6.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth2195c28: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 7e:35:1f:a8:90:b9  txqueuelen 0  (Ethernet)
        RX packets 740  bytes 2567060 (2.5 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 544  bytes 40789 (40.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vethd16839b: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 6e:6e:0d:17:4f:04  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2  bytes 180 (180.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I have turned off all firewalls, including Hyper-V. Then I found that the problem should be that the port between wsl2 and the host in mirrored network mode cannot communicate. I looked for some related network configuration. I'm not sure if there's something wrong with that.

Anyway, thank you very much. It is indeed not a problem with network-mirrored.service. I think I need to find other related issues

@shigenobuokamoto
Copy link
Author

@KingOfGitH
when configuring network settings such as NetworkManager on Linux, wsl may not be able to communicate, but i do not see anything wrong with the information provided.
send logs to https://github.com/microsoft/WSL/issues and microsoft may be able to provide a solution.

@trungly1
Copy link

trungly1 commented Jun 6, 2024

you can try it.

$ sudo systemctl --now enable network-mirrored

Very grateful for this, spent countless hours trying to figure it out, no amount of gpt could save me.

@shigenobuokamoto
Copy link
Author

this script is no longer needed.

please use Docker 27.3.0 or later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment