I’ve been asked how to check if both WANs are working on fail-over dual WAN linux box because from time to time when the fail-over triggers backup WAN was not working and nobody knew it. So, the mission here is to create a way to get Zabbix checking on it and report on failure.
Implementation
To get it working I used iptables CONNMARK, multiple ip route tables, ip rules, probe scripts and users.
routing tables
I’ve created two extra routing tables: wan1 and wan2. They just have a default route to each wan.
1 2 3
# ip ro ls table wan1 default via 192.168.1.1 dev eth1 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.2
1 2 3
# ip ro ls table wan2 default via 192.168.2.1 dev eth2 192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.2
routing rules
Created two custom rules to redirect packets with CONNMARK 0x1 and 0x2 to table wan1 or table wan2.
1 2 3 4 5 6 7
# ip ru ls 0: from all lookup local 220: from all lookup 220 32764: from all fwmark 0x2 lookup wan2 32765: from all fwmark 0x1 lookup wan1 32766: from all lookup main 32767: from all lookup default
interfaces file
To get it reconfigured after reboots I’ve used post-up on interfaces file.
# Prepara tabela de roteamento wan1 post-up ip route add default via 192.168.1.1 dev eth1 table wan1 post-up ip route add 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.2 table wan1 post-up ip rule add fwmark 1 lookup wan1
# prepara tabela de roteamento wan2 post-up ip route add default via 192.168.2.1 dev eth2 table wan2 post-up ip route add 192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.2 table wan2 post-up ip rule add fwmark 2 lookup wan2
iptables
The magic happens here. At iptables mangle table I’ve marked all packets from users testwan1 and testwan2 with their matching CONNMARK.
1 2 3 4 5 6 7 8
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -m owner --uid-owner testwan1 -j MARK --set-mark 1 iptables -t mangle -A OUTPUT -m owner --uid-owner testwan1 -j CONNMARK --save-mark
iptables -t mangle -A OUTPUT -m owner --uid-owner testwan2 -j MARK --set-mark 2 iptables -t mangle -A OUTPUT -m owner --uid-owner testwan2 -j CONNMARK --save-mark
Now all packets generated from user testwan1 are going to use eth1 and all packets from testwan2 are going to use eth2.
probe scripts
To get this information always available to Zabbix, two probe scripts have been created and executed by each user crontab every 5min.
1 2 3 4 5
# cat /usr/local/bin/get-eth1-public-ip #!/bin/sh
IP=$(curl -s icanhazip.com) echo "$IP" | tee /tmp/public-ip.wan1
1 2 3 4 5
# cat /usr/local/bin/get-eth2-public-ip #!/bin/sh
IP=$(curl -s icanhazip.com) echo "$IP" | tee /tmp/public-ip.wan2
cron
1 2 3 4
# crontab -u testwan1 -l
*/5 * * * *
1 2 3
# crontab -u testwan2 -l
*/5 * * * * /usr/local/bin/get-eth2-public-ip
Conclusion
Now, Zabbix can be configured to check files /tmp/public-ip.wan1 and /tmp/public-ip.wan2 and do whatever needed. This implementation can be enhanced to don’t use crontab in favor of setuid on probe scripts and to do not generate output files.