Issues with NAT and multiple links

[Thakowbbery]

Good afternoon,

Gentlemen, my scenario consists of one machine and 2 links from different ISPs and my internal network (eth1 = ISP1, eth2 = ISP2, eth0 = internal network).
Iīve created 2 routing tables (1 and 2) to treat those links as well as iptablesīs NAT rules to nat outgoing connections.
The problem is: Iīve checked outgoing traffic with tcpdump -n -i eth(1 or 2) and saw lots of connections getting mixed (eth1 showed outgoing traffic with ISP2īs IP, and vice-versa). I believe thatīs whatīs causing some slow connections issues Iīve been having.

Hereīs some output of ip rule and ip route commands, as well as iptables -t nat -nv -L POSTROUTING and tcpdump:

ip route list table 1:
<ISP1_network_addr> dev eth1 scope link src <ISP1_eth1_IP>
default via <ISP1_gw_addr> dev eth1

ip route list table 2:
<ISP2_network_addr> dev eth2 scope link src <ISP2_eth2_IP>
default via <ISP2_gw_addr> dev eth2

ip route list default:
<ISP1_network_addr> dev eth1 proto kernel scope link src <ISP1_eth1_IP>
<ISP2_network_addr> dev eth2 proto kernel scope link src <ISP2_eth2_IP>
<Internal_network_addr> dev eth0 proto kernel scope link src <Internal_eth0_IP>
default
nexthop via <ISP1_gw_addr> dev eth1 weight 1
nexthop via <ISP2_gw_addr> dev eth1 weight 1

ip rule list:
0: from all lookup 255
32742: from <ISP1_network_addr> lookup 1
32743: from <ISP2_network_addr> lookup 2
(Below some rules that Iīve created for some services that must be directed throught specific links. Those work just fine, so I wonīt be showing them here)
32766: from all lookup main
32767: from all lookup default

iptables -t nat -nv -L POSTROUTING (without byte count - those rules work, since the byte count is not zero):
SNAT 0 -- * eth2 <Internal_IP_3> 0.0.0.0/0 to:<ISP_eth2_IP_3>
SNAT 0 -- * eth2 <Internal_IP_2> 0.0.0.0/0 to:<ISP_eth2_IP_2>
SNAT 0 -- * eth2 <Internal_IP_1> 0.0.0.0/0 to:<ISP_eth2_IP>
SNAT 0 -- * eth1 <Internal_IP_1> 0.0.0.0/0 to:<ISP_eth1_IP>
SNAT 0 -- * eth1 <Internal_IP_2> 0.0.0.0/0 to:<ISP_eth1_IP_2>
SNAT 0 -- * eth1 <Internal_IP_3> 0.0.0.0/0 to:<ISP_eth1_IP_3>

tcpdump -n -i eth1 | grep <3_first_ISP2_network_addr_octets>:
14:31:45.712821 IP <ISP2_eth2_IP>.50994 > <some_external_IP>.443: P 1288183484:1288183506(22) ack 1657430099 win 65535

tcpdump -n -i eth2 | grep <3_first_ISP1_network_addr_octets>:
14:33:01.609178 IP <ISP1_eth1_IP>.40358 > <some_other_external_IP>.443: P 3099327519:3099327907(38 ack 2329044274 win 4006 <nop,nop,timestamp 104355152 3347211>

And thatīs it. Outgoing connections through eth2 using eth1 IP, and vice-versa.
Iīve tried to check where is the problem, but its beyond my current limit.

I just hope someone out there sees this and help me out. I really donīt expect to find a solution right away, but at least some ideas or suggestions.

Thank you for reading this.

[jwpaek]

Hi.
I read your post. And I met same problem. I am using two ethernet. And sometimes eth1 use eth2 IP address.
Did you find reasone? Did you fix it?
I need your comment.
BRs.