Hi,

I m trying to setup a feature on our centos router to enable advanced routing policies based on port.
The router has 2 wan connections, what I want first is the vpn traffic to be routed via the wan2 and the regular traffic via wan1.
I already did it successfully in forwarding mode but I encountered some troubles when the router itself initiates the connection.

Here is my configuration:
> uname -a
Linux <hostname> 2.6.18-238.9.1.el5.centos.plus #1 SMP Tue Apr 12 20:34:33 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

iptables tags:
iptables -A POSTROUTING -p udp --dport 1194 -j MARK --set-mark 7
iptables -A OUTPUT -p udp --dport 1194 -j MARK --set-mark 7
iptables -A FORWARD -p udp --dport 1194 -j MARK --set-mark 7

> route -n
...
...
0.0.0.0 <ip_gw1> 0.0.0.0 UG 0 0 0 eth0

> ip route show table vpn
default via <ip_gw2> dev eth1

> cat /etc/iproute2/rt_tables
255 local
254 main
253 default
200 vpn
0 unspec

> ip rule
0: from all lookup 255
500: from all fwmark 0x7 lookup vpn
32766: from all lookup main
32767: from all lookup default

Here comes the troubles:
the router cannot initiate the vpn tunnel

Debug attemps:
Hi tried with tcp 1194 port instead of the udp for easier troubleshooting and here is the output

#on the local router
> tcpdump -i any -nn tcp port 1194 and host <ip_wan2>
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
13:26:15.107253 IP <ip_wan2>.60064 > <ip_distant_server>.1194: S 1315331880:1315331880(0) win 5840 <mss 1460,nop,wscale 7>
13:26:15.111063 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:26:18.106463 IP <ip_wan2>.60064 > <ip_distant_server>.1194: S 1315331880:1315331880(0) win 5840 <mss 1460,nop,wscale 7>
13:26:18.111134 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:26:18.115007 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:26:24.107214 IP <ip_wan2>.60064 > <ip_distant_server>.1194: S 1315331880:1315331880(0) win 5840 <mss 1460,nop,wscale 7>
13:26:24.111147 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:26:24.117008 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:26:36.114991 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>

#on the distant server
> tcpdump -i any -nn tcp port 1194 and host <ip_wan2>
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
13:27:14.577336 IP <ip_wan2>.60064 > <ip_distant_server>.1194: S 1315331880:1315331880(0) win 5840 <mss 1460,nop,wscale 7>
13:27:14.577367 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:27:17.576623 IP <ip_wan2>.60064 > <ip_distant_server>.1194: S 1315331880:1315331880(0) win 5840 <mss 1460,nop,wscale 7>
13:27:17.576642 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:27:17.580788 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:27:23.577298 IP <ip_wan2>.60064 > <ip_distant_server>.1194: S 1315331880:1315331880(0) win 5840 <mss 1460,nop,wscale 7>
13:27:23.577318 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:27:23.581572 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:27:35.581148 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>
13:27:59.781287 IP <ip_distant_server>.1194 > <ip_wan2>.60064: S 3456487863:3456487863(0) ack 1315331881 win 5840 <mss 1460,nop,wscale 7>

I m not a tcpdump guru but it seems that the two servers can see each others but cannot communicate as they are stuck at the synchronization step. It seems that the router received the tcp ACK packet but does not take it into account as it always send the same TCP sequence number...
For the reference, when I fix the route to the wan1 like that:

> ip route show table vpn
default via <ip_gw1> dev eth0

It works as wanted:

#on the local router
> tcpdump -i any -nn tcp port 1194 and host <ip_wan1>
the connection can establish with success:
13:40:17.441602 IP <ip_gw1>.47859 > <ip_distant_server>.1194: F 1986705812:1986705812(0) ack 4125140739 win 46
13:40:17.645706 IP <ip_distant_server>.1194 > <ip_gw1>.47859: . ack 1 win 46
13:40:18.018062 IP <ip_gw1>.36262 > <ip_distant_server>.1194: S 2202443917:2202443917(0) win 5840 <mss 1460,nop,wscale 7>
13:40:18.021604 IP <ip_distant_server>.1194 > <ip_gw1>.36262: S 46505363:46505363(0) ack 2202443918 win 5840 <mss 1460,nop,wscale 7>
13:40:18.021624 IP <ip_gw1>.36262 > <ip_distant_server>.1194: . ack 1 win 46
13:40:23.849964 IP <ip_gw1>.36262 > <ip_distant_server>.1194: P 1:8(7) ack 1 win 46
13:40:23.855672 IP <ip_distant_server>.1194 > <ip_gw1>.36262: . ack 8 win 46
13:50:23.850760 IP <ip_gw1>.36262 > <ip_distant_server>.1194: F 8:8(0) ack 1 win 46
13:50:23.894575 IP <ip_distant_server>.1194 > <ip_gw1>.36262: . ack 9 win 46

#on the distant server
> tcpdump -i any -nn tcp port 1194 and host <ip_wan1>
13:41:16.913368 IP <ip_gw1>.47859 > <ip_distant_server>.1194: F 1986705812:1986705812(0) ack 4125140739 win 46
13:41:17.113811 IP <ip_distant_server>.1194 > <ip_gw1>.47859: . ack 1 win 46
13:41:17.489619 IP <ip_gw1>.36262 > <ip_distant_server>.1194: S 2202443917:2202443917(0) win 5840 <mss 1460,nop,wscale 7>
13:41:17.489647 IP <ip_distant_server>.1194 > <ip_gw1>.36262: S 46505363:46505363(0) ack 2202443918 win 5840 <mss 1460,nop,wscale 7>
13:41:17.493062 IP <ip_gw1>.36262 > <ip_distant_server>.1194: . ack 1 win 46
13:41:23.321855 IP <ip_gw1>.36262 > <ip_distant_server>.1194: P 1:8(7) ack 1 win 46
13:41:23.321878 IP <ip_distant_server>.1194 > <ip_gw1>.36262: . ack 8 win 46
13:51:23.323284 IP <ip_gw1>.36262 > <ip_distant_server>.1194: F 8:8(0) ack 1 win 46
13:51:23.363158 IP <ip_distant_server>.1194 > <ip_gw1>.36262: . ack 9 win 46