Packets refuse to go via Tunnel

[jserink]

Hi All:

I have a strange problem:
I have a debian based router that I have setup IPSec with GRE over top. The tunnel addresses are 192.168.2.97 locally, the other side is 192.168.2.110. The tunnel is 192.168.2.96/28. The end points are locally 192.168.1.97(eth1) and 192.168.1.1 the other side's eth 1. Both local ethernet's are behind NAT of course.

Now, I have ahost, 192.168.1.101 on the 192.168.1.96/28 network behind the debbian router. Here is my routing table:
rx1000test:~# ip route show
202.42.98.1 dev ppp1 proto kernel scope link src 202.42.98.62
192.168.1.0/28 dev GDC1 scope link
192.168.1.96/28 dev eth1 scope link
default dev ppp1 scope link

This seems really simple to me, anything going to 192.168.1.0/28 must go through tunnel GDC1. Here is the tunnel:

15: GDC1@NONE: <POINTOPOINT,NOARP,PROMISC,UP> mtu 1428 qdisc noqueue
link/gre 192.168.1.97 peer 192.168.1.1
inet 192.168.2.97 peer 192.168.2.110/28 scope global GDC1

Ok, now to check this I run tcpdump -i GDC1, tcpdump -i eth1 not tcp port 22 and I ping from 192.168.1.101 to 192.168.1.2, here is what I get:

C:\>ping 192.168.1.2 -n 2

Pinging 192.168.1.2 with 32 bytes of data:

Request timed out.
Request timed out.

Ping statistics for 192.168.1.2:
Packets: Sent = 2, Received = 0, Lost = 2 (100% loss),

rx1000test:~# tcpdump -i GDC1
tcpdump: WARNING: arptype 778 not supported by libpcap - falling back to cooked socket
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on GDC1, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
13:40:01.296907 IP 192.168.1.2 > 192.168.1.101: icmp 40: echo reply seq 7424
13:40:06.587157 IP 192.168.1.2 > 192.168.1.101: icmp 40: echo reply seq 7680

rx1000test:~# tcpdump -i eth1 not tcp port 22
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
13:40:01.267813 IP 192.168.1.101 > 192.168.1.2: icmp 40: echo request seq 7424
13:40:06.571007 IP 192.168.1.101 > 192.168.1.2: icmp 40: echo request seq 7680

This is bizzare:
The ping to 192.168.1.2, which is clearly part of the 192.168.1.0/28 network enters Eth1 but does does NOT go through the GDC1 tunnel on its way to 192.168.1.2....but the routing table tells us that it MUST go that way...no?
Then the replay comes back via the tunnel but never goes out eth1. eh????

I am watching /var/log/syslog and shorewall is not doing anything, but just to make sure I stop shorewall and redo the test, exactly the same thing.

Does anyone know WHY the pings to 192.168.1.2 are not going into the GDC1 tunnel?
Does anyone know WHY the retunr pings do not get forwarded out eth1?

Cheers,
John

[KenJackson]

I'm not as familiar with 'ip route show'. I usually use 'netstat -nr'.

But when I do 'ip route show' on my system here with a similar arrangement (using OpenVPN via 'tun' devices) I get:
192.168.1.0/24 via 192.168.60.10 dev tun0
compared to your
192.168.1.0/28 dev GDC1 scope link

Mine seems to have additional information (the IP of the local tun device). I don't know if that's significant. It might be helpful if you could post the result of 'netstat -nr'.

Are you sure that 'scope link' is what you want? I just dug out the IPROUTE2 book and it says in 5.1, "link - the address is link local, i.e. it is valid only on this device." I vaguely remember trying to specify 'scope link' at some point and being dissatisfied with the results.

One other thing, even if the originating side is routing everything correctly, the ping will fail if the receiving side does not route the return correctly.

[jserink]

Hi Ken:

Many thanx for having a look.
Ok, have tried the things you suggested, the original routing table:
rx1000test:~# ip route
202.42.98.1 dev ppp1 proto kernel scope link src 210.24.254.213
192.168.1.0/28 via 192.168.2.110 dev GDC1
192.168.2.96/28 dev GDC1 proto kernel scope link src 192.168.2.97
192.168.1.96/28 dev eth1 scope link
default dev ppp1 scope link

Destination Gateway Genmask Flags MSS Window irtt Iface
202.42.98.1 0.0.0.0 255.255.255.255 UH 0 0 0 ppp1
192.168.1.0 192.168.2.110 255.255.255.240 UG 0 0 0 GDC1
192.168.2.96 0.0.0.0 255.255.255.240 U 0 0 0 GDC1
192.168.1.96 0.0.0.0 255.255.255.240 U 0 0 0 eth1
0.0.0.0 0.0.0.0 0.0.0.0 U 0 0 0 ppp1

Same results pinging 192.168.1.1 from 192.168.1.101, echo request does not go via tun GDC1 but returns via the tunnel. Hoever, the return does not get sent back to 192.168.1.101, it never goes out eth1 so the ping times out.Similarly if I try to ping 192.168.1.2 form 192.168.1.101.

I have tried experimenting changing the scope to global, then routing via dev GDC1 and trying host...no difference. Same behavior. SSH into the 192.168.1.97 gateway, I can ping everything. But behind the gateway, I can't ping anything except the gateway and the tunnel IPs.

My present setup:
rx1000test:~# ip route
202.42.98.1 dev ppp1 proto kernel scope link src 210.24.254.213
192.168.1.0/28 via 192.168.2.110 dev GDC1 proto static
192.168.2.96/28 dev GDC1 proto kernel scope link src 192.168.2.97
192.168.1.96/28 dev eth1 scope link
default dev ppp1 scope link
rx1000test:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
adsl1500-1.dyn9 * 255.255.255.255 UH 0 0 0 ppp1
192.168.1.0 192.168.2.110 255.255.255.240 UG 0 0 0 GDC1
192.168.2.96 * 255.255.255.240 U 0 0 0 GDC1
192.168.1.96 * 255.255.255.240 U 0 0 0 eth1
default * 0.0.0.0 U 0 0 0 ppp1

rx1000test:~# ip link
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: eth1: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0a:dc:04:7d:dc brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:0a:dc:04:7d:dd brd ff:ff:ff:ff:ff:ff
6: w1adsl: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:77:77:77:7b:b3 brd ff:ff:ff:ff:ff:ff
9: gre0: <NOARP> mtu 1428 qdisc noop
link/gre 0.0.0.0 brd 0.0.0.0
25: ppp1: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1452 qdisc pfifo_fast qlen 3
link/ppp
26: GDC1@NONE: <POINTOPOINT,NOARP,PROMISC,UP> mtu 1428 qdisc noqueue
link/gre 192.168.1.97 peer 192.168.1.1

rx1000test:~# ip tun
gre0: gre/ip remote any local any ttl inherit nopmtudisc
GDC1: gre/ip remote 192.168.1.1 local 192.168.1.97 ttl 255

rx1000test:~# ip addr
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
3: eth1: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0a:dc:04:7d:dc brd ff:ff:ff:ff:ff:ff
inet 192.168.1.97/28 brd 192.168.1.111 scope global eth1
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:0a:dc:04:7d:dd brd ff:ff:ff:ff:ff:ff
6: w1adsl: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:77:77:77:7b:b3 brd ff:ff:ff:ff:ff:ff
9: gre0: <NOARP> mtu 1428 qdisc noop
link/gre 0.0.0.0 brd 0.0.0.0
25: ppp1: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1452 qdisc pfifo_fast qlen 3
link/ppp
inet 210.24.254.213 peer 202.42.98.1/32 scope global ppp1
26: GDC1@NONE: <POINTOPOINT,NOARP,PROMISC,UP> mtu 1428 qdisc noqueue
link/gre 192.168.1.97 peer 192.168.1.1
inet 192.168.2.97 peer 192.168.2.110/28 brd 192.168.2.111 scope global GDC1
rx1000test:~# ifconfig GDC1
GDC1 Link encap:UNSPEC HWaddr C0-A8-01-61-01-40-50-F9-00-00-00-00-00-00-00-00
inet addr:192.168.2.97 P-t-P:192.168.2.110 Mask:255.255.255.240
UP POINTOPOINT RUNNING NOARP MTU:1428 Metric:1
RX packets:27 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1812 (1.7 KiB) TX bytes:1248 (1.2 KiB)

DO you have any more ideas?
Any thoughts are appreciated.

Cheers,
John

I'm not as familiar with 'ip route show'. I usually use 'netstat -nr'.

But when I do 'ip route show' on my system here with a similar arrangement (using OpenVPN via 'tun' devices) I get:
192.168.1.0/24 via 192.168.60.10 dev tun0
compared to your
192.168.1.0/28 dev GDC1 scope link

Mine seems to have additional information (the IP of the local tun device). I don't know if that's significant. It might be helpful if you could post the result of 'netstat -nr'.

Are you sure that 'scope link' is what you want? I just dug out the IPROUTE2 book and it says in 5.1, "link - the address is link local, i.e. it is valid only on this device." I vaguely remember trying to specify 'scope link' at some point and being dissatisfied with the results.

One other thing, even if the originating side is routing everything correctly, the ping will fail if the receiving side does not route the return correctly.

[KenJackson]

Don't know. But if you are pinging through a machine, including sending a packet from one device to another, I believe you have to have forwarding enabled. Try this:

# sysctl net.ipv4.ip_forward

If the answer isn't 1, turn it on:

# sysctl -w net.ipv4.ip_forward=1

[jserink]

rx1000test:~# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1


Cheers,
John