Hi All:

Ok, I am running a router with the following:
Linux rx1000test 2.6.8-16-486-rx #1 Wed Mar 15 15:33:23 UTC 2006 i586 GNU/Linux
rx1000test:~# ip -V
ip utility, iproute2-ss041019
rx1000test:~# shorewall version
2.2.3
rx1000test:~# ipsec version
Linux Openswan U2.2.0/K2.6.8-16-486-rx (native)
Openswan is using the native 2.6 netkey stack NOT the klips interface.

Ok, what I am attempting to do is run GRE over the IPSec connection in a hub a spoke arrangement. The Hub is a Cisco 2821, the spokes are rx1000 routers with the above mentioned stuff installed.
The IPsec works fine except the following:
1. I cannot ping from 1 RX1000 to another,
2. There are no interfaces on the Cisco hence, no routes can be entered which is the reason for 1.

However, comms fromthe spokes to the hub are fine over IPsec. Now, I want to introduce some traffic control into the equation and make it so that RX1000's can ping each other so....GRE over IPsec s the answer. Fine, here is the RX1000 setup for the GRE:
modprobe ip_gre
ip tunnel add GDC1 mode gre remote 192.168.1.1 local 192.168.1.97 ttl 255
ip link set GDC1 up
ip addr add 192.168.2.97/28 peer 192.168.2.110/28 dev GDC1
ip route del 192.168.1.0/28 via 160.96.97.248
ip route add 192.168.1.0/28 via 192.168.2.110
Notes:
-I am using a subnet of 192.168.2.96/28 for the tunnel,
-I had to delete the IPsec route from the Linux before I could install the tunnel route.

Cisco side is dead simple:
interface Tunnel6
ip address 192.168.2.110 255.255.255.240
ip mtu 1400
tunnel source GigabitEthernet0/1
tunnel destination 192.168.1.97

ip route 0.0.0.0 0.0.0.0 160.96.97.250
ip route 192.168.1.96 255.255.255.240 Tunnel6
ip route 192.168.1.97 255.255.255.255 GigabitEthernet0/0

Now, the Cisco routes are a bit strange, the first is the default router, the second is what you'd expect, the route to the network behind the RX1000, the third is required to get the Cisco from producing a recurrent route error and bringing down tun 6.

I am running "debig tunnels" on the cisco and tcpdump -i GDC1 on the RX1000 so that I can see what is going on.

The good:
From the RX1000 ssh console, I can ping every interface and host in my network.
From the Cisco SSH session I can ping everyting except:
-the RX1000 ethernet interface,
-anytning behind the RX1000.

From any host behind the cisco I can ping the RX1000, the Cisco, all tunnel intefaces but not anything behind the RX1000.

From a host behind the RX1000, I can ping the tunnel interfaces and the RX1000 but that's all. Can't ping the Cisco or anything behind the cisco.

Now, watching the tcp dump on the GDC1 interface, here are pings to the cisco(ping 192.168.1.1 -w 2):
02:20:04.197414 IP 192.168.2.97 > 192.168.1.1: icmp 64: echo request seq 1
02:20:04.214860 IP 192.168.1.1 > 192.168.2.97: icmp 64: echo reply seq 1
02:20:05.198859 IP 192.168.2.97 > 192.168.1.1: icmp 64: echo request seq 2
02:20:05.215704 IP 192.168.1.1 > 192.168.2.97: icmp 64: echo reply seq 2


Perfect.
Now, here are pings froma host behind the RX1000 to the local interface of the GDC tunnel:
Successful ping but nothing to see on tcpdump as the packets neve entered the tunnel.
Pinging the other side of the GDC1 tunnel(ping 192.168.2.110 -n 2):
02:24:49.676603 IP 192.168.1.101 > 192.168.2.110: icmp 40: echo request seq 39424
02:24:49.693395 IP 192.168.2.110 > 192.168.1.101: icmp 40: echo reply seq 39424
02:24:50.686451 IP 192.168.1.101 > 192.168.2.110: icmp 40: echo request seq 39680
02:24:50.703303 IP 192.168.2.110 > 192.168.1.101: icmp 40: echo reply seq 39680


Perfect, successful pings as it should be. Now it gets exciting, here is the tcp dump from pings to th cisco, 192.168.1.1:
02:25:11.815092 IP 192.168.1.1 > 192.168.1.101: icmp 40: echo reply seq 39936
02:25:17.054163 IP 192.168.1.1 > 192.168.1.101: icmp 40: echo reply seq 40192

Now this is strange. It says that 192.168.1.1 is replying to an echo, through the tunnel but the initial echo request did NOT go via the tunnel. AND, the ping was unsuccessful.
Here are the RX1000 IP particulars:
rx1000test:~# ip addr show
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0a:dc:04:7d:dc brd ff:ff:ff:ff:ff:ff
inet 192.168.1.97/28 brd 192.168.1.255 scope global eth1
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:0a:dc:04:7d:dd brd ff:ff:ff:ff:ff:ff
6: w1adsl: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:77:77:77:7b:b3 brd ff:ff:ff:ff:ff:ff
9: gre0: <NOARP> mtu 1428 qdisc noop
link/gre 0.0.0.0 brd 0.0.0.0
11: ppp1: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1452 qdisc pfifo_fast qlen 3
link/ppp
inet 202.42.98.4 peer 202.42.98.1/32 scope global ppp1
12: GDC1@NONE: <POINTOPOINT,NOARP,PROMISC,UP> mtu 1514 qdisc noqueue
link/gre 192.168.1.97 peer 192.168.1.1
inet 192.168.2.97 peer 192.168.2.110/28 scope global GDC1
rx1000test:~# ip link show
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0a:dc:04:7d:dc brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:0a:dc:04:7d:dd brd ff:ff:ff:ff:ff:ff
6: w1adsl: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:77:77:77:7b:b3 brd ff:ff:ff:ff:ff:ff
9: gre0: <NOARP> mtu 1428 qdisc noop
link/gre 0.0.0.0 brd 0.0.0.0
11: ppp1: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1452 qdisc pfifo_fast qlen 3
link/ppp
12: GDC1@NONE: <POINTOPOINT,NOARP,PROMISC,UP> mtu 1514 qdisc noqueue
link/gre 192.168.1.97 peer 192.168.1.1
rx1000test:~# ip tun show
gre0: gre/ip remote any local any ttl inherit nopmtudisc
GDC1: gre/ip remote 192.168.1.1 local 192.168.1.97 ttl 255


Now, from hosts behind the cisco, they can ping the RX1000 no problem, but similar problems when they attempt to ping anything behind the RX1000.

Ok, I am stumped. Watching syslog with tail -f and there are no shorewall errors. Does anybody have any ideas on this. I am soooo close to getting this working, just need a tip or two here.

Cheers,
John