ubuntu subnetting problem

[unwornmoon]

Hi,

I am pretty new to linux networking. Now I am trying to set up a very simple three-machine network, with the following topology:

dst (eth0) --- (eth1) router (eth0) --- (eth0) src

where:
-- dst (eth0) is configured 192.168.0.25
-- (eth1) router is configured 192.168.0.9
-- router (eth0) is configured 10.10.10.16
-- (eth0) src is configured 10.10.10.19

1) I configured the IPs as follows:

-- for dst, edit /etc/network/interfaces:

auto eth0
iface eth0 inet static
address 192.168.0.25
netmask 255.255.255.0


-- for router, edit /etc/network/interfaces:

auto eth0
iface eth0 inet static
address 10.10.10.16
netmask 255.255.255.0

auto eth1
iface eth1 inet static
address 192.168.0.9
netmask 255.255.255.0

-- for src, edit /etc/network/interfaces:
auto eth0
iface eth0 inet static
address 10.10.0.19
netmask 255.255.255.0


Then I ran /etc/init.d/networking restart on all machines.

And I added

route add default gw 10.10.10.16

on the src machine.

2) I enable the forwarding capability on "router", by:
echo "1" > /proc/sys/net/ipv4/ip_forward

3) But then, the src can only ping router, and dst can only ping router. Src cannot ping dst.

When I ran "arp -a" on src, I saw the entry for dst (192.168.0.25) is "incomplete". So I added that entry by "arp -s". Then src can ping dst now.

But I don't know why ARP cannot automatically do this?

4) When I tried to also add an ARP entry on dst for src, by using "arp -s", it said: "network unreachable".

I wonder why dst failed while src succeeded? And since src can ping dst, doesn't it mean dst can also return an ICMP reply to src?

Thanks!!!!

[HROAdmin26]

ARP is a broadcast protocol *not* passed by routers. ARP resolves IP => MAC address. In normal network communications, a system does not know the MAC of NIC's *not* located on its local subnet.

DST and SRC are on different networks. Only the router will know the MAC of both DST and SRC.

In addition to telling SRC how to reach DST, you have to tell DST how to communicate *back* to SRC. This means adding a network route on DST as well. It doesn't have to be the default route.

man route

[unwornmoon]

Thanks! Now it makes so much sense to me!

Is my understanding below correct? When SRC wants to send packets to DST (192.168.0.25), it will first find the "next-hop" as 10.10.10.16 (which is eth0 of router) via its routing table (if correctly configured), and then try to find the MAC of 10.10.10.16 via ARP. In this case, the corresponding ARP cache entry should be:

10.10.10.16 aa:bb:cc:dd:ee:ff (MAC of router eth0)

So basically, the ARP cache of SRC should only contain entries with IP addresses on the same local network. So 192.168.0.25 (DST) should not have been in the ARP cache entry?

In my previous configuration, "192.168.0.25 <incomplete>" mysteriously appeared in my SRC ARP cache. I guess there was something wrong with routing configuration on SRC. Then I worked around by adding an ARP entry:

192.168.0.25 (DST) aa:bb:cc:dd:ee:ff (MAC of router eth0)

That forced packets for DST to go to router eth0.

When I tried to add "10.10.10.19 (SRC) MAC_OF_ROUTER_ETH1" on DST, it says "network unreachable", I guess it was because dst somehow detects SRC is not on the same local network, and ARP entries should not contain foreign IPs?

Thanks for the insights!!!

[HROAdmin26]

Thanks! Now it makes so much sense to me!

Is my understanding below correct? When SRC wants to send packets to DST (192.168.0.25), it will first find the "next-hop" as 10.10.10.16 (which is eth0 of router) via its routing table (if correctly configured), and then try to find the MAC of 10.10.10.16 via ARP. In this case, the corresponding ARP cache entry should be:

10.10.10.16 aa:bb:cc:dd:ee:ff (MAC of router eth0)

So basically, the ARP cache of SRC should only contain entries with IP addresses on the same local network. So 192.168.0.25 (DST) should not have been in the ARP cache entry?

In my previous configuration, "192.168.0.25 <incomplete>" mysteriously appeared in my SRC ARP cache. I guess there was something wrong with routing configuration on SRC. Then I worked around by adding an ARP entry:

192.168.0.25 (DST) aa:bb:cc:dd:ee:ff (MAC of router eth0)

That forced packets for DST to go to router eth0.

When I tried to add "10.10.10.19 (SRC) MAC_OF_ROUTER_ETH1" on DST, it says "network unreachable", I guess it was because dst somehow detects SRC is not on the same local network, and ARP entries should not contain foreign IPs?

Thanks for the insights!!!

Yes, that's correct