Route/forward to identical LANs

**unixrb** · 07-21-2011

Hi.

I've been assigned a task to configure a Fedora box (release 14 with 2.6.35.6-45.fc14.i686) to provide routing in a lab environment.
To that end I started reading man pages for iptables, googled a great deal and did some testing.
But, I've not been successful.
The task is that the users from their office LAN (LAN-1) should to be able to connect to all the 16 nodes (transceivers) that exist in every rack (two racks for now).
These racks are completely separate to each other and are configured as LANs (LAN-2a, LAN-2b, ...) in a lab environment.
Unfortunately, all the racks/LANs are identical regarding the IP addresses for nodes, which means that for example 169.254.167.102 exists in all racks/LANs.
The idea is that to define an IP address range (10.128.116.0/24) that are used in a cross reference table, which assists the users to access their desired node and rack.

Attached (Layout2a.pdf) you'll find an illustration for the task.

Following is done:

1- The network admin has configured a rule that all calls to 10.128.116.0/24 goes to eth0.

2- The Fedora NICs are configured as:

::::::::::::::
ifcfg-eth0
::::::::::::::
DEVICE=eth0
BOOTPROTO=none
HWADDR=00:22:64:1C:22:AC
IPADDR=10.128.61.68
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
IPV6INIT=no
USERCTL=no
PREFIX=24
GATEWAY=10.128.61.254
::::::::::::::
ifcfg-eth1
::::::::::::::
DEVICE=eth1
BOOTPROTO=none
NETMASK=255.255.255.0
HWADDR=00:1b:21:33:04:fc
IPADDR=169.254.167.1
ONBOOT=yes
TYPE=Ethernet
IPV6INIT=no
USERCTL=no
PREFIX=24
::::::::::::::
ifcfg-eth2
::::::::::::::
DEVICE=eth2
BOOTPROTO=none
NETMASK=255.255.255.0
IPADDR=169.254.167.2
ONBOOT=yes
TYPE=Ethernet
HWADDR=00:07:e9:1f:fc:bf
IPV6INIT=no
USERCTL=no
PREFIX=24

3- # echo 1 > /proc/sys/net/ipv4/ip_forward

4- # ifdown eth2 ; ifup eth1

5- # iptables -t nat -A PREROUTING -i eth0 -d 10.128.116.2 -j DNAT --to-destination 169.254.167.100

6- # iptables -t nat -A POSTROUTING -d 169.254.167.100 -j MASQUERADE

7- Here I could from a node in LAN-1 connect to 10.128.116.2 and actually login to 169.254.167.100 in rack-1 (LAN-2a).

8- Just as a test, knowing that 10.128.116.2 is not aimed for rack-2 (LAN-2b), started testing the same as above with eth2 instead of eth1.

9- # ifdown eth1 ; ifup eth2

10- Repeated the steps 5 and 6.

11- Here I couldn't from a node in LAN-1 connect to 10.128.116.2 and login to 169.254.167.100 in rack-2 (LAN-2b).

So, am I on the right track at all?
If not, what is the recommendation?
If yes, I would appreciate a lot for any help that I can get.

Kindest regards.

**Irithori** · 07-21-2011

Honestly, you should consider to rework that setup.
The NATing will cause a lot of overhead, I am not even sure if it can work like you intend it to work.
Also, the machines in LAN2* will not be able to connect to LAN1

My recommendation would be:
Choose a big enough private network for all LAN2* hosts
or -if the LAN2*s shall be separated- choose one dedicated network per LAN"* rack.

Then you can use simple static routing, and later on firewalling (if neccessary)

**unixrb** · 07-21-2011

Irithori:

Thanks for your feedback.

I seem to have done a mistake bay having a box labeled ".0". Sorry for that! That box is just a "dummy" hub.
Furthermore, all the nodes in the racks are just telecommunication devices (transceivers), i.e. they are not traditional computers. In other words, there is no requirement for any connection from LAN-2* to LAN-1.
By the way, the reason behind the whole thing is that to make it more practical for users when they want to do some tests on the transceivers. As for now they have to go to the lab and manually connect their laptops to each rack.
So if we provide them with this feature, they can sit on their office and connect to the desired transceiver/rack by just using the cross reference table and telnet to the appropriate IP address.

**Lazydog** · 07-23-2011

Originally Posted by unixrb

::::::::::::::
ifcfg-eth0
::::::::::::::
DEVICE=eth0
BOOTPROTO=none
HWADDR=00:22:64:1C:22:AC
IPADDR=10.128.61.68
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
IPV6INIT=no
USERCTL=no
PREFIX=24
GATEWAY=10.128.61.254
::::::::::::::
ifcfg-eth1
::::::::::::::
DEVICE=eth1
BOOTPROTO=none
NETMASK=255.255.255.0
HWADDR=00:1b:21:33:04:fc
IPADDR=169.254.167.1
ONBOOT=yes
TYPE=Ethernet
IPV6INIT=no
USERCTL=no
PREFIX=24
::::::::::::::
ifcfg-eth2
::::::::::::::
DEVICE=eth2
BOOTPROTO=none
NETMASK=255.255.255.0
IPADDR=169.254.167.2
ONBOOT=yes
TYPE=Ethernet
HWADDR=00:07:e9:1f:fc:bf
IPV6INIT=no
USERCTL=no
PREFIX=24

You first issue is the netmasks in bold above. You cannot split out your network this way, and I do not understand why the network admin didn't see this.

You might want to look at cutting the network size down.

5- # iptables -t nat -A PREROUTING -i eth0 -d 10.128.116.2 -j DNAT --to-destination 169.254.167.100

6- # iptables -t nat -A POSTROUTING -d 169.254.167.100 -j MASQUERADE

This is most likely your next issue and maybe you shouldn't be doing it this way. In fact on an internal LAN there is really no need to do this at all. You can route 169.254.0.0.

I would do the following:

1. Cut the mask down to cover what I have in the rack plus 2. In your case 8 or 16 hosts. 16 will allow you to add to the rack.

2. Router the network 169.254.0.0 and remove all iptables rules that do any MASQ, SNAT or DNAT.

3. Configure DNS so the client only ned to know the host name of the device they want to get to. Will save the confusion later in life.

**unixrb** · 07-27-2011

Sorry for the long delay in getting back to you.
I've been testing a great deal, and finally made it work.

For those of you that wonder why there have to be identical LANs:

- Each rack has a control panel (with a NIC) that is configured (hard-coded) to offer the "slots" as 169.254.167.100-115, i.e. they cannot be changed.
- The racks are identical.

Now the solution (Please refer to TRX-Layout2.pdf):

Code:

01. # more /etc/sysconfig/network-scripts/ifcfg-eth*
    ::::::::::::::
    ifcfg-eth0
    ::::::::::::::
    DEVICE=eth0
    BOOTPROTO=none
    HWADDR=00:22:64:1C:22:AC
    IPADDR=10.128.61.68
    NETMASK=255.255.255.0
    ONBOOT=yes
    TYPE=Ethernet
    IPV6INIT=no
    USERCTL=no
    PREFIX=24
    GATEWAY=10.128.61.254
    ::::::::::::::
    ifcfg-eth1
    ::::::::::::::
    DEVICE=eth1
    BOOTPROTO=none
    NETMASK=255.255.255.0
    HWADDR=00:1b:21:33:04:fc
    IPADDR=169.254.167.1
    ONBOOT=yes
    TYPE=Ethernet
    IPV6INIT=no
    USERCTL=no
    PREFIX=24
    ::::::::::::::
    ifcfg-eth2
    ::::::::::::::
    DEVICE=eth2
    BOOTPROTO=none
    NETMASK=255.255.255.0
    IPADDR=169.254.167.2
    ONBOOT=yes
    TYPE=Ethernet
    HWADDR=00:07:e9:1f:fc:bf
    IPV6INIT=no
    USERCTL=no
    PREFIX=24
02. On the router in LAN-1:
    ml_router#conf t
    Enter configuration commands, one per line.  End with CNTL/Z.
    ml_router(config)#ip route 10.128.116.0 255.255.255.0 10.128.61.68
    ml_router(config)#end
03. # echo 1 > /proc/sys/net/ipv4/ip_forward
04. # vi /etc/iproute2/rt_tables
    . . .
    2       eth1
    3       eth2
05. # vi /etc/rc.d/rc.local
    . . .
    ip rule add fwmark 1 table eth1
    ip rule add fwmark 2 table eth2
    ip route add 169.254.167.0/24 dev eth2  scope link table eth2 src 169.254.167.2
    ip route add 169.254.167.0/24 dev eth1  scope link table eth1 src 169.254.167.1
06. # ip rule add fwmark 1 table eth1
07. # ip rule add fwmark 2 table eth2
08. # ip route add 169.254.167.0/24 dev eth2  scope link table eth2 src 169.254.167.2
09. # ip route add 169.254.167.0/24 dev eth1  scope link table eth1 src 169.254.167.1
10. # rule ls
    0:      from all lookup local 
    32764:  from all fwmark 0x2 lookup eth2 
    32765:  from all fwmark 0x1 lookup eth1 
    32766:  from all lookup main 
    32767:  from all lookup default 
11. # echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter 
12. # echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter 
13. # echo 0 > /proc/sys/net/ipv4/conf/eth2/rp_filter 
14. # echo 0 > /proc/sys/net/ipv4/conf/all/arp_filter         
15. # echo 0 > /proc/sys/net/ipv4/conf/eth1/arp_filter         
16. # echo 0 > /proc/sys/net/ipv4/conf/eth2/arp_filter
17. # vi /root/iptables.sh
    service iptables stop
    service iptables save
    service iptables start
    iptables -t mangle -A PREROUTING -d 10.128.116.0/27 -j MARK --set-mark 0x1
    iptables -t mangle -A PREROUTING -d 10.128.116.32/27 -j MARK --set-mark 0x2
    m=2
    n=100
    while [ "$m" -lt "18" ]; do
      # eth1
      iptables -t nat -A PREROUTING -d 10.128.116.$m -j DNAT --to-destination 169.254.167.$n
      iptables -t nat -A POSTROUTING -s 169.254.167.$n -m mark --mark 0xa -j SNAT --to-source 10.128.116.$m
      # eth2
      t=$(($m+32))
      iptables -t nat -A PREROUTING -d 10.128.116.$t -j DNAT --to-destination 169.254.167.$n
      iptables -t nat -A POSTROUTING -s 169.254.167.$n -m mark --mark 0x14 -j SNAT --to-source 10.128.116.$t
      m=$(($m+1))
      n=$(($n+1))
    done
    iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
    iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE
    iptables -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 0xa
    iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 0x14
    iptables -t mangle -A OUTPUT -s 169.254.167.1 -j MARK --set-mark 0x1
    iptables -t mangle -A OUTPUT -s 169.254.167.2 -j MARK --set-mark 0x2
    service iptables save
18. # /root/iptables.sh
19. # vi /etc/sysctl.conf
    . . .
    net.ipv4.ip_forward = 1
    . . .
20. # vi /etc/sysconfig/iptables-config 
    . . .
    IPTABLES_MODULES_UNLOAD="yes"
    . . .
    IPTABLES_SAVE_ON_STOP="yes"
    . . .
    IPTABLES_SAVE_ON_RESTART="yes"
    . . .
21. The cross reference table (eth3-8 are for future use!):

    Telnet to             Via Interface          Final Destination
    -----------------------------------------------------------------------
    10.128.116.2-17       169.254.167.1/eth1     rack-1/169.254.167.100-115
    10.128.116.34-49      169.254.167.2/eth2     rack-2/169.254.167.100-115
    10.128.116.66-81      169.254.167.3/eth3     rack-3/169.254.167.100-115
    10.128.116.98-113     169.254.167.4/eth4     rack-4/169.254.167.100-115
    10.128.116.130-145    169.254.167.5/eth5     rack-5/169.254.167.100-115
    10.128.116.162-177    169.254.167.6/eth6     rack-6/169.254.167.100-115
    10.128.116.194-209    169.254.167.7/eth7     rack-7/169.254.167.100-115
    10.128.116.226-241    169.254.167.8/eth8     rack-8/169.254.167.100-115

As mentioned, this was tested and verified by following the packets with tcpdump.

I would like to thank everybody that took time to help me on this.
Kind regards.

**Lazydog** · 07-27-2011

If it is true what you said about the rack cannot be changed then you need to goto the manufacture and let them know what a POS they have.

Thread Tools

Display

Route/forward to identical LANs

Posting Permissions