2 NICs same subnet routing issue

[karlochacon]

hi guys

I setting a Centos 5.6
server has 2 NICs
eth0 - 192.168.10.150/24 - 00.0C.29.DC.10.CA (MAC)
eth1 - 192.168.10.151/24 - 00.0C.29.DC.10.DE (MAC)
I need both in the same subnet since they will ne connected to another device - storage - point to point

but I found something while doing the installation
doing ARPs from a workstation I found the behavior below

[root@workstation ~]# arp -n
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.10.151           ether   00:0C:29:DC:10:CA   C                     eth0
192.168.10.150           ether   00:0C:29:DC:10:CA   C                     eth0

as you can see it's like eth0 gets all the requests
in fact If I disconnect the cable from eth1 IP .151 will be still pingable which is what I don't want
and If I disconnect cable from eth0 none of the IPs .150 - .151 is pingable and I was hoping .151 would be pingable...


so why is this behavior?
I need eth0 respond the request for .150 and don't respond .151
and eth1 respond .151 request

seems like eth0 owns the IPs

This is not working

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

neither is this

net.ipv4.conf.default.arp_filter=1
net.ipv4.conf.all.arp_filter=1

any help?
thanks alot

[Lazydog]

hi guys

I setting a Centos 5.6
server has 2 NICs
eth0 - 192.168.10.150/24 - 00.0C.29.DC.10.CA (MAC)
eth1 - 192.168.10.151/24 - 00.0C.29.DC.10.DE (MAC)
I need both in the same subnet since they will ne connected to another device - storage - point to point

First off never,Never connect two interface to the same network without them being bonded together. This causes issue on the network and if you are connected to an intelligent switch one of the interfaces will be disabled.

but I found something while doing the installation
doing ARPs from a workstation I found the behavior below

[root@workstation ~]# arp -n
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.10.151           ether   00:0C:29:DC:10:CA   C                     eth0
192.168.10.150           ether   00:0C:29:DC:10:CA   C                     eth0

as you can see it's like eth0 gets all the requests
in fact If I disconnect the cable from eth1 IP .151 will be still pingable which is what I don't want
and If I disconnect cable from eth0 none of the IPs .150 - .151 is pingable and I was hoping .151 would be pingable...

so why is this behavior?

Sounds like you are connected to an intelligent switch which has shut down the second port because of the routing loop you have created.

I need eth0 respond the request for .150 and don't respond .151
and eth1 respond .151 request

seems like eth0 owns the IPs

As long as the interface is up the address will be reachable via eth0. The system knows about both IP Addresses and both IP's are on the same subnet thus it show both on eth0.

This is not working

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

neither is this

net.ipv4.conf.default.arp_filter=1
net.ipv4.conf.all.arp_filter=1

any help?
thanks alot

Bond both of your interface together or break down the subnet. Those are the choices you have.

[karlochacon]

First off never,Never

Sounds like you are connected to an intelligent switch which has shut down the second port because of the routing loop you have created.

thanks a lot for your answer

well a switch is not the case
since server and storage device which is attached to is directly connected back to back that's why I have to use same subnet

I've read this needs some more magic like working with routing and the parameters I added above but I don't know how to create the magic routes

[karlochacon]

looks like I have to use iproute2 just don't know how
any guidance

thanks

[Lazydog]

since server and storage device which is attached to is directly connected back to back that's why I have to use same subnet

Like this:

[LAN]<-->[server]<-->[storage]

You do not have to use the same subnet in this case. The connection from the server to the storage can be anything you want as they do not touch any network other then their own. I would still use something that is not in use though.

LAN/Server can be on 192.168.10.* while the Server/Storage can be on 192.168.100.*.

Don't make this more complicated then it needs to be.

[karlochacon]

guys

I am getting there

this does what I want

iproute2

echo 200 storage_table >> /etc/iproute2/rt_tables
ip rule add from 192.168.10.151 table storage_table
ip route add 192.168.10.0/24 via 192.168.10.151 dev eth1 table storage_table

sysctl parameters

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

but I need to make this changes permanent after reboot
where I should add them?
I already tried /etc/init.d/boot.local but no luck

echo 200 storage_table >> /etc/iproute2/rt_tables
ip rule add from 192.168.10.151 table storage_table
ip route add 192.168.10.0/24 via 192.168.10.151 dev eth1 table storage_table

any idea?
thanks

[HROAdmin26]

Using the same network for 2 NIC's that are really on different networks is flatly incorrect. Please listen to Lazydog's advice so that you do not have to hack around the correct routing the kernel's TCP stack is trying to perform. Each NIC should be on it's own network.

I need both in the same subnet since they will ne connected to another device - storage - point to point

This is not correct.