Can't ping via direct connection using auto-ip address

[McGoo]

I have two Linux hosts that I am trying to connect. Each host has 4 NICs. They have each been auto-configured with 169.XXX.XXX.XXX addresses. The interfaces on each are up and operational. I can arping the peer from either host just fine. Getting back replies.

However, regular ping fails in both directions. I suspect it is a routing issue since ping works at layer 3 vs arping which works at layer 2. Just not sure what I am missing.

Here are my routes:

Host #1
======
# ip route show
default dev eth0
169.254.0.0/16 dev eth0 src 169.254.184.195
169.254.0.0/16 dev eth1 src 169.254.197.39
169.254.0.0/16 dev eth2 src 169.254.184.155
169.254.0.0/16 dev eth3 src 169.254.106.224

Host #2
======
# ip route show
default dev eth0
169.254.0.0/16 dev eth0 src 169.254.249.230
169.254.0.0/16 dev eth1 src 169.254.113.198
169.254.0.0/16 dev eth2 src 169.254.63.21
169.254.0.0/16 dev eth3 src 169.254.119.110

I am seeing the same behavior when I try to direct connect a Windows laptop to one of these hosts. I can arping the laptop from the Linux host, but ping fails.

I have some additional information about eth0 on my Linux host.

# ip link list
eth0: <BROADCAST, MULTICAST, UP, 10000> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether cc:52af:ea:7d:e0 brd ff:ff:ff:ff:ff:ff

# ip address show
eth0: <BROADCAST, MULTICAST, UP, 10020> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether cc:52af:ea:7d:e0 brd ff:ff:ff:ff:ff:ff
inet 169.254.184.195/16 brd 169.254.255.255 scope global eth0

#arp -a
?(169.254.27.12) at 00:26:9e:cf:09:ce
[ether] on eth0
?(169.254.27.12) at <incomplete> on eth3

I have discovered that if I down (ifconfig ethX down) eth1, eth2, and eth3, I'm able to ping just fine. What I don't understand is why I can't do this when all the interfaces are up. I guess ping is getting confused when it sees four interfaces all with the same network address. I was nonetheless expecting that if I did a ping -I eth0, that it would avoid any confusion over which interface to send the ping request over.