Ahh so your ssh was actually bound to a specific IP..

mine is unbound so it will accept a connection to any IP on the box.. Thought it was that way by default.

# What ports, IPs and protocols we listen for
Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
listen address is commented out so there are no restrictions.