I have 2 webservers linked together with a load balancer. All requests go through the load balancing ip and distributes them between the two servers.

The only exception is that mail is on just one server.

So basically I have:
xxx.xx.xxx.110 > load balance ip
xxx.xx.xxx.111 > web server + mail server
xxx.xx.xxx.114 > web server (rsync'd with exceptions)

Some ISPs (like Roadrunner rr) refuse when I try to send mail from this website:
SMTP error from remote mail server after initial connection:
host cdptpa-smtpin01.mail.rr.com [xx.xxx.xxx.xxx]:
554 5.7.1 - ERROR: Mail refused - <xxx.xx.xxx.111>

Any ideas? It routes traffic quite well, and this is the only problem I am experiencing.