IPTables configuration possibly causing trouble

[Lister432]

Hi,

For one of my customers I'm running a web application on a RHEL 5 server. To protect against any intruders, I'm running iptables on this server.

Most of the time this works just fine, but once every few months the network links with either the other servers in the network or to the outside world get completely blocked. This often limits itself to certain ports, like http or the ports we're using for RPC communication between servers. Any attempted connections on these ports will result in a timeout.

Restarting iptables usually helps, though today we had to reboot the entire server because iptables refused to shut down properly (hang after saying "unloading iptables modules").

Since i'm no iptables expert at all, I would appreciate if someone could have a look at my setup to see if I'm doing something silly. This is the way I have things configured right now:

--------------------------------------------

*filter

# Set default policies
:INPUT DROP [1:44]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [27040:2493902]

# Accept all traffic from the loopback interface.
-A INPUT -i lo -j ACCEPT

# Accept legitimate responses to traffic we generate.
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

# Accept SSH connections
-A INPUT -p tcp -m tcp --dport 2225 -j ACCEPT

# Allow ICMP
-A INPUT -p icmp -j ACCEPT

# Allow inbound HTTP and HTTPS
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT

# Allow apache to connect to JBoss
-A INPUT -i lo -p tcp -m tcp --dport 8080 -j ACCEPT
-A INPUT -s 127.0.0.1 -p tcp -m tcp --dport 8080 -j ACCEPT

# Allow traffic from our other servers
-A INPUT -s 1.2.3.1 -p tcp -m tcp --dport 1024:65535 -j ACCEPT
-A INPUT -s 1.2.3.2 -p tcp -m tcp --dport 1024:65535 -j ACCEPT
-A INPUT -s 1.2.3.3 -p tcp -m tcp --dport 1024:65535 -j ACCEPT

# Allow traffic from cloud servers
-A INPUT -s 10.163.184.11 -p tcp -m tcp --dport 1024:65535 -j ACCEPT
-A INPUT -s 10.163.184.12 -p tcp -m tcp --dport 1024:65535 -j ACCEPT
-A INPUT -s 10.163.184.15 -p tcp -m tcp --dport 1024:65535 -j ACCEPT

# Allow NFS from other server
-A INPUT -s 1.2.3.3 -m state --state NEW -p udp --dport 111 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p tcp --dport 111 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p tcp --dport 2049 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p tcp --dport 32803 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p udp --dport 32769 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p tcp --dport 892 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p udp --dport 892 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p tcp --dport 875 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p udp --dport 875 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p tcp --dport 662 -j ACCEPT
-A INPUT -s 1.2.3.3 -m state --state NEW -p udp --dport 662 -j ACCEPT


# Allow NFS from cloud
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p udp --dport 111 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p tcp --dport 111 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p tcp --dport 2049 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p tcp --dport 32803 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p udp --dport 32769 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p tcp --dport 892 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p udp --dport 892 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p tcp --dport 875 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p udp --dport 875 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p tcp --dport 662 -j ACCEPT
-A INPUT -s 10.163.184.0/255.255.255.0 -m state --state NEW -p udp --dport 662 -j ACCEPT

# Allow all related traffic to/from non-privileged ports.
-A INPUT -p tcp -m tcp --sport 1024:65535 --dport 1024:65535 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p udp --dport 67:68 -j DROP

# Log the rest of the incoming messages (all of which are dropped)
# with a maximum of 15 log entries per minute
-A INPUT -m limit --limit 15/minute -j LOG --log-level 7 --log-prefix "Dropped by firewall: "

# Make it all true.
COMMIT

--------------------------------------------

I've replaced our public ip's with 1.2.3.*

Is there anything here that might explain these strange problems?

Thanks in advance for your help!

[Quantum]

Well are you configuring iptables manually in a script? If so, this is very prone to errors. Please use Shorewall.

Second, you MUST run Shorewall on ALL your machines. It's entirely possible that one or more are compromised.

[Lazydog]

Most of the time this works just fine, but once every few months the network links with either the other servers in the network or to the outside world get completely blocked. This often limits itself to certain ports, like http or the ports we're using for RPC communication between servers. Any attempted connections on these ports will result in a timeout.

I have looked at your rules and I have a few questions

# Allow traffic from our other servers
-A INPUT -s 1.2.3.1 -p tcp -m tcp --dport 1024:65535 -j ACCEPT
-A INPUT -s 1.2.3.2 -p tcp -m tcp --dport 1024:65535 -j ACCEPT
-A INPUT -s 1.2.3.3 -p tcp -m tcp --dport 1024:65535 -j ACCEPT

# Allow traffic from cloud servers
-A INPUT -s 10.163.184.11 -p tcp -m tcp --dport 1024:65535 -j ACCEPT
-A INPUT -s 10.163.184.12 -p tcp -m tcp --dport 1024:65535 -j ACCEPT
-A INPUT -s 10.163.184.15 -p tcp -m tcp --dport 1024:65535 -j ACCEPT

Why are you using these rules? What are you expecting to achive with them?

# Allow all related traffic to/from non-privileged ports.
-A INPUT -p tcp -m tcp --sport 1024:65535 --dport 1024:65535 -m state --state RELATED,ESTABLISHED -j ACCEPT

Since in the beginning you have the ESTABLISHED,RELATED rule there is no need for it here.
You can delete this rule.

# Log the rest of the incoming messages (all of which are dropped)
# with a maximum of 15 log entries per minute
-A INPUT -m limit --limit 15/minute -j LOG --log-level 7 --log-prefix "Dropped by firewall: "

Logging is great for troubleshooting as you can see what is being dropped.

Is there anything here that might explain these strange problems?

Thanks in advance for your help!

Nothing that jumps out at me. I am just wondering if your interface might be over whelmed?

You can have a look at this IPTABLES Tutorial
This explains all the commands and what they do and how to use them.

Well are you configuring iptables manually in a script? If so, this is very prone to errors. Please use Shorewall.

Please, one only needs shorewall when they do not understand how iptables works or how to configure it.

[Quantum]

Please, one only needs shorewall when they do not understand how iptables works or how to configure it.

Nice put-down Robert. What you don't realize is that almost no one has the TIME to study and become an expert in bit-twiddling, to the point of NEVER making a mistake. Unfortunately you must ALWAYS be right, and they only have to be right once.

I for one, have much more important things to do.

[Lazydog]

PM inbound

[Quantum]

PM deleted without reading.

[Lister432]

Thanks for your explanation and suggestions!

Why are you using these rules? What are you expecting to achive with them?

Our other servers need to be able to make RPC connections to the application running on this server. The application more or less randomly assigns a port for each process our other servers in a range between 2000 an 5000. I guess I would design it differently today, but that's the way our application works for the moment.

Since in the beginning you have the ESTABLISHED,RELATED rule there is no need for it here.
You can delete this rule.

Thanks! I've deleted it.

Nothing that jumps out at me. I am just wondering if your interface might be over whelmed?

The server is not all that busy, traffic-wise, but I'll set up some monitoring to see if the traffic is heavier than I thought.

You can have a look at this IPTABLES Tutorial
This explains all the commands and what they do and how to use them.

I'll certainly have a look at it, so I can hopefully better understand what I'm doing