Who's using my bandwidth?

This morning I noticed for the last 12 hours we've had a consistently large amount of outgoing traffic and I'd like some guidance as to how I can find out whats causing this.

I've got a Linux gateway with Slackware 12 installed. I'm using rrdtool to log and graph traffic volume. A weekly graph is below showing the sudden volume of traffic. After logging traffic for a year this has never been seen before. I also have a squid proxy with SARGE, but the reports show this isn't web traffic. This is on a network with about 30 systems.

Attachment 3993

I found the offending server through trial, error, and a bit of luck. Now I've got to figure out what exactly it is spewing.

I thought that finding the offending client would be as easy as running netstat or some other magical utility, but I couldn't figure it out. Sifting through /proc/net/ip_conntrack seems to be a good idea, but with 30 systems behind my firewall there's a lot of info there. I'm just not command-line savvy enough to split each data element, sum-filter-sort to spit out the answer I was looking for.

I'm still curious about the solution to this problem. If anyone knows how to determine which clients are utilizing the most bandwidth, or highest number of connections it would be very helpful for the future!

On the gateway/router box:

Code:

---

% iftop -i <lan-interface>

---

That should show you:

1] who's sending packets
2] where the packets are destined
3] how much bandwidth he's using

Cheers

Thanks. Ended up being an e-mail with attachment sending every 30 seconds to an uncooperative server which was severing the connection with no notice.

I don't have iftop (or ntop maybe) on there but that would have been easiest. I later threw this together which parses ip_conntrak, groups the output by source IP, and counts the number of established connections through the gateway. This wouldn't have helped in my situation since our server was only connecting to one client to send one e-mail. But if a system had been spewing spam or virus e-mails the number of established connections would be through the roof. In the output below I was testing this by running an nmap scan to find web servers serving on port 80 across some random block of IP's and you can see how that effected the results.

Code:

---

# cat /proc/net/ip_conntrack | grep "ESTABLISHED" | egrep --regexp='src=[0-9]+.[0-9]+.[0-9]+.[0-9]+' -o | cut -d "=" -f 2 | sort | uniq -c | sort -nr | head -n 20
    679 192.168.0.250
    43 192.168.0.3
    16 192.168.0.115
    15 192.168.0.2
    12 192.168.0.41
      8 192.168.0.70
      8 192.168.0.116
      7 71.187.24.19
      7 192.168.33.53
      6 108.16.81.82
      5 67.217.66.244
      5 67.217.65.244
      5 216.219.116.244
      5 173.62.135.31
      4 98.114.34.36
      4 209.213.219.27
      4 192.168.0.200
      4 173.63.50.164
      4 108.16.109.111

---