Bash script to pull and count IP addresses out of an apache log

[Bursar]

I have written a quick shell command to go through the output of an apache log and pull out the IP addresses and the number of times they occur. The IP addresses appear in the 2nd column of the log.

for i in `awk '{print $2}' logfile | sort -u`; do echo $i " " `grep -c $i logfile`; done

This provides the desired output, but takes a while to run against a big file. It also feels a bit of a brute force method.

Does anyone have a more elegant solution that won't take hours on a large file as my shell scripting is kind of self taught and I may be missing something obvious to someone else!

Thanks

[drl]

Hi.

One important idea is to "touch" the data as few times as possible. Using the pipeline and appropriate options on commands to isolate the data, then sort it, then count it in as few passes as possible would be something like this:

awk '{print $2}' logfile | sort | uniq -c

I think the awk will be fast enough, but you could compare it with a version using cut.

See man pages for details.

Best wishes ... cheers, drl

[Bursar]

Cheers for that.
Much appreciated.

[AnitaHummer]

try something like this,

#!/bin/bash
cat /var/www/html | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' | tr " " "\n" | uniq -c # >> apachelogips.txt

if you want to send the information to a text file instead of sending the output to the screen, just remove the pound sign.