Find the answer to your Linux question:
Results 1 to 4 of 4
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1

    Bash script to pull and count IP addresses out of an apache log

    I have written a quick shell command to go through the output of an apache log and pull out the IP addresses and the number of times they occur. The IP addresses appear in the 2nd column of the log.

    for i in `awk '{print $2}' logfile | sort -u`; do echo $i " " `grep -c $i logfile`; done

    This provides the desired output, but takes a while to run against a big file. It also feels a bit of a brute force method.

    Does anyone have a more elegant solution that won't take hours on a large file as my shell scripting is kind of self taught and I may be missing something obvious to someone else!


  2. #2
    Linux Engineer drl's Avatar
    Join Date
    Apr 2006
    Saint Paul, MN, USA / CentOS, Debian, Slackware, {Free, Open, Net}BSD, Solaris

    One important idea is to "touch" the data as few times as possible. Using the pipeline and appropriate options on commands to isolate the data, then sort it, then count it in as few passes as possible would be something like this:
    awk '{print $2}' logfile | sort | uniq -c
    I think the awk will be fast enough, but you could compare it with a version using cut.

    See man pages for details.

    Best wishes ... cheers, drl
    Welcome - get the most out of the forum by reading forum basics and guidelines: click here.
    90% of questions can be answered by using man pages, Quick Search, Advanced Search, Google search, Wikipedia.
    We look forward to helping you with the challenge of the other 10%.
    ( Mn, 2.6.n, AMD-64 3000+, ASUS A8V Deluxe, 1 GB, SATA + IDE, Matrox G400 AGP )

  3. #3
    Cheers for that.
    Much appreciated.

  4. $spacer_open
  5. #4
    try something like this,

    cat /var/www/html | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' | tr " " "\n" | uniq -c # >> apachelogips.txt
    if you want to send the information to a text file instead of sending the output to the screen, just remove the pound sign.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts