Find the answer to your Linux question:
Results 1 to 4 of 4
I have written a quick shell command to go through the output of an apache log and pull out the IP addresses and the number of times they occur. The ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Feb 2011
    Posts
    2

    Bash script to pull and count IP addresses out of an apache log


    I have written a quick shell command to go through the output of an apache log and pull out the IP addresses and the number of times they occur. The IP addresses appear in the 2nd column of the log.

    for i in `awk '{print $2}' logfile | sort -u`; do echo $i " " `grep -c $i logfile`; done

    This provides the desired output, but takes a while to run against a big file. It also feels a bit of a brute force method.

    Does anyone have a more elegant solution that won't take hours on a large file as my shell scripting is kind of self taught and I may be missing something obvious to someone else!

    Thanks

  2. #2
    drl
    drl is offline
    Linux Engineer drl's Avatar
    Join Date
    Apr 2006
    Location
    Saint Paul, MN, USA / CentOS, Debian, Slackware, {Free, Open, Net}BSD, Solaris
    Posts
    1,288
    Hi.

    One important idea is to "touch" the data as few times as possible. Using the pipeline and appropriate options on commands to isolate the data, then sort it, then count it in as few passes as possible would be something like this:
    Code:
    awk '{print $2}' logfile | sort | uniq -c
    I think the awk will be fast enough, but you could compare it with a version using cut.

    See man pages for details.

    Best wishes ... cheers, drl
    Welcome - get the most out of the forum by reading forum basics and guidelines: click here.
    90% of questions can be answered by using man pages, Quick Search, Advanced Search, Google search, Wikipedia.
    We look forward to helping you with the challenge of the other 10%.
    ( Mn, 2.6.n, AMD-64 3000+, ASUS A8V Deluxe, 1 GB, SATA + IDE, Matrox G400 AGP )

  3. #3
    Just Joined!
    Join Date
    Feb 2011
    Posts
    2
    Cheers for that.
    Much appreciated.

  4. #4
    Just Joined!
    Join Date
    Aug 2011
    Posts
    5
    try something like this,

    Code:
    #!/bin/bash
    cat /var/www/html | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' | tr " " "\n" | uniq -c # >> apachelogips.txt
    if you want to send the information to a text file instead of sending the output to the screen, just remove the pound sign.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •