Find the answer to your Linux question:
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 14
Hi guys I'm new to this forum, so hello to all. I have done quite a bit of searching for this but so far nothing I've found helps to answer ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Jun 2011
    Posts
    2

    tail log file, pipe to grep and email output


    Hi guys

    I'm new to this forum, so hello to all.

    I have done quite a bit of searching for this but so far nothing I've found helps to answer my question.

    OK.... here we go.

    I'm relatively new to bash scripting but I'd like to produce a script to solve my issue, so I'll explain what I'm trying to achieve.

    I want to tail a log file and grep for a specific entry....

    # tail -f /var/log/messages | grep "search string"

    If the grep produces an output then I want to email the output as an alert.

    Problem is I'm not sure of the best way to approach this. Also I don't want someone to say "here you go, here's a script I have already". I'd like to achieve some of this myself. I'm just stuck with the right direction to head in.

    If some of you kind people can give me some pointers I'd be really grateful, or some similar examples.

    Thank you in advance.

    Mikey

  2. #2
    Just Joined!
    Join Date
    Sep 2007
    Location
    Silver Spring, MD
    Posts
    95

    Sending a line item through email

    # tail -f /var/log/messages | grep "search string"

    It sounds like you are on the right track.

    Have you ever worked with cron or crontab?

    You can use cron to schedule a time to run a particular process ("grep" for instance).

    Since this is a site to help people resolve problems, take a look at this way, you might want to look at putting this in a file and executing it from a cron job:

    lookup=grep -i "`date "+%b %d`" /var/log/messages | egrep -i "search string" # lookups today's info and egrep looks up multiple search strings

    echo $lookup | mail -s "search string" <email address>

    That is a good start; however, there are a multitude of ways to get this done.

    One thing to note, is sendmail or postfix running on your machine, (service sendmail status or service postfix)?

    T
    Last edited by tdsan; 06-23-2011 at 11:39 PM.

  3. #3
    Just Joined!
    Join Date
    Feb 2009
    Posts
    5
    You don't want the -f flag. What that does is continuously read the file and output as data is appended. The way you have it, the tail will never end, so it will continue to pipe into grep... which will also never end...

    tom.

  4. #4
    Just Joined!
    Join Date
    Jan 2011
    Location
    Fairfax, Virginia, USA
    Posts
    94
    Hi, I do exactly what you want to do. You want to use -F to follow and also be tolerating of when logrotate moves the file. Here is the command I use:
    Code:
    tail -F --max-unchanged-stats=5 /var/log/secure

  5. #5
    Just Joined!
    Join Date
    Feb 2009
    Posts
    5
    If you are following syslog (either -f or -F), you are going to have a problem mailing the output.

    The following example won't work. The tail never closes its output file, so grep never closes its output, and 'mail' never sees end of file, so it won't finish either.

    # tail -f /var/log/messages | grep "search string" | mail -s "Danger Will Robinson" youremail@yourhost.com

    Even without the -f flag, it still has the flaw where it will email empty alerts (so I'm not giving away the solution, as you asked).

    Hint: you need to tail and grep separately from testing for data that needs to be mailed...

  6. #6
    Just Joined!
    Join Date
    Jun 2011
    Posts
    2
    Hi Guys

    Thanks for all of you advice. Sorry for the delay in replying, things been a bit hectic this last week.

    After reading all of your input and a bit more googling I came up with the following...



    #!/bin/bash


    while true

    do

    tail /var/log/messages | grep "search string" > alert.txt


    while true

    do

    if [ ! -e /home/mikey73/alert.txt ]

    then

    EMAILSUBJECT="***** You've got a problem *****"
    EMAILADDRESS="whoever@needstoknow.com"
    MESSAGEBODY="/home/mikey73/alert.txt"

    echo "syslog says...Houston, we have a problem" >> $MESSAGEBODY

    mail -s "$EMAILSUBJECT" "$EMAILADDRESS" < $MESSAGEBODY

    fi

    mv alert.txt /home/mikey73/oldalerts

    done


    done


    The idea here is that if the search string exists grep will output it to alert.txt

    Then if alert.txt exists the e-mail alert will be generated.

    Unfortunately, it appears that grep touches alert.txt even when there is no output so I get an e-mail regardless of whether the search string exists or not.

    I can't decide wheter or not to leave this looping or just to set a cron to run the script every minute. I just need to get around the problem of grep creating the .txt file without output!! Any ideas???

    Chances are there is a much easier way of achieving this, but it's nice to play around with the script. Linux rocks!!

  7. #7
    Linux Newbie glene77is's Avatar
    Join Date
    Dec 2009
    Location
    Memphis, TN
    Posts
    125
    Mik,

    Just read a post on this problem :

    http://www.linuxforums.org/forum/deb...not-found.html


    digitalamish

    in sh or bash using mail for mailer

    Code:

    x=`grep -c word logfilename
    if [ x -eq 0 ]
    then
    echo "email text" | mail -s subject receiver@domain
    fi
    glene77is

  8. #8
    Just Joined!
    Join Date
    Feb 2009
    Posts
    5
    I'm not exactly sure bash is the greatest language to solve this problem. But I've taken a stab at it myself, based on your code.

    The problem that you run into for starters is one I mentioned in a previous reply - if you use 'tail' with an '-f' or '-F' to follow a file, it never ends and never closes it's output file. So the first problem is how to tail the file. I stole some code from another site to try to address this.

    This seems to work. However, I think that there is the possibility of missing the alerts that you are looking for.

    If the shell is sleeping when a message of interest is written to syslog and then the log is rotated, I'm fairly certain this script will miss it. Frankly, I'm a poor bash coder and don't know a way around this. (Pretend it doesn't exist? Declare victory and take a job in management?)

    On the plus side, it seems to handle a logrotate ok. As long as the logrotate happens while it is asleep...

    Code:
    #!/bin/bash
    
    EMAILADDRESS="whoever@needstoknow.com"
    EMAILSUBJECT="***** You've got a problem *****"
    MESSAGEBODY="/home/mikey73/alert.txt"
    cat /dev/null > $MESSAGEBODY
    
    SEARCHSTRING='search string'
    SYSLOG=/var/log/messages
    LASTOCCURANCE=`egrep -n "$SEARCHSTRING" $SYSLOG | tail -1`
    LASTLINE=`echo $LASTOCCURANCE | cut -d':' -f1`
    
    while true
    do
    
    END=`wc -l $SYSLOG | cut -d' ' -f1`
    if [ $END -lt $LASTLINE ]
    then
            LASTLINE=$END
    fi
    
    tail -n +$LASTLINE $SYSLOG | grep "$SEARCHSTRING" > $MESSAGEBODY
    
    if [ `stat -c %s $MESSAGEBODY` -ne '0' -a $LASTLINE -ne $END ]
    then
    
            echo "syslog says...Houston, we have a problem" >> $MESSAGEBODY
    
            mail -s "$EMAILSUBJECT" "$EMAILADDRESS" < $MESSAGEBODY
    
            ((LASTLINE=$LASTLINE+1))
    fi
    
    LASTLINE=$END
    mv $MESSAGEBODY /home/tommy/oldalerts
    cat /dev/null > $MESSAGEBODY
    sleep 6
    
    done

  9. #9
    Just Joined!
    Join Date
    Mar 2007
    Location
    Bogotá, Colombia
    Posts
    43
    Hi guys!

    I was just thinking for a while ...

    Why just give up on the tail -f ??? Yes, I know the output will never end, but isn't that the whole idea????

    You can pipe the output of the tail | grep to read ... that way you can process one line at a time, something like this:

    Code:
    tail -f /var/log/yourlog | grep "whatever it is you're looking" | while read LINE
    do
      Do what must be done with $LINE ...
    done
    that way you can do whatever it is you need to do with each line (for instance send an email each time) and still receive continuous output from tail.

    cheers!

  10. #10
    Just Joined!
    Join Date
    Feb 2009
    Posts
    5
    Nope. That doesn't work. I just tried it.

    This, of course works.

    From one session I typed
    Code:
    [root@athlon ~]# logger "search string"
    [root@athlon ~]# logger "search string"
    From another I had this running and got this output
    Code:
    [root@athlon ~]# tail -f /var/log/messages | grep "search string"
    Jun 30 12:27:22 athlon root: search string
    Jun 30 12:27:23 athlon root: search string
    ^C
    But the problem still remains that when you pipe the output of the grep into anything, that process is not seeing its input in a timely manner. With 'read' it seems to be waiting until there is a full buffer, then it processes all of the lines at once.

    My code
    Code:
    #!/bin/bash -x
    
    tail -f /var/log/messages | grep "search string" | while read LINE
    do
     echo $LINE ...
    done
    I have to put over 90 "logger 'search string'" commands in another console before I see any output from the script.

    I can't see anything that can alter that behavior.

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •