[SOLVED] cron & grep

[charlie205]

Anyone have any ideas? I'm at a loss...

Why would a script make grep matches from the shell, but not from crontab? I thought it might be a permissions issue, so i set both the script and the file "important_dates" to 777. The tempfile is created and deleted within the script, so I'm not sure how it's permissions are set. Also, I fully qualified the path to every file involved.

The below script is designed to read a particular file (important_dates) grep for particular lines based on the date. Then email me the results.

It finds matches and works perfectly from the command prompt when I execute it with

sh test.sh

but it does not find any matches when i run it from cron

*/2 * * * * /path/to/test.sh 1>/path/to/results.out 2>&1

here is the script:

#!/bin/bash

TODAY=`date +%j`

DATETODAY=`date "+%A %B %d"`

FIRST=`expr $TODAY + 5`
if [ $FIRST -gt 366 ] ; then
FIRST=`expr $FIRST - 366`
fi

SECOND=`expr $FIRST + 1`
if [ $SECOND -gt 366 ] ; then
SECOND=`expr $SECOND - 366`
fi

THIRD=`expr $SECOND + 1`
if [ $THIRD -gt 366 ] ; then
THIRD=`expr $THIRD - 366`
fi

FOURTH=`expr $THIRD + 1`
if [ $FOURTH -gt 366 ] ; then
FOURTH=`expr $FOURTH - 366`
fi

FIFTH=`expr $FOURTH + 1`
if [ $FIFTH -gt 366 ] ; then
FIFTH=`expr $FIFTH - 366`
fi

SIXTH=`expr $FIFTH + 1`
if [ $SIXTH -gt 366 ] ; then
SIXTH=`expr $SIXTH - 366`
fi

SEVENTH=`expr $SIXTH + 1`
if [ $SEVENTH -gt 366 ] ; then
SEVENTH=`expr $SEVENTH - 366`
fi



date
echo "Today is $DATETODAY" > /path/to/tempfile
echo >> /path/to/tempfile
chmod 777 /path/to/tempfile
echo "$TODAY, $FIRST, $SECOND, $THIRD, $FOURTH, $FIFTH, $SIXTH, $SEVENTH" >> /path/to/tempfile
grep -w "\^^$FIRST" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile
grep -w "\^^$SECOND" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile
grep -w "\^^$THIRD" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile
grep -w "\^^$FOURTH" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile
grep -w "\^^$FIFTH" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile
grep -w "\^^$SIXTH" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile
grep -w "\^^$SEVENTH" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile

mail user@domain.com < /path/to/tempfile -s "Upcoming Events"


rm -f /path/to/tempfile

[Ben Cotton]

Does anything get mailed? If so, what? If not, comment out your rm and post what gets written to your temp file. Could you also post your results.out file? Nothing strikes me as being obvious in your script, but I could be missing something.

[charlie205]

Does anything get mailed? If so, what? If not, comment out your rm and post what gets written to your temp file. Could you also post your results.out file? Nothing strikes me as being obvious in your script, but I could be missing something.

yes, i receive an email that has the following content:

Today is Wednesday July 16

198, 203, 204, 205, 206, 207, 208, 209


I commented out the rm line and looked at the tempfile...it matches the email

also, this is the contents of results.out:

[user@host]# cat results.out
Wed Jul 16 21:28:01 EDT 2008


So it looks like every function works properly except for the grep part. So either grep isn't finding matches or grep is finding matches, but not writing them to tempfile

I have no idea...i'm still troubleshooting...thanks for your help, let me know if you have any ideas, they are much appreciated.

[charlie205]

I've figured out where it's failing.

grep -w "\^^$FIRST" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile

I figured this out by some standard debugging. The >> isn't writing to the tempfile. This makes sense when you consider the symptoms. So here's the question:

Why would >> work from the shell but not from cron?

[Ben Cotton]

I want to say the problem is that the cron environment is slightly different from your login environment (have anything interesting in ~/.bashrc or ~/.profile?), but I'm not enough of a bash pro to say for sure.

So the question is: why does the mail need to occur in the script? If you have the script print everything to STDOUT, and don't redirect the cron output, cron will mail you the output. You could try that at least temporarily to help isolate the problem.

[scm]

The most likely explanation is that the grep pattern isn't matching - can you take out the cut and see what that produces?

>> works fine through a pipeline for me in a simple test, so could try letting the cut feed stdout and redirect the cron command to /path/to/tempfile.

[charlie205]

Ben, as to your question: mail needs to be executed from the script because the purpose of the script is to send mail to a non-local account.

The most likely explanation is that the grep pattern isn't matching - can you take out the cut and see what that produces?

>> works fine through a pipeline for me in a simple test, so could try letting the cut feed stdout and redirect the cron command to /path/to/tempfile.

You are correct. I should have said that this line:

grep -w "\^^$FIRST" /path/to/important_dates | cut -d " " -f 2-20 >> /path/to/tempfile

is failing NOT NECESSARILY that the >> was failing. Further tests proved you correct. I removed the redirection and tested the grep by itself and got the same results (i.e. worked from sh, not from cron).

What IS happening is grep not finding any matches. Why would grep find a match from SH but not from cron??

I'm appending the contents of my .bash_profile per the request

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH
unset USERNAME

[scm]

First check you're using the same grep in both environments - type "which grep" and then put the full path to grep in the script.

The other possibility is that somehow your pattern is being interpreted differently by the shell when your script is run via cron (though I can't think why it should be, but the "\^" intrigues me - what does your nput file look like?) so try a simpler one and see if that works. If it does, we can go forward from there.

[charlie205]

[user@host]# which grep
/bin/grep

I modified the grep command in the script to include the full path as seen above, but got the same results. I am going to try the simplifying the ^^pattern now and see if that changes anything.

[charlie205]

The other possibility is that somehow your pattern is being interpreted differently by the shell when your script is run via cron (though I can't think why it should be, but the "\^" intrigues me - what does your nput file look like?) so try a simpler one and see if that works. If it does, we can go forward from there.

Thats it!! I changed ^^ to @@ in both the script and the input file. Now it works like a charm! Awesome!!

Thanks for all the help!