Problem with Crontab (or perhaps svnadmin)

[JanxnaJ]

I've been using crontab for running backup scripts on my linux file server, and it's working perfectly... well, mostly. One part of my backup is to perform a backup (svnadmin dump) of my svn repositories, and I have a strange problem here which seems to be related to crontab.

The crontab I'm using is the root crontab, edited by being root and running "crontab -e", and this is what it contains:

# m h  dom mon dow   command
00 03 * * 1 /bin/raidbackup.sh

I've created a minimal version of the script /bin/raidbackup.sh to isolate the problem. This is it:

#!/bin/bash

TGTDIR=/srv/dshare/backup/rshare/raidbackup_$(date +%Y%m%d-%H%M%S)/
mkdir $TGTDIR

# Copy (dump) repositories from the raid share
mkdir ${TGTDIR}svndump
svnadmin dump /srv/rshare/svn/java > ${TGTDIR}svndump/java
svnadmin dump /srv/rshare/svn/php > ${TGTDIR}svndump/php

The backup runs (as you can see) monday morning at 03:00, and after this run the backup directory contains these files:

-rw-r--r-- 1 root root 36589134 2011-09-22 21:31 java
-rw-r--r-- 1 root root      195 2011-09-22 21:31 php

This is a bit strange, since I know that my php repository is noticably larger than my java repository. However, if I choose to run the script /bin/raidbackup.sh manually as root the result is a different one:

-rw-r--r-- 1 root root  49265304 2011-09-22 21:35 java
-rw-r--r-- 1 root root 202055932 2011-09-22 21:35 php

This dump seems to be correct.

The question is: what could possibly cause the script to produce an incorrect result when running through crontab, but a correct result when executed manually as root?

I hope someone can give me an idea of what could be wrong as this problem really puzzles me. Of course it may be entirely related to svnadmin, in which case this is probably not the right forum.

Anyway... any suggestions are most welcome.

[ptkobe]

Shouldn't the dates of the cron backup files be 03:00 of a Monday and not 21:31 of 2011-09-22? I ask it to check if you are sure that it is the root cron job that is producing wrong results.

Did you try to see the results of running the script as you, not root?

Sorry, can't think of anything really useful.
Luis

[atreyu]

Perhaps there are script errors you are missing when the cronjob runs? Try this to send it to a log:

00 03 * * 1 /bin/raidbackup.sh > /tmp/raidbackup-cronjob.log 2>&1

You can set -x at the top of the script too, to enable a bunch of Bash debug output.

Also, anything helpful in /var/log/cron?

[JanxnaJ]

Shouldn't the dates of the cron backup files be 03:00 of a Monday and not 21:31 of 2011-09-22? I ask it to check if you are sure that it is the root cron job that is producing wrong results.

Did you try to see the results of running the script as you, not root?

Sorry, can't think of anything really useful.
Luis

Very sharp observation, ptkobe. Yes, the timestamps should be (and are) 3 am when running the normal weekly backup. However, I tried running it repeatedly by changing the time in crontab to try to isolate the problem, and the output pasted are from some of these attempts.

I also tried running the scripts with my user, but then I weren't even allowed to read all the necessary directories.

Thank you for your suggestions.

/Jan

[JanxnaJ]

Perhaps there are script errors you are missing when the cronjob runs? Try this to send it to a log:

00 03 * * 1 /bin/raidbackup.sh > /tmp/raidbackup-cronjob.log 2>&1

You can set -x at the top of the script too, to enable a bunch of Bash debug output.

Also, anything helpful in /var/log/cron?

Thanks atreyu.

These are good suggestions. I'll try this as soon as I get home.

/Jan

[JanxnaJ]

Perhaps there are script errors you are missing when the cronjob runs? Try this to send it to a log:

00 03 * * 1 /bin/raidbackup.sh > /tmp/raidbackup-cronjob.log 2>&1

You can set -x at the top of the script too, to enable a bunch of Bash debug output.

Also, anything helpful in /var/log/cron?

Well, it seems my problem is solved by following your suggestions, atreyu. But I'm even more puzzled than before.

There was nothing in /var/log/cron (should it be a file or directory? - anyway it doesn't exist), so I followed your other suggestion of sending the output to the log. I wrote exactly as you suggested (well, with another path). And then my svnadmin dump produced the full dump.

So, I created the following reduced script (/bin/testcron.sh):

#!/bin/bash

TGTDIR=/srv/dshare/backup/rshare/raidbackup_$(date +%Y%m%d-%H%M%S)/
mkdir $TGTDIR

mkdir ${TGTDIR}svndump
svnadmin dump /srv/rshare/svn/java > ${TGTDIR}svndump/java
svnadmin dump /srv/rshare/svn/php > ${TGTDIR}svndump/php

When my crontab contains this, the svnadmin dump produces an incorrect result:

# m h  dom mon dow  command
03 * * * * /bin/testcron.sh

But when I redirect output to a log file, it works:

# m h  dom mon dow  command
03 * * * * /bin/testcron.sh > /var/log/testcron_output.log 2>&1

This seems really weird to me. Does anyone have a logical explanation for this? Could it be something with svnadmin dump failing when unable to produce an output? Sounds unlikely I guess.

Anyway, thank you very much for your help. I should now be able to sleep well at night knowing my repositories are fully backed up

[atreyu]

If you really want to know, maybe you could try just redirecting STDERR to a log, and see if there are any clues there, e.g.:

00 03 * * 1 /bin/raidbackup.sh 2> /tmp/raidbackup-cronErr.log

maybe something in the script wants a TERM to be able to write to or something, but if you redirect to logs that avoids it?

just guessing, but anyway glad it's working for you!

[ptkobe]

Found this: Reasons why crontab does not work - Ask Ubuntu - Stack Exchange

I have already stumbled on the no final empty line issue. Hope it may help.
Regards
Luis