Home directory backup script for newbies

04-11-2006	#1 (permalink)
watchman Just Joined! Join Date: Apr 2006 Location: Maine, USA Posts: 10	Home directory backup script for newbies As a linux newbie I set up a samba server in my home (I have 7 kids and a wife who use computers too and we have a couple dozen of the dang things networked here). I wanted a good backup script to backup all the home directories, and not being much of a programmer I searched the web for a good way to do it. Unfortunately all the scripts I found did not quite suit my needs, specifically: 1. I wanted an email reporting the status of the backup job 2. I wanted to backup each users home directory into a seperate easily restored and compressed file. I didn't want to backup the whole /home directory as the archive size would be very large and unwieldly. 3. I wanted several days worth of backups 4. I wanted each archive file to include a date/time in the filename. 5. I didn't want to have to modify the script simply because a new user was added or one was removed So I wrote the following script, made it executable and added it to cron. It may not be perfect, or perfect in every setup, but it works great for me. I hope this helps another newbie who is trying to come up with a decent "poor man's" backup script. #!/bin/bash # A Simple (Poor Man) disk based backup script # Will backup all user home directories in seperate archives # as a single /home backup file can be quite large and unwieldly. # Does not need to be modified if users are added/deleted # Step - 1 Create Timestamp # BUDTSTAMP = Backup Date/Time Stamp BUTDSTAMP=$(date +%Y%m%d) # Step - 2 Start Email Message To Be Sent # Remove mail message from previous backup # I do this at the beginning of the script instead of the end # in case the mail does not send for whatever reason or # I need to debug it rm mail.txt echo System Backup $BUTDSTAMP >> mail.txt # The email sends a user friendly note showing the start and # end time/dates. This is important so you can compare logs # and see if a backup ran ok. echo Backup Began $(date) >> mail.txt # Step - 3 Rotate Backups # Simple three backupset rotation, keeps only last three # Use directory /vol/backup as an example, make sure you change this path # to fit your local settings. Have three directories in /vol/backup named daily, # yesterday and daybefore. # Make sure /vol/backup is on a different disk (preferably a different machine) # than the files you are backing up rm /vol/backup/daybefore/* mv /vol/backup/yesterday/* /vol/backup/daybefore/ mv /vol/backup/daily/* /vol/backup/yesterday/ # Step - 4 Archive Home Directories # Creates a seperate tar file for each directory in the home directory # Remember that root's home does not usually reside in /home so add it # and any other directories you want to back up, i.e.: # tar -czvf /vol/backup/daily/root-$BUTDSTAMP.tar.gz /root # tar -czvf /vol/backup/daily/itunesmusic-$BUTDTAMP.tar.gz /share/music cd /home for FOLDERNAME in * do # Archives are created in the format someuser-datetime.tar.gz tar -czvf /vol/backup/daily/$FOLDERNAME-$BUTDSTAMP.tar.gz /home/$FOLDERNAME done # Step - 5 Finish Email Report and Send echo Backup ended $(date) >> mail.txt # df -h includes a human readable disk usage report of the media that /vol/backup # is mounted on. Good to now if your backup disk is running out of space. # Of course /dev/hdb1 is the device I use, modify it for your local settings df -h /dev/hdb1 >> mail.txt mail somebody@somedomain.com -s "Backup Job Report" -v < mail.txt # Step - 6 All Done

04-12-2006

#2 (permalink)

jpalfree

Linux Newbie

Join Date: Jul 2005

Location: Montreal, CA

Posts: 198

nice idea. thanks for posting it.
I've tried to make your script a bit more general and safe (not that it was dangerous to begin with, i'm just a paranoid person)
Check it out. It now can backup multiple directories by simply adding them to a list.

Quote:

#!/bin/bash
# A Simple (Poor Man) disk based backup script
# Will backup all user home directories in seperate archives
# as a single /home backup file can be quite large and unwieldly.
# Does not need to be modified if users are added/deleted

# Step - 1 Create Timestamp and set up variables and functions
# BUDTSTAMP = Backup Date/Time Stamp
BUTDSTAMP=$(date +%Y%m%d)

# variable holding directories containing files to backup eg: BACKUPTHESE="/home /root /etc"
BACKUPTHESE="/home"

#directory containing today's backup
BKUPDIR="/vol/backup/daily"
#directory containing yesterday's backup
YDBKUPDIR="/vol/backup/yesterday"
#directory containing the day before yesterday's backup
DBBKUPDIR="/vol/backup/daybefore"

# log files
LOG="mail.txt"
ERRORLOG="ERROR.txt"

# tag for backup file name
TAG="homebackup"

# function that quits and logs on error
exiterror()
{
# use this function by supplying $LINENO as first arg
echo "Fatal error caused by line ${1} of ${0}" >> $ERRORLOG
mail somebody@somedomain.com -s "Backup Job ERROR" -v < $ERRORLOG
mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG
exit 1
}

# Step - 2 Start Email Message To Be Sent

# Remove mail message from previous backup
# I do this at the beginning of the script instead of the end
# in case the mail does not send for whatever reason or
# I need to debug it

rm $LOG > /dev/null 2>&1
echo "System Backup $BUTDSTAMP" >> $LOG

# The email sends a user friendly note showing the start and
# end time/dates. This is important so you can compare logs
# and see if a backup ran ok.

echo "Backup Began $(date)" >> $LOG

# Step - 3 Rotate Backups
# Simple three backupset rotation, keeps only last three
# Use directory /vol/backup as an example, make sure you change this path
# to fit your local settings. Have three directories in /vol/backup named daily,
# yesterday and daybefore.
# Make sure /vol/backup is on a different disk (preferably a different machine)
# than the files you are backing up

#looks specifically for backup files in case other files are kept in these directories
for bkfile in $DBBKUPDIR/*; do
echo $bkfile | grep $TAG >/dev/null &&
(rm $bkfile || exiterror $LINENO )
done
for bkfile in $YDBKUPDIR/*; do
echo $bkfile | grep $TAG >/dev/null &&
(mv $bkfile $DBBKUPDIR/ || exiterror $LINENO )
done
for bkfile in $BKUPDIR/*; do
echo $bkfile | grep $TAG >/dev/null &&
(mv $bkfile $YDBKUPDIR/ || exiterror $LINENO )
done

# Step - 4 Archive Home Directories
# Creates a seperate tar file for each directory in the directories in $BACKUPTHESE

# change to dir to backup. Check in case cd failed for some reason.
cd $BACKUPTHIS || exiterror $LINENO
#backup all files listed in BACKUPTHESE
for DIRTOBACKUP in $BACKUPTHESE; do
for FOLDERNAME in $DIRTOBACKUP/*
do
# Archives are created in the format someuser-$TAG-datetime.tar.gz
# basename is used here so as not to include absolute paths
# -p preserves permissions
echo -e "-------------------\n>>>taring ${FOLDERNAME}\n\n" >> $LOG
tar -czvpf ${BKUPDIR}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} >> $LOG || exiterror $LINENO
done
done

# Step - 5 Finish Email Report and Send

echo "Backup ended $(date)" >> $LOG

# df -h includes a human readable disk usage report of the media that /vol/backup
# is mounted on. Good to now if your backup disk is running out of space.
# Of course /dev/hdb1 is the device I use, modify it for your local settings

df -h /dev/hdb1 >> $LOG
mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG

# Step - 6 All Done

__________________
Avatar from xkcd.com, a hilarious computer related webcomic.

04-12-2006	#3 (permalink)
stokes Linux User Join Date: Oct 2004 Location: UK Posts: 260	This is pretty useful, thanks for posting! __________________ Registered Linux user #389109 My Linux Blog

04-13-2006

#4 (permalink)

watchman

Just Joined!

Join Date: Apr 2006

Location: Maine, USA

Posts: 10

Quote:

Originally Posted by jpalfree

I've tried to make your script a bit more general and safe.

Thanks, though I will humbly admit you have quite a bit more scripting knowledge than I, I have copied your script and have learned quite a bit from it.

That's why a forum like this is great.

04-13-2006	#5 (permalink)
genesus Linux Engineer Join Date: Oct 2004 Location: Vancouver Posts: 1,366	wow, nice script watch, I've been going over the base script and mods now...looks like something I may implement! __________________ Operating System: GNU Emacs

04-13-2006	#6 (permalink)
antidrugue Linux Guru Join Date: Oct 2005 Location: Montreal, Canada Posts: 3,213	Yes, very nice, thanks guys. Why not tar it in bzip2 instead of gzip? __________________ "To express yourself in freedom, you must die to everything of yesterday. From the 'old', you derive security; from the 'new', you gain the flow." -Bruce Lee

04-14-2006

#7 (permalink)

jpalfree

Linux Newbie

Join Date: Jul 2005

Location: Montreal, CA

Posts: 198

Quote:

Originally Posted by antidrugue

Yes, very nice, thanks guys.

Why not tar it in bzip2 instead of gzip?

you're probably right, antidrugue. bzip2 would be better in this case. actually another fix would be to renice the tar process so that it works nicely in the background, like so:

Quote:

...
nice --adjustment=20 tar -cjvpf ${BKUPDIR}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.bz2
...

there might be a way to renice the whole script but I am unfamiliar with one.

__________________
Avatar from xkcd.com, a hilarious computer related webcomic.

09-28-2006	#8 (permalink)
simonsez Just Joined! Join Date: Jul 2005 Posts: 6	Is this script written in Perl? And would I need to execute it from the cgi-bin? thanks.

09-28-2006	#9 (permalink)
bigtomrodney /etc/init.d/moderator Join Date: Nov 2004 Location: Sunny South-East of Ireland Posts: 6,031	No this script is written in bash, as you can see in the shabang Code: #!/bin/bash All you need to do is to save it to a file, and make that file executable Code: chmod +x mybackupscript.sh __________________ Registered Linux user #378740 New members read here / Forum Rules #linuxforums on irc.freenode.net