rsync not doing incremental backups???

[tqz]

Hello all

I am using rsync to backup dirs on my ubuntu server onto a NAS (which is mounted onto the filesystem), but the problem is that it is constantly doing full backups rather than doing incrementals and I am not really sure why.

After doing a bit of expermienting with the script I noticed that if I just backed up a home dir (/home/user) the incremental backups work fine. If however I was to back up a dir like (/home/domain/user) it always does full backups.

I have tried various different scripts but still the same end result.

The latest script is a variation on the a script found on the samba rsync examples webpage, see below...

#!/bin/bash
# rsyncbu.sh -- backup to nas using rsync

# This script backups files listed in BDIR to the BSERVER. The verbose output along with the date is listed in the LOG_FILE specified

# verbose output
VERBOSE="-v"

# directory to backup
BDIR=/home/abc

# excludes file - this contains a wildcard pattern per line of files to exclude
EXCLUDES=/bin/exclude_file.txt

# the name of the backup machine
BACKUPDIR=/NAS/test/

LOG_FILE="/home/def/testfile.txt"

# delete contents of log so it doesnt grow too large
echo > $LOG_FILE

#add date to the first line of the log file
string=`date +%m/%d/%y`
echo $string >> $LOG_FILE


OPTS="--delete-excluded --exclude-from=$EXCLUDES --delete --backup --backup-dir=/$BACKUPDIR -rt"


# transfer files
rsync $VERBOSE $OPTS $BDIR $BACKUPDIR >> $LOG_FILE
Any ideas what I am doing wrong???

Thanks in adv.

t.

[tqz]

Okay so I have done some more investigating and have found the following:-

It seems that the reason why a full backup is being performed rather than an incremental is because the modification times of the files are different when the files are copied over.

So if i had a file called test.doc on my ubuntu server its file properties may be:-

Created 18 Jan 2010 10:45:11
Modified 18 Jan 2010 10:45:11

Rsync copies the file over to the NAS box and the file properties for test.doc on there would be:-

Created 18th April 2009 17:44
Modified 18 Jan 2010 10:45:10

The creation times are completely different but the modification times seem to be out by only a second.

Is this simply a case of changing the NAS box date and time settings (I dont have the perms to do this, so cant just simply try this that why posting on here before getting others involved)...and why is that only certain dirs are affected???...

Thanks for any responses in advance!

[Rubberman]

You need to use the --times (or -t) option to make sure that the target has the same time stamp as the source. Also, you should make sure that both your system and your NAS are time-sync'd to the same NTP source so their internal clocks are in agreement.

[tqz]

Hi Rubberman and thanks for your response!

I am using the t option (if you see one of my many scripts in the first post i have a line OPTS="--delete-excluded --exclude-from=$EXCLUDES --delete --backup --backup-dir=/$BACKUPDIR -rt").

I have also got someone to change the date and time on the NAS boxes, but apart from correcting the creation dates of the files it still hasnt solved the problem! The modification and now the creation times are out by 1 second!

The NTP sources of both should be ntp.ubuntu.com...

I have a feeling that the full backups are occuring for when the files are shared using samba (so my files are stored on ubuntu but the files which are accessed and created by the windows clients?)...these files however make up the bulk of what is stored on my ubuntu server...

So any further suggestions??

[Rubberman]

If I were consulting for you on this, I'd need to poke around your file systems and try some controlled experimentation with backup processes to see what the true root cause is. As it stands now, all we can do is to advise you where to look and at the moment I am plumb out of ideas.

As for the 1 second differences in the time stamps, I'm not sure why that is happening, though it would be enough to make rsync think the backup files need to be refreshed from source. You might want to verify that the NAS devices are properly syncing with the NTP service. Since you have apparently configured them to use the same address as your file server are using, they should show the same date+time as your Ubuntu server. That makes me think that either the Ubuntu server, or the NASes, are not properly syncing their clocks. You can check in the NTP log file, if there is one, or in /var/log/messages if there isn't. My CentOS ntpd logs the sync events to /var/log/messages. Try: sudo grep -i ntp /var/log/messages

[jplunario]

rsync -avztr --modify-window=1 SOURCE DESTINATION