Find the answer to your Linux question:
Results 1 to 2 of 2
Hey everyone, Anyone familiar with rsync, this one's for you! Simple task: * Backup a directory (say, /home, or /storage/games) to a backup location (say, /mnt/my_server/backup). * The source is ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Jan 2005
    Posts
    54

    Unhappy rsync - Destination (NTFS) Larger than Source (EXT3)??


    Hey everyone,

    Anyone familiar with rsync, this one's for you!

    Simple task:
    * Backup a directory (say, /home, or /storage/games) to a backup location (say, /mnt/my_server/backup).
    * The source is NTFS, and the destination is EXT3 (mounted via cifs/smbfs).

    The size of the home directory:
    Code:
    du -hs /storage/games
    returns 80GB

    The command:
    Code:
    rsync -avhcSz --delete --progress --stats /storage/games /mnt/my_server/backup/games
    This will run for a few hours .. then, when it's all said and done ..

    The size of the backed up home directory:
    (ssh'd into the destination box...)
    Code:
    du -hs /storage/backup/games
    returns
    111+GB

    What gives?

    I've tried the same on two other directories, and the destination result is always HUGE compared to the actual source when going from NTFS to EXT3. As you can see, my rsync command has become a little more "complicated", but I started out with even the simplest test of:
    Code:
    rsync -avh --progress
    , and even that gave the problems. That's when I started reading the rsync man more carefully trying to see if there were switches to fix this problem. (Especially the --delete switch, and the -S switch I figured would fix it ... but nope)

    Please note that my "/mnt/my_server" is a CIFS (SMBFS) mount to a samba fileserver.

    Does anyone have any idea what's going on with the filesize?
    1. I'm worried about data integrity
    2. I'm worried that if the original size is 80GB, and my backup size gets to be 111GB somehow ... then if I lose my data some day and need to restore the backup, will it take 80GB, 111GB, or MORE GB???

  2. #2
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,574
    There are a number of reasons why your backup on ext3 file systems will differ in size from NTFS as reported by du.

    1. du, like df, only reports only what the file system tells it. The NTFS driver is getting size information from the NTFS directories, and the EXT3 driver from the inodes on the EXT3 file system. One may round the results to the sector size and the other might not. The size of directories that contain many files may differ significantly.
    2. The sector size used by the file system may well differ. It might be 512 bytes on one and 2048 bytes on another. If you have a lot of small files, the latter file system will use a lot more disc space. NTFS was a VAST improvement over FAT file systems in space utilization. FYI, the sector size used in Linux file systems depends a lot on the actual disc size, and is configurable with mkfs when the file system is formatted, so it can be tuned for systems with a lot of small files, or fewer big files, as can NT file systems when they are formatted.

    In my experience, the restore will take about the same space as the original data set, since the actual file I/O is absolute size based, not rounded data that du and similar tools may show. The best way to tell, is to go to the source file system and pick a directory that has a number of files in it, then run "ls -l" to show the actual size in bytes of the files and that directory. Then, go to the backup version of the same directory and run "ls -l" again, just to verify what the actual physical size of the files are and that they are the same.
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •