Find the answer to your Linux question:
Results 1 to 9 of 9
I've been testing our backups here at my office. After I write the tar file to the tape drive and remove it, the cksum & md5sum are different. I tried ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Jun 2004
    Posts
    5

    cksum & md5sum fingerprints change after tar...HELP!!!


    I've been testing our backups here at my office. After I write the tar file to the tape drive and remove it, the cksum & md5sum are different. I tried small files at first, with cksum it was different everytime.

    If I turned hardware compression off on the st, the md5sum came up the same on a small file. When I tried our 5.5GB tar file, the md5sum came out different. I untar'd the file and pulled everything out of it, as far as I could tell, everything's the same. Should I be alarmed by this?

    The only differences are this, the original is named nightly-backup.tar and located in /backups. After it's writen to the tape, it's renamed to lastweek-backup.tar. I pulled the original off the tape and stored in /backups/restore as nightly-backup.tar. Will this cause the differences?

    This is what I get from an md5sum of the two:

    f9a64b1c9c54e0316141c0f4a85e4ea7 lastweek-backup.tar
    3036927f219a752359bf5b005a2e7e99 nightly-backup.tar

    This is what I get from a cksum of the two:

    251230307 5562952704 lastweek-backup.tar
    1523033174 5562952704 nightly-backup.tar

    I tried renaming the files so they'd have the same name, just different location, and the numbers still don't match.

    Any help is appreciated.

    TIA.

    PS
    I did a search before posting and couldn't find anything helpful, but I may not know what it is I'm looking for...

  2. #2
    Just Joined!
    Join Date
    Jun 2004
    Posts
    5
    Anybody????

  3. #3
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578
    I don't have that much tape drive experience, but I think I know what it is. If I recall correctly, the tape doesn't keep an EOF record, and thus it reads until the last of the stored block. Thus, if the tar file's size isn't a multiple of the tape's block size, the read file will have some trailing garbage. Tar will ignore the trailing garbage, but md5sum naturally won't.

    If you copy a tarball to the tape and then back again, do the sizes match?

  4. $spacer_open
    $spacer_close
  5. #4
    Just Joined!
    Join Date
    Jun 2004
    Posts
    5
    Quote Originally Posted by Dolda2000
    If you copy a tarball to the tape and then back again, do the sizes match?
    Yes, the sizes are the same, in the output of the ck sum below, the second number is the size:

    251230307 5562952704 lastweek-backup.tar
    1523033174 5562952704 nightly-backup.tar

    The tape is a reman DLT8000. I'm just trying to rule out that it's defective. If I use a small file (~20MB) the cksum and md5sum is the correct, but only if I turn hardware compression off.

    Thanks for the info, I'll check into it the EOF thing. Thanks!

  6. #5
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578
    Try making a hexdump of the files and diff them to see where they are different. Try something like this:
    Code:
    diff -u <&#40;od -t x1 lastweek-backup.tar&#41; <&#40;od -t x1 nightly-backup.tar&#41;
    Try only with files smaller than 2 GBs... I'm not sure how od would do with the offsets if the files are too large.
    In particular, check the offsets where the files differ. If it's very close to the end, then it's most likely the EOF thing.

  7. #6
    Just Joined!
    Join Date
    Jun 2004
    Posts
    5
    The od worked fine, but the diff can't handle the files, I get 'diff: Memory exhausted'. I did a tail on the two files and they are different from there. Even the files sizes of the od output are different:

    433291264 Jun 24 16:12 od_lastweek.log
    441884672 Jun 24 16:12 od_nightly.log

    The nightly-backup.tar is the one off the tape, the script renames the original nightly to lastweek after it's been written to the tape.

    Someone else suggested the dump command. I'm not familiar with it at all and am curious how to write one file onto the tape with it, and then pull that one file off the tape.

    TIA!!

  8. #7
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578
    Hmmm... try with a couple of smaller files. That way diff probably wouldn't have the same trouble.

  9. #8
    Just Joined!
    Join Date
    Jun 2004
    Posts
    5
    I tried it with a windows .zip file that was roughly about 920MB (it contained two txt files that were about 460MB each). The cksums where different after putting it on the tape and taking it off. I tried to unzip it from the windows side and got:

    Extracting Copy of test2.txt
    bad CRC 0ccee123 (should be c6dbf9a2)
    Extracting test2.txt
    bad CRC 1c2db79b (should be c6dbf9a2)

    Do you think the tape drive is corrupting the file?

  10. #9
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578
    That indeed seems as a bad thing.
    On the other hand, as far as I know, Zip files store the file directory at the end of the Zip file, in which case it could be understandable if the file is extended with garbage. Tar, on the other hand, is specifically designed to cope with such things.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •