Files Corrupted during cp Process

[ainsof]

Hi,

Here's something I've never seen before.

I've copied a number of ISOs from our old file server (Windows 2000) to our new OpenSuse 11.2 based smb server and a number of them have turned up corrupt. (md5 check fails).

To the best of my knowledge, this shouldn't matter, but files were copied from an NTFS filesystem to Reiser.

Both servers are on the same managed switch. Logs show nothing unusual.

I mapped the old software directory to a local mount point in /mnt, then used cp to copy the ISOs over. Process completed with no errors.

Can anyone suggest what's going on here?

[D-cat]

I've never seen it before either. It's possible you're only dealing with differing HDD block sizes between the file systems and there's no actual damage to the files' data areas. It's also possible that there was data corruption introduced in the pipe, but it just happened to never affect any of the TCP or NFS/SMB control packets; If a file copy isn't verified and no error makes itself known, cp doesn't know whether there was an error or not, it just assumes there was none. cp is not network aware; it doesn't know if a mounted drive is local or network, it doesn't care. That also means that is has no way of detecting errors in a network stream copy as certain other protocols do.

The only thing I might suggest is to "copy" the files again with rsync. With the proper options specified, rsync will hash both source and destination files and recopy only the damaged (differing) parts. This will take a while on large files, but should ensure clean copies.

[HROAdmin26]

1) Ensure you are using a valid apples-to-apples MD5 checksum. IE - use the same MD5 program from one machine. With the Windows share mounted on the SuSE machine, run md5sum on the Linux box against the file before and after the copy.

2) Give details. You didn't specify how you mounted the Windows share - make sure you *don't* use "-t smbfs." SMB is deprecated and does not handle larger files. Use -t cifs.

Also ensure there is no firewall/AV/network filters on the Windows machine affecting the data.

[ainsof]

Thanks D-Cat, I'll give rsync a whirl.

[ainsof]

Thanks HROAdmin26,

Sorry about the lack of detail.

I did use the md5 program from the linux box after mounting the windows share via cifs. Hence my profound confusion.

BTW, no firewalls / filters between the linux and windows boxes at the time.

[HROAdmin26]

I wouldn't put this new system into use until I was sure there isn't a disk, CPU, or memory issue corrupting the data.

Your Windows machine may have some LSP causing problems. I'd work to isolate the issue to the Windows machine or the SuSE machine - like use a third system to mount a SuSE share, calculate MD5, and then copy off the SuSE machine. Also possibly use the third system against the Windows machine too. If it occurs on both, it's likely a network issue.

[ainsof]

Excellent idea. Thanks again.

[D-cat]

Oh yeah... I HAVE seen it.... once when a memory stick was bad. Thanks HROAdmin, I can't believe I forgot that. Definitely do a memtest pass on each machine in addition to HROAdmin's advice, probably first since bad memory has the potential to corrupt all sorts of stuff.