rsync back-up daemon
I want to create a rsync solution for back-up. The back-up is around the 50TB and there is one remote machine who is the director of the operation. Like i think it's gonna go.
The director read the whole 50TB en calculates all the checksums. And it's do the same for the data on the back-up server. So it's knows which file changed since the last back-up. So the director reads twice the 50TB (1 from the storage and 1 from the back-up server). Like i heard you can run Rsync as daemon to increase the speed. Is it true that if i install rsync on the back-up server the director only need to read the data from the storage and calculate the checksum and that the back-upserver calculate the checksum local and send this to the director? This would decrease the load for the network.
I think reading and checksumming 50TB (twice) is always going to be fairly slow :eek: so is it possible to break it into smaller data sets?
We guess that the duration of the back-up is around 9-10 days for the first full back-up. Afterwards only the changed files need to be transferred. Only the checksum needs to be calculate for all the files. But if we use 1 pc who runs rsync the problem is he need to pull the data over the network for the checksum. But my question is. When we install a rsync daemon is it possible to calculate the checksum remotely so it only needs to transfer the checksum of the file to the PC over the network.
I'm not sure, even after reading this. Unless I'm completely misunderstanding it, which is possible as I've only had one coffee, the start up section seems to imply the opposite. Maybe it will help though.
Also just found unison which may or may not be more appropriate.
Thank you for the suggestion!!
We are thinking to just run rsync without checksum so it's only looks at the timestamp.
It's a little less safe than checksum so there is a greater risk for file corruption, but normally it's much quicker
Because, like you said, check summing of 50TB takes a lot of time!