Find the answer to your Linux question:
Results 1 to 6 of 6
Like Tree1Likes
  • 1 Post By elija
Hello I want to create a rsync solution for back-up. The back-up is around the 50TB and there is one remote machine who is the director of the operation. Like ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Mar 2013
    Location
    Belgium
    Posts
    20

    Question rsync back-up daemon


    Hello

    I want to create a rsync solution for back-up. The back-up is around the 50TB and there is one remote machine who is the director of the operation. Like i think it's gonna go.
    The director read the whole 50TB en calculates all the checksums. And it's do the same for the data on the back-up server. So it's knows which file changed since the last back-up. So the director reads twice the 50TB (1 from the storage and 1 from the back-up server). Like i heard you can run Rsync as daemon to increase the speed. Is it true that if i install rsync on the back-up server the director only need to read the data from the storage and calculate the checksum and that the back-upserver calculate the checksum local and send this to the director? This would decrease the load for the network.

    Thanks

  2. #2
    Penguin of trust elija's Avatar
    Join Date
    Jul 2004
    Location
    Either at home or at work or down the pub
    Posts
    3,535
    I think reading and checksumming 50TB (twice) is always going to be fairly slow so is it possible to break it into smaller data sets?
    What do we want?
    Time machines!

    When do we want 'em?
    Doesn't really matter does it!?


    The Fifth Continent

  3. #3
    Just Joined!
    Join Date
    Mar 2013
    Location
    Belgium
    Posts
    20
    We guess that the duration of the back-up is around 9-10 days for the first full back-up. Afterwards only the changed files need to be transferred. Only the checksum needs to be calculate for all the files. But if we use 1 pc who runs rsync the problem is he need to pull the data over the network for the checksum. But my question is. When we install a rsync daemon is it possible to calculate the checksum remotely so it only needs to transfer the checksum of the file to the PC over the network.

  4. #4
    Penguin of trust elija's Avatar
    Join Date
    Jul 2004
    Location
    Either at home or at work or down the pub
    Posts
    3,535
    I'm not sure, even after reading this. Unless I'm completely misunderstanding it, which is possible as I've only had one coffee, the start up section seems to imply the opposite. Maybe it will help though.
    What do we want?
    Time machines!

    When do we want 'em?
    Doesn't really matter does it!?


    The Fifth Continent

  5. #5
    Penguin of trust elija's Avatar
    Join Date
    Jul 2004
    Location
    Either at home or at work or down the pub
    Posts
    3,535
    Also just found unison which may or may not be more appropriate.
    Pimpntel likes this.
    What do we want?
    Time machines!

    When do we want 'em?
    Doesn't really matter does it!?


    The Fifth Continent

  6. #6
    Just Joined!
    Join Date
    Mar 2013
    Location
    Belgium
    Posts
    20
    Thank you for the suggestion!!
    We are thinking to just run rsync without checksum so it's only looks at the timestamp.
    It's a little less safe than checksum so there is a greater risk for file corruption, but normally it's much quicker
    Because, like you said, check summing of 50TB takes a lot of time!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •