Find the answer to your Linux question:
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 14
Hello We want to backup a lot of data (+- 50TB!) from a NAS to a hard disks What do you suggest to backup all this efficiently If you advize ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Mar 2013
    Location
    Belgium
    Posts
    20

    Question backup a lot of data


    Hello

    We want to backup a lot of data (+- 50TB!) from a NAS to a hard disks
    What do you suggest to backup all this efficiently
    If you advize to take a software pakket to do this we want a free opensource way to do this.

    Thank you

  2. #2
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,398
    For professional use, bacula comes to mind. It is a versatile and powerful tool, although it does require a bit of infrastructure work and learning effort.
    I used it to backup several hundred machines, and would apply it to several thousands also.
    Bacula, the Open Source, Network Backup Tool for Linux, Unix, and Windows

    However, if it is *only* one NAS that shall be backuped.
    Then maybe a simpler tool based on rsync works also.
    e.g. http://www.rsnapshot.org/
    You must always face the curtain with a bow.

  3. #3
    Just Joined!
    Join Date
    Mar 2013
    Location
    Belgium
    Posts
    20
    Yes indeed all of the 50TB data is stored on the NAS
    I had found rsync already as possibility.
    We have a lot of VTRAK disks where we can store the backup of the data.
    What do you prefer to do this backup efficiently and automatic rsync or Bacula?
    I found Bacula is a free tool but did it need much configuration to do this all?

  4. #4
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,398
    Both are efficient, automatic and relieable.

    But their usecases are different.
    Bacula is a full featured, cross platform, highly configurable backup suite with optional webinterface and statistics.
    But it takes some time to learn and also requires a backend db (postgres or MySQL) and deploy mechanisms for the Clients.
    rsnapshot is much easier. "copy stuff from here to there and keep x versions"


    In your case I recommend to test rsnaphot and see if it fits your needs.
    Even if it does not, you wont loose much time.
    You must always face the curtain with a bow.

  5. #5
    Just Joined!
    Join Date
    Mar 2013
    Location
    Belgium
    Posts
    20
    It would be nice if then end users just can go in time.
    For example one off the users accidentally delte a file but he know that one week ago that file was saved in a directory.
    Then the user can browse to the back-up drive where a read-only file exist of that specific file.
    This is a task in a large company so it must be reliable en efficient

    We also have a database server with his own storage (Not on the NAS) that we want to backup
    On the Database server there or 2 kinds of databases a MySQL database and a PostgresQL database
    Maybe it is possible to configure Bacula to backup the data of the NAS and backup the database server
    if this is possible is maybe better than rsync because than we can back-up everything with on software program
    so everything with backup in centralized
    Last edited by JackieJarvis; 03-13-2013 at 01:06 PM.

  6. #6
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,398
    Iirc: rsnapshot creates a new directory for each run. The dir name is a timestamp.
    The content is always a complete backup. unmodified files compared to the last run are realized via hardlinks.
    You would need additional Software, e.g. samba, to grant network wide readonly Access.

    But as you have more than one box to Backup, bacula becomes more attractive.
    Restore is done via a local client and (gui) console.

    For the db usecase: bacula can execute scripts before and after Jobs.
    Perfect for dumping a database in a File before backup
    You must always face the curtain with a bow.

  7. #7
    Just Joined!
    Join Date
    Mar 2013
    Location
    Belgium
    Posts
    20
    I think Bacula is more attractive for us.
    Then we can do all the backup in one program.

    another question:
    I don't know if you know something about postgresQL with postGIS
    but we have postGIS installed and want to backup the whole database
    But in the past we had tried this and when we restore this on a database with an other version of postGIS
    We got a lot of errors with Functions that are wrong
    i found a answer for this with a special perl script in postGIS or to backup it without functions
    Do you advize something else for this problem?
    Or haven't you any exprerience with this?

  8. #8
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,398
    Ok, I recommend a bacula testsetup first to get experience with the tool and also to use postgresql as catalog db.

    No idea about postGIS.
    You could open another thread, maybe someone else can help.
    You must always face the curtain with a bow.

  9. #9
    Just Joined!
    Join Date
    Mar 2013
    Location
    Belgium
    Posts
    20
    another question:

    If we want to backup the 50TB of data with Bacula, how much storage do we need to provide if we want a good backup solution
    with full and incremtal backup

  10. #10
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,398
    Backup capacity planning depends on these factors:
    - size and duration of a typical full backup
    - size and duration of a typical incremental backup
    - size and duration of a typical differential backup
    - backup strategy (e.g. full on every friday, incremental on all other days)
    - retention time (e.g. 4 weeks)
    - use and effectiveness of compression. (A db dump may have a good compression factor, jpg pictures and movies will not)
    - Network constraints
    - IO constraints on both clients (disks) and server (spooling disk, tape library limits)
    - Database processing. The more files are backuped, the bigger the catalog db will be. I found postgresql more capable of dealing with bacula
    You must always face the curtain with a bow.

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •