Find the answer to your Linux question:
Results 1 to 9 of 9
I know the command for backing it up, but can I back up while the server is running and writing to disk or do I have to use a live ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Oct 2009
    Posts
    59

    Back up disk with dd


    I know the command for backing it up, but can I back up while the server is running and writing to disk or do I have to use a live Linux to copy the data.
    Does it also copy the master boot record when I copy the whole disk with dd?
    What if I have a disk that uses only 10 gigs out of 100, when I use dd the backup disk has to have at least 100 gb to be able for dd to copy, right?

  2. #2
    Linux Engineer Kloschüssel's Avatar
    Join Date
    Oct 2005
    Location
    Italy
    Posts
    773
    Code:
    man dd
    Convert and copy a file, write disk headers, boot records, create a boot floppy. dd can make an exact clone of an (unmounted) disk, this will include all blank space so the output destination must be at least as large as the input.
    The disc must be unmounted to use this command. Using it on a server that usually stays online 24h requires to take the server into maintanance during the backup. This is usually a bad idea if I had to decide this.

    Depending on the type of data stored on the disc, you could do the backup with a cronjob that zips the content to a backup server (hereby I assume you have at least RAID6 and other data integrity precautions, to ensure that data is only lost when the server goes directly to hell in fire).

    If I had to build a server that complies to a 99.99999% availability (max 5 minutes down time per year due to planned maintanance), I would bet on having a clustered filesystem spanned over multiple servers. The pro is obvious: Writing to that filesystem transparently does the backup "everytime". And when one server fails, the backup server can jump in and do the job.

  3. #3
    Linux Engineer Freston's Avatar
    Join Date
    Mar 2007
    Location
    The Netherlands
    Posts
    1,049
    Quote Originally Posted by SkyHiRider
    Does it also copy the master boot record when I copy the whole disk with dd?
    Jep, it does when you copy the whole disk.

    Quote Originally Posted by SkyHiRider
    What if I have a disk that uses only 10 gigs out of 100, when I use dd the backup disk has to have at least 100 gb to be able for dd to copy, right?
    Normally yes, because you get a bit for bit copy. But you can pipe dd through gzip of bzip to get a compressed image.


    Perhaps when you want to backup live running servers, `rsync` is better suited... as explained by Kloschüssel.
    Can't tell an OS by it's GUI

  4. $spacer_open
    $spacer_close
  5. #4
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,664
    I've been designing and working with 6-sigma systems used by major manufacturing enterprises (mostly semiconductor and related industries) for many, many years. There are a lot of factors to consider - power, network, disc, processors (server hardware), application servers, database servers, databases, logging, etc. To get true 24x365 live operation you need redundancy and automatic failover for all of these components, and as per your issue of data backup, some strategy for live backups of critical system files and data as well as point-in-time images of the operating system and application binary and configuration data.

    1. To do a point-in-time system image, with the system in an "idle" state, use something like dd and gzip to make a complete image of the system drives, including boot sector and partition tables.
    2. For live data backup, use something like rsync to keep a backup system up-to-date on a regular basis.
    3. For databases, many commercial ones such as Oracle have their own clustering and hot backup tools that will help a lot, though they are usually difficult to set up and configure. They are generally worth the effort for those infrequent but inevitable "Oh ****!" moments, like when there is a power failure and the backup power systems don't function as planned/designed...
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

  6. #5
    Just Joined!
    Join Date
    Oct 2009
    Posts
    59
    Well, I am planning to enable raid 1 on my server, to keep a safe image in case the disk fails. Thanks for mentioning rsync, I will use it for incremental backups over the network to a different pc just in case of a flood, theft, fire etc. Will just set up cron to do it and I should be fine.
    Can anyone please post an example command that backups a whole disk including grub and a command that keeps the image up to date?

  7. #6
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,664
    Well, you would only do the full bit-image backup to an external drive or tape when the system is in maintenance mode. You cannot do that safely when the system is operating normally. So, do it when you are down for maintenance - that would be one of the maintenance jobs. Then, make sure that any file changes are backed up with a cron'd rsync to an external device that can in its turn be incrementally backed up to something like tape between rsync calls. As mentioned, a cron job is the normal way of doing stuff like that.
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

  8. #7
    pom
    pom is offline
    Just Joined!
    Join Date
    Nov 2004
    Posts
    20
    Hi,

    On this dd command,

    I have two HDDs on the my desktop, a 250GB HDD (dual boot XP and Fedora2), df command shows two disk sectors /dev/sda1 and /dev/sda2
    and the second brand new drive 800GB /dev/sdb3.
    I would like to copy the entire 250GB drive onto 800GB. my question is how do I use dd command,
    will this works: dd if=/dev/sda of=/dev/sdb conv=noerror, sync

    thanks for your time.

  9. #8
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,664
    If you use dd to copy the entire disc, then the "image" you will get on the second disc will be the same size as the first, losing all that extra space. What you can do is to first partition the second disc with the first two partitions set to at least the same size as on the first disc, and then use dd to copy the partitions, one at a time. You would also need to install grub on the new drive so it can boot from it (I assume you want to do that).

    Finally, you will need to do these operations when the source disc partitions are not mounted. IE, boot from a live/rescue CD/DVD/USB drive. FWIW, you can use Clonezilla or Gparted boot discs to do this. In fact, rather than using dd, use Clonezilla since it can pretty much do all that you want without a lot of manual futzing about to get the new disc functional.
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

  10. #9
    oz
    oz is offline
    forum.guy
    Join Date
    May 2004
    Location
    arch linux
    Posts
    18,733
    Quote Originally Posted by SkyHiRider View Post
    I know the command for backing it up, but can I back up while the server is running and writing to disk or do I have to use a live Linux to copy the data.
    Does it also copy the master boot record when I copy the whole disk with dd?
    What if I have a disk that uses only 10 gigs out of 100, when I use dd the backup disk has to have at least 100 gb to be able for dd to copy, right?
    You might want to take a look at the possibility of using FSArchiver:

    FSArchiver

    It can restore images to a disk or partition that's a different size than that the image was made from and it handles lots of different filesystem types. I've been using it for a while and it's working perfectly for me. It is command line only, but it's not very hard to handle. Note that it is better to run it on unmounted partitions, though.
    oz

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •