Find the answer to your Linux question:
Results 1 to 4 of 4
Hi there, Probably an easy one but can't seem to find it in any searches. Basically I want to back up my log files to another server via rdiff-backup. works ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Nov 2010
    Posts
    2

    rdiff-backup in-use log files


    Hi there,

    Probably an easy one but can't seem to find it in any searches. Basically I want to back up my log files to another server via rdiff-backup. works really well for all the old rotated log files, but the current files won't sync, I'm guessing because they're in use. Duh. I'd rather not disable logging or shut off services, so is there any way to copy the current apache and mail logs while they're still in use? The term "Shadow copy" comes to mind but I think that was a windows thing.

  2. #2
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,356
    In unix, the approach to files is more optimistic, ie: you will never get a "file is in use by another process" error.
    Consequently, there is no need for a "shadow copy", at least for this purpose.


    On the other hand: you need to make sure or be sure, that the file you access is consistent.
    As you can imagine, a logfile is always open and in use.

    You can still watch it via e.g. "tail",
    you can still copy it
    or even write to it.

    But the copy will probably miss some lines.

    The standard approach towards (apache) logfiles is
    1) to use s/th like cronolog in your apache conf for logrotating purposes
    2) and then rsyslog to follow the logfiles and send them to a central rsyslog server
    You must always face the curtain with a bow.

  3. #3
    Just Joined!
    Join Date
    Nov 2010
    Posts
    2
    With regards to file consistency, I figured with rdiff backup, each file may not be 100% at any one time, but each time it's synced the previous missed lines would be caught up. So it would only ever be a few lines behind.

    You say though that linux should have no problems copying files while in use, but that's exactly the problem I'm having. All the old rotated logs are coming through fine, but the current log files aren't coming through the rdiff.

    Pretty basic command I'm using. Maybe there's some magic parameter I'm missing?

    /usr/bin/rdiff-backup -v 2 SOURCESERVER::/var/log /DESTDIR/settings/logs

  4. #4
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,356
    Probably because rdiff-backup realizes that the file changed while copying and therefore discards it.
    Perfectly reasonable logic for a backup application.

    The easy approach is to restrict the backup to rotated files.

    The slightly more sophisticated is the one I outlined above:
    Utilize rsyslog, which is a syslog implementation with quite a lot of usefull features like templates, RELP, multiple input and output modules, etc
    You must always face the curtain with a bow.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •