Find the answer to your Linux question:
Results 1 to 4 of 4
What is the most effective bash script according to performance to find and copy files from one LINUX server to another using specifications described below. I need to get a ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Sep 2012
    Posts
    2

    Copy files by creation time and specific conditions


    What is the most effective bash script according to performance to find and copy files from one LINUX server to another using specifications described below.

    I need to get a bash script which finds only new files created in server A in directories with name "Z" between interval from 0 to 10 minutes ago. Then transfer them to server B. I think it can be done by formatting a query and executing it for each founded new file scp /X/Y.../Z/file root[eta]hostname:/X/Y.../Z/ If script finds no such remote path on server B it will continue copying second file which directory exists. File should be copied with permissions, group, owner and creation time.

    X/Y... are various directories path. I want setup a cron job to execute this script every 10 minutes. So the performance is very important in this case.

    Thank you.

  2. #2
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    you could combine find with rsync. here's an example:

    backup - rsync only files created or modified after a date and time - Stack Overflow

  3. #3
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,356
    I would advise against cron here.
    Because on each run, the whole directory tree needs to be read and a filelist is generated, which is very IO heavy.
    Instead, use lsyncd

    lsyncd is based on inotify, so you can act on hooks like write/close.
    There is also logic to gather changes for some time in order to minimize the number of sync runs.
    Your requirement of filenamechecking can be met also, even complex logic is possible via lua.

    One downside is, that this fails if one of the machines or the network is down.
    To recover from that and prevent inconsistencies, you could have additional "full" runs once a day, which sync the directories on both machines, regardless of timestamps.
    Last edited by Irithori; 09-19-2012 at 08:41 AM.
    You must always face the curtain with a bow.

  4. #4
    Just Joined!
    Join Date
    Sep 2012
    Posts
    2
    Ok, I forgtot that cron should list directory first. But does Lsync or Unison can copy just files which have changed 10 minutes ago?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •