Find the answer to your Linux question:
Results 1 to 5 of 5
Im having an issue with my nightly cronjobs that use rsync to mirror a large amount of data. Here is the command: /usr/bin/rsync -av -e ssh --delete --exclude "*Locks" --exclude ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Mar 2008
    Posts
    47

    Massive cronjob mails when using rsync


    Im having an issue with my nightly cronjobs that use rsync to mirror a large amount of data. Here is the command:

    /usr/bin/rsync -av -e ssh --delete --exclude "*Locks" --exclude "CCC" --bwlimit=0 server1:/folder1/* /folder1/

    Im getting cronjob mails with quite large sizes. Basically its listing every file, which could be in the region of 60Gb.

    However if omit the "v" it doesnt report on the wrote/read/etc at the end of the task.

    Is there a way to strip the filelist while processing? I really dont need to see a couple of hundred thousand files listed. All i want is the end line. Any failures would be good too.

  2. #2
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,348
    I looked briefly through the man page and there seems to be no way to have just the report.

    One can influence the --out-format, but not disable it.
    The following will output a empty line for each file, which then getīs deleted by sed.

    Not exactly great, I know

    Code:
    /usr/bin/rsync -av --out-format="" -e ssh --delete --exclude "*Locks" --exclude "CCC" --bwlimit=0 server1:/folder1/* /folder1/ | sed '/^$/d'
    You must always face the curtain with a bow.

  3. #3
    Just Joined!
    Join Date
    Mar 2008
    Posts
    47
    That works. TBH i have never seen a failure when using rsyn so far (over 3 yrs) so i guess the blanks wont really matter.

    However the script for the cronjob uses "spawn" before the command, and somehow i cant put | sed after that:

  4. #4
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,348
    You should still see all errors.

    a) sed works on stdout of rsync, not stderr
    b) The sed *only* deletes empty lines, and more specifically empty lines with no char at all: Thatīs what the ^$ matches.
    You must always face the curtain with a bow.

  5. #5
    Just Joined!
    Join Date
    Mar 2008
    Posts
    47
    Thanks very much for the help.

    If i can ask you one last question it would be, what is the correct syntax for putting it in an expect/spawn script?


    #!/usr/bin/expect

    spawn /usr/bin/rsync -av --out-format= -e ssh --delete --bwlimit=0 blitzwing:/folder1/* /folder2/ | /bin/sed '/^$/d'

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •