Find the answer to your Linux question:
Results 1 to 9 of 9
I need a script that will run as a daily cron job which will: 1. find & zip up the previous day's log file, 2. & if the file is ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Dec 2012
    Posts
    5

    Need Script to zip & delete log file daily & delete zip files <365days


    I need a script that will run as a daily cron job which will:

    1. find & zip up the previous day's log file,
    2. & if the file is successfully zipped then delete that file which was just zipped (the previous day's log file)
    3. & also find & delete any zip files older than 365 days

    AND the script will send an email notification to zachs-at-domain.com that each of the tasks has completed or failed.

    zip is located in /usr/bin/zip
    mail is located in /bin/mail

    The DETAILS ARE:

    The script has to search the folder /usr/iplanet/servers/https-company/logs and identify the previous day's access & error log files which are named as:

    access.yyyymmdd0000 & errors.yyyymmdd0000

    & compress them both into separate zip files named

    accessyyyymmdd.zip & errorrsyyyymmdd.zip (or any other simple naming)

    If the zip function completes successfully, then the original access or error log file should be deleted.

    If it does NOT zip properly then the access or error log should NOT be deleted.

    Lastly it has to search the same folder & delete zip files older than 365 days.

    Can you help? In advance, thanks!

  2. #2
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,439
    Hi and welcome

    Have a look at logrotate
    Code:
    man logrotate
    You must always face the curtain with a bow.

  3. #3
    Just Joined!
    Join Date
    Dec 2012
    Posts
    5
    thanks. we're running the logrotate in iplanet... this has to be outside of that...
    I've got this script put together but i think it needs some help in finding the "log" file...?
    because the access & error logs don't have the file extension noted in the script '*.log'

    Code:
    #!/bin/bash
    cd /usr/iplanet/servers/https-company/logs
    # keep today's log files as text
    find . -name '*.log' -mtime +1 -type f |
    while read fname
    do
       /usr/bin/zip ${fname}.zip $fname
       if [ ? -ne o ] ; then
          echo "fatal error compressing log $fname" | /bin/mail -s 'Log Error' zachs@domain.com
          exit 1
       fi
       rm $fname || echo "fatal error $fname" | /bin/mail -s 'Log Error' zachs@domain.com && exit 1
    done
    # delete old zip files after one year
    find . -type f -name '*.zip' -mtime +365 | 
    while read fname 
    do
      rm $fname || echo "fatal error $fname" | /bin/mail -s 'Log Error' zachs@domain.com && exit 1
    done
    # good exit
    echo "log maintenance completed" | /bin/mail -s 'Logs ok' zachs@domain.com
    exit 0
    Last edited by atreyu; 12-12-2012 at 05:45 AM. Reason: added code tags for readability

  4. $spacer_open
    $spacer_close
  5. #4
    Trusted Penguin Irithori's Avatar
    Join Date
    May 2009
    Location
    Munich
    Posts
    3,439
    Reinventing the wheel with a shell script is possibly, but why?
    logrotate is designed for this kind of tasks.

    Even more if you already use it.
    Two tools for the same task is just causing confusion.
    So my recommendation would be to extend the logrotate config that you have.
    You must always face the curtain with a bow.

  6. #5
    Just Joined!
    Join Date
    Dec 2012
    Posts
    5
    When I do a ls -l in that folder there are no file extensions on the log files.... I need to fix the script to find these files to compress & delete

    -rw-r--r-- 1 deals deals 1046826 Dec 11 08:59 access
    -rw-r--r-- 1 deals deals 92941280 Dec 7 00:00 access.201212070000
    -rw-r--r-- 1 deals deals 3044935 Dec 8 00:00 access.201212080000
    -rw-r--r-- 1 deals deals 2711784 Dec 8 23:59 access.201212090000
    -rw-r--r-- 1 deals deals 2653255 Dec 9 23:59 access.201212100000
    -rw-r--r-- 1 deals deals 2688456 Dec 11 00:00 access.201212110000
    drwxr-xr-x 2 root root 4096 Jul 24 2007 archive
    -rw-r--r-- 1 deals deals 24136805 Dec 11 08:59 errors
    -rw-r--r-- 1 deals deals 147018158 Dec 7 00:00 errors.201212070000
    -rw-r--r-- 1 deals deals 88812144 Dec 7 23:59 errors.201212080000
    -rw-r--r-- 1 deals deals 50790062 Dec 8 23:58 errors.201212090000
    -rw-r--r-- 1 deals deals 47430403 Dec 9 23:59 errors.201212100000
    -rw-r--r-- 1 deals deals 47537385 Dec 11 00:00 errors.201212110000

  7. #6
    Just Joined!
    Join Date
    Dec 2012
    Posts
    5
    I don't want to tell my boss otherwise (as your suggestion may work too), I was asked to get this to work so I'm doing it his way... because i don't want to hit the highway!

  8. #7
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    Your find command is looking for files ending in ".log". You are already in the right dir, so why not just modify your find command to find all files in that dir and let your mtime do the work, e.g.:

    Code:
    find . -mindepth 1 -maxdepth 1 -mtime +1 -type f |
    or if you want to be more specific about the filename:
    Code:
    find . -mindepth 1 -maxdepth 1 -mtime +1 -type f -name 'access*' -o -name 'error*' |
    btw, the mindepth/maxdepth just prevents recursive searches.

  9. #8
    Just Joined!
    Join Date
    Dec 2012
    Posts
    5
    atreyu - that definately helped! it zipped up a file & deleted the unzipped version but it failed after the first file.

    This is the directory
    [auspensky@dealwf1-ocp zach]$ ls -l
    total 564996
    -rw-r--r-- 1 auspensky auspensky 92941280 Dec 7 00:00 access.201212070000
    -rw-r--r-- 1 auspensky auspensky 3044935 Dec 8 00:00 access.201212080000
    -rw-r--r-- 1 auspensky auspensky 2711784 Dec 8 23:59 access.201212090000
    -rw-r--r-- 1 auspensky auspensky 2653255 Dec 9 23:59 access.201212100000
    -rw-r--r-- 1 auspensky auspensky 2688456 Dec 11 00:00 access.201212110000
    -rw-r--r-- 1 auspensky auspensky 2969669 Dec 12 00:00 access.201212120000
    -rw-r--r-- 1 auspensky auspensky 147018158 Dec 7 00:00 errors.201212070000
    -rw-r--r-- 1 auspensky auspensky 88812144 Dec 7 23:59 errors.201212080000
    -rw-r--r-- 1 auspensky auspensky 50790062 Dec 8 23:58 errors.201212090000
    -rw-r--r-- 1 auspensky auspensky 47430403 Dec 9 23:59 errors.201212100000
    -rw-r--r-- 1 auspensky auspensky 47537385 Dec 11 00:00 errors.201212110000
    -rw-r--r-- 1 auspensky auspensky 89305772 Dec 11 23:59 errors.201212120000
    -rw-r--r-- 1 auspensky auspensky 6 Dec 11 08:30 pid

    & I ran the first portion of the script with this result


    [auspensky@dealwf1-ocp zach]$ find . -mindepth 1 -maxdepth 1 -mtime +1 -type f -name 'access*' -o -name 'error*' |
    > while read fname
    > do
    > /usr/bin/zip ${fname}.zip $fname
    > if [ ? -ne o ] ; then
    > echo "fatal error compressing log $fname" | /bin/mail -s 'Log Error' zachs@domain.com
    > exit 1
    > fi
    > rm $fname || echo "fatal error $fname" | /bin/mail -s 'Log Error' zachs@domain.com && exit 1
    > done
    adding: access.201212070000 (deflated 94%)
    -bash: [: ?: integer expression expected

    So you can see in the directory listing after the script is run the access.201212070000 was zipped & deleted but it failed with the integer error?

    -bash: [: ?: integer expression expected

    [auspensky@dealwf1-ocp zach]$ ls -l
    total 479496
    -rw-rw-r-- 1 auspensky auspensky 5473963 Dec 13 12:55 access.201212070000.zip
    -rw-r--r-- 1 auspensky auspensky 3044935 Dec 8 00:00 access.201212080000
    -rw-r--r-- 1 auspensky auspensky 2711784 Dec 8 23:59 access.201212090000
    -rw-r--r-- 1 auspensky auspensky 2653255 Dec 9 23:59 access.201212100000
    -rw-r--r-- 1 auspensky auspensky 2688456 Dec 11 00:00 access.201212110000
    -rw-r--r-- 1 auspensky auspensky 2969669 Dec 12 00:00 access.201212120000
    -rw-rw-r-- 1 auspensky auspensky 0 Dec 13 12:56 do
    -rw-rw-r-- 1 auspensky auspensky 0 Dec 13 12:56 echo
    -rw-r--r-- 1 auspensky auspensky 147018158 Dec 7 00:00 errors.201212070000
    -rw-r--r-- 1 auspensky auspensky 88812144 Dec 7 23:59 errors.201212080000
    -rw-r--r-- 1 auspensky auspensky 50790062 Dec 8 23:58 errors.201212090000
    -rw-r--r-- 1 auspensky auspensky 47430403 Dec 9 23:59 errors.201212100000
    -rw-r--r-- 1 auspensky auspensky 47537385 Dec 11 00:00 errors.201212110000
    -rw-r--r-- 1 auspensky auspensky 89305772 Dec 11 23:59 errors.201212120000
    -rw-rw-r-- 1 auspensky auspensky 0 Dec 13 12:56 exit
    -rw-rw-r-- 1 auspensky auspensky 0 Dec 13 12:56 fi
    -rw-r--r-- 1 auspensky auspensky 6 Dec 11 08:30 pid
    -rw-rw-r-- 1 auspensky auspensky 0 Dec 13 12:56 rm
    -rw-rw-r-- 1 auspensky auspensky 0 Dec 13 12:56 while
    [auspensky@dealwf1-ocp zach]$

  10. #9
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    heh heh..looks like your script exploded into bits and pieces into your log dir. why not clean that up first, e.g.:
    Code:
    for file in do echo exit fi pid rm while; do [ -e $file -a ! -s $file ] && rm -i $file ;done
    that will remove (after prompting you) any of those files in the current working dir that exist, and are empty (0 bytes).

    Two things regarding your execution of the script:

    1) are you trying to run that whole script on the command line? I would try putting it in a script first.

    2) you have some Bash syntax errors. what leaps out are a missing $ (in $?) and a "o" (lower case oh) instead of a 0 (zero). see your code below, with mods in red:

    Code:
    #!/bin/bash
    cd /usr/iplanet/servers/https-company/logs
    # keep today's log files as text
    find . -name '*.log' -mtime +1 -type f |
    while read fname
    do
       /usr/bin/zip ${fname}.zip $fname
       if [ $? -ne 0 ] ; then
          echo "fatal error compressing log $fname" | /bin/mail -s 'Log Error' zachs@domain.com
          exit 1
       fi
       rm $fname || echo "fatal error $fname" | /bin/mail -s 'Log Error' zachs@domain.com && exit 1
    done
    # delete old zip files after one year
    find . -type f -name '*.zip' -mtime +365 | 
    while read fname 
    do
      rm $fname || echo "fatal error $fname" | /bin/mail -s 'Log Error' zachs@domain.com && exit 1
    done
    # good exit
    echo "log maintenance completed" | /bin/mail -s 'Logs ok' zachs@domain.com
    exit 0

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •