Find the answer to your Linux question:
Results 1 to 3 of 3
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1

    Script reading log file and if several lines match, trigger action


    Hello,

    there is an Http attack on my server:

    /var/log/apache2/other_vhosts_access.log
    ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:33 +0000] "GET / HTTP/1.0" 200 668 "8574at.info" "Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.0sz9o7t8r25.com/)"
    ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:39 +0000] "GET / HTTP/1.0" 200 666 "wgcki.net" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv.xxx) Gecko/20041027 Mnenhy/0.6.0.104"
    ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:30 +0000] "GET / HTTP/1.0" 200 669 "61lqveh.info" "Mozilla/5.0 (compatible; http://www.3d3128.com/bot/ )"

    One IP loading the page /
    what i want to do is to create script, which will like every 5 second check above log file and when in last 100 lines are following match in one line line

    ns1.customersite.com:80 SOMEIP
    and
    GET SOMEPATH HTTP/1.0"

    in at least 20 lines, IP (SOMEIP) will be 600 seconds blocked by IPtables. Is it good idea? How would you improve it?

    What i dont understnad is the part how to create that extracting script. Im really newbie in regexes and matching. please can anyone help?

  2. #2
    You may want to check out the Fail2Ban project.

  3. #3
    Quote Originally Posted by HROAdmin26 View Post
    You may want to check out the Fail2Ban project.
    Thank you, it appears to be ideal solution. I just need to come with some "regex" to match above mentioned requests and input it into fail2ban templates foldlers (action.d , filter.d)
    But to the core of the question:

    Im really newbie in regexes and matching. please can anyone help?
    The examples of fail2ban filter regexes:

    failregex = [[]client <HOST>[]] File does not exist: .*/~.*

    failregex = [[]client <HOST>[]] (File does not exist|script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)
    [[]client <HOST>[]] script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat *$


    This is example line from apache2 logfile:
    [Thu Dec 11 00:32:03 2008] [error] [client XXX.XXX.XXX.XXX] File does not exist: /home/www/moodle/favicon.ico, referer: https://site/message/discussion.php?id=1


    In my case i need to apply regex on?

    GET / HTTP/1.0

    my example bad log line:
    ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:33 +0000] "GET / HTTP/1.0" 200 668 "8574at.info" "Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.0sz9o7t8r25.com/)"

    this is the regex to match whole apache log line (from IP to the line end): http://RegExr.com?2vqh8
    (\S+)\s(\S+)\s(\S+)\s\[([^\]]+)\]\s"(\S+)\s([^\s]+)\s([^"]+)"\s(\S+)\s(\S+)\s"([^"]*)"\s"([^"]+)"

    This match log line with GET / HTTP/1.0:
    (\S+)\s(\S+)\s(\S+)\s\[([^\]]+)\]\s"GET / HTTP/1.0"\s(\S+)\s(\S+)\s"([^"]*)"\s"([^"]+)"

    I dont know how quickly google bot can load pages, so i dont block it? and how to exclude google bot? (exclude by entering "Ignoreregex:" into fail2ban filter file)

    UPDATE: i think i did it, here is part of my ./jail.local
    [apache-toomanyrequests]

    enabled = true
    filter = apache-toomanyrequests
    action = iptables-multiport[name=ApacheTooManyRequests, port="http,https"]
    sendmail-buffered[name=ApacheTooManyRequests, lines=5, dest=mymail@gmail.com]
    logpath = /var/log/httpd/access_log
    findtime = 60
    bantime = 240
    maxretry = 50
    and part of my filter ./filter.d/apache-toomanyrequests.conf file
    failregex = [[]client <HOST>[]] File does not exist: .*/~.*
    (<HOST>)\s(\S+)\s(\S+)\s\[([^\]]+)\]\s"GET / HTTP/1.0"\s(\S+)\s(\S+)\s"([^"]*)"\s"([^"]+)"
    # any apache line (<HOST>)\s(\S+)\s(\S+)\s\[([^\]]+)\]\s"(\S+)\s([^\s]+)\s([^"]+)"\s(\S+)\s(\S+)\s"([^"]*)"\s"([^"]+)"
    the test worked: fail2ban-regex .mytestlogfile apache-toomanyrequests.conf
    Last edited by postcd; 11-18-2013 at 12:39 PM.

  4. $spacer_open
    $spacer_close

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •