Find the answer to your Linux question:
Results 1 to 3 of 3
Hello, there is an Http attack on my server: /var/log/apache2/other_vhosts_access.log ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:33 +0000] "GET / HTTP/1.0" 200 668 "8574at.info" "Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.0sz9o7t8r25.com/)" ns1.site.com:80 212.185.56.58 - - ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Linux User postcd's Avatar
    Join Date
    Apr 2011
    Posts
    327

    Script reading log file and if several lines match, trigger action


    Hello,

    there is an Http attack on my server:

    /var/log/apache2/other_vhosts_access.log
    ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:33 +0000] "GET / HTTP/1.0" 200 668 "8574at.info" "Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.0sz9o7t8r25.com/)"
    ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:39 +0000] "GET / HTTP/1.0" 200 666 "wgcki.net" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv.xxx) Gecko/20041027 Mnenhy/0.6.0.104"
    ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:30 +0000] "GET / HTTP/1.0" 200 669 "61lqveh.info" "Mozilla/5.0 (compatible; http://www.3d3128.com/bot/ )"

    One IP loading the page /
    what i want to do is to create script, which will like every 5 second check above log file and when in last 100 lines are following match in one line line

    ns1.customersite.com:80 SOMEIP
    and
    GET SOMEPATH HTTP/1.0"

    in at least 20 lines, IP (SOMEIP) will be 600 seconds blocked by IPtables. Is it good idea? How would you improve it?

    What i dont understnad is the part how to create that extracting script. Im really newbie in regexes and matching. please can anyone help?

  2. #2
    Linux Guru
    Join Date
    Nov 2007
    Posts
    1,756
    You may want to check out the Fail2Ban project.

  3. #3
    Linux User postcd's Avatar
    Join Date
    Apr 2011
    Posts
    327
    Quote Originally Posted by HROAdmin26 View Post
    You may want to check out the Fail2Ban project.
    Thank you, it appears to be ideal solution. I just need to come with some "regex" to match above mentioned requests and input it into fail2ban templates foldlers (action.d , filter.d)
    But to the core of the question:

    Im really newbie in regexes and matching. please can anyone help?
    The examples of fail2ban filter regexes:

    failregex = [[]client <HOST>[]] File does not exist: .*/~.*

    failregex = [[]client <HOST>[]] (File does not exist|script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)
    [[]client <HOST>[]] script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat *$


    This is example line from apache2 logfile:
    [Thu Dec 11 00:32:03 2008] [error] [client XXX.XXX.XXX.XXX] File does not exist: /home/www/moodle/favicon.ico, referer: https://site/message/discussion.php?id=1


    In my case i need to apply regex on?

    GET / HTTP/1.0

    my example bad log line:
    ns1.site.com:80 212.185.56.58 - - [16/Nov/2013:21:32:33 +0000] "GET / HTTP/1.0" 200 668 "8574at.info" "Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.0sz9o7t8r25.com/)"

    this is the regex to match whole apache log line (from IP to the line end): http://RegExr.com?2vqh8
    (\S+)\s(\S+)\s(\S+)\s\[([^\]]+)\]\s"(\S+)\s([^\s]+)\s([^"]+)"\s(\S+)\s(\S+)\s"([^"]*)"\s"([^"]+)"

    This match log line with GET / HTTP/1.0:
    (\S+)\s(\S+)\s(\S+)\s\[([^\]]+)\]\s"GET / HTTP/1.0"\s(\S+)\s(\S+)\s"([^"]*)"\s"([^"]+)"

    I dont know how quickly google bot can load pages, so i dont block it? and how to exclude google bot? (exclude by entering "Ignoreregex:" into fail2ban filter file)

    UPDATE: i think i did it, here is part of my ./jail.local
    [apache-toomanyrequests]

    enabled = true
    filter = apache-toomanyrequests
    action = iptables-multiport[name=ApacheTooManyRequests, port="http,https"]
    sendmail-buffered[name=ApacheTooManyRequests, lines=5, dest=mymail@gmail.com]
    logpath = /var/log/httpd/access_log
    findtime = 60
    bantime = 240
    maxretry = 50
    and part of my filter ./filter.d/apache-toomanyrequests.conf file
    failregex = [[]client <HOST>[]] File does not exist: .*/~.*
    (<HOST>)\s(\S+)\s(\S+)\s\[([^\]]+)\]\s"GET / HTTP/1.0"\s(\S+)\s(\S+)\s"([^"]*)"\s"([^"]+)"
    # any apache line (<HOST>)\s(\S+)\s(\S+)\s\[([^\]]+)\]\s"(\S+)\s([^\s]+)\s([^"]+)"\s(\S+)\s(\S+)\s"([^"]*)"\s"([^"]+)"
    the test worked: fail2ban-regex .mytestlogfile apache-toomanyrequests.conf
    Last edited by postcd; 11-18-2013 at 11:39 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •