Find the answer to your Linux question:
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 17
A reminder that using automated software to produce a local copy of this website for offline browsing (or any other reason) is strictly forbidden. Running any automated software against this ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Linux Guru
    Join Date
    Apr 2003
    Location
    London, UK
    Posts
    3,284

    "Downloading" LinuxForums for "Offline Browsing"


    A reminder that using automated software to produce a local copy of this website for offline browsing (or any other reason) is strictly forbidden.

    Running any automated software against this site specifically, or my servers in general is viewed as an attempted denial service attack, and will be reported to your ISP.

    Thanks for you're co-operation.

  2. #2
    Linux Newbie
    Join Date
    May 2005
    Posts
    108
    That's nice, and what, pray tell, is the ISP obligated to do? What crime has been committed?

  3. #3
    Linux Guru
    Join Date
    Apr 2003
    Location
    London, UK
    Posts
    3,284
    Quote Originally Posted by eatinglemur
    That's nice, and what, pray tell, is the ISP obligated to do? What crime has been committed?
    It is an automated DOS against my server.
    One individual yesterday sent 20,000 HTTP requests for PHP pages in 1 hour, seriously degrading the performance of this site. Such floods are no different to using a SYN or ICMP flood or any other form of malicious denial of service attack that ties up the resources of my server preventing legitimate traffic getting to the site. Many ISP's state in their terms of service that such activity is not allowed.

  4. #4
    Linux Newbie
    Join Date
    May 2005
    Posts
    108
    Quote Originally Posted by jasonlambert
    One individual yesterday sent 20,000 HTTP requests for PHP pages in 1 hour.
    How rude!

  5. #5
    Linux Engineer cheetahman's Avatar
    Join Date
    Dec 2004
    Location
    New York
    Posts
    1,130
    Quote Originally Posted by eatinglemur
    Quote Originally Posted by jasonlambert
    One individual yesterday sent 20,000 HTTP requests for PHP pages in 1 hour.
    How rude!
    Very True
    \"TTFN Taa Taa For Now\" by Tigger in Winnie the Pooh
    http://www.distrowatch.com Linux Distros
    We Live in a Windows World but there is Linux to save the day

  6. #6
    Linux Enthusiast
    Join Date
    Jan 2005
    Posts
    575

    Re: "Downloading" LinuxForums for "Offline Browsing"

    Quote Originally Posted by jasonlambert
    A reminder that using automated software to produce a local copy of this website for offline browsing (or any other reason) is strictly forbidden.
    Out of curiosity isn't that what Google does ?

  7. #7
    Linux Guru kkubasik's Avatar
    Join Date
    Mar 2004
    Location
    Lat: 39:03:51N Lon: 77:14:37W
    Posts
    2,396
    Not exactly, google uses a 'crawler' that follows links and behaves much more like a human (ie viewing 1 page at at time) automated downloaders tend to open several simultanious connections to a server at the same time and download as much content as possible. In addition search engines respect the file robots.txt which may contain a list of files to be indexed, and files not to be indexed. (linuxforums.org has one, here should you want to see one) its a fine line, but in the end, the only *serious* difference that I know is google does this in moderation, and behaves more like a fast reader would as opposed to a DOS style script which has no reguards for moderation.
    Avoid the Gates of Hell. Use Linux
    A Penny for your Thoughts

    Formerly Known as qub333

  8. #8
    Linux Guru lakerdonald's Avatar
    Join Date
    Jun 2004
    Location
    St. Petersburg, FL
    Posts
    5,035
    It also has to do with the frequency of the indexing. Somebody using one the aforementioned programs most likely will index a site more times than something like a google crawler would (and with the billions of pages indexed by google, I think the explanation is self-explanatory) causing bandwidth to be adversely affected more, and more often.

  9. #9
    oz
    oz is offline
    forum.guy
    Join Date
    May 2004
    Location
    arch linux
    Posts
    18,733
    Quote Originally Posted by jasonlambert
    It is an automated DOS against my server. One individual yesterday sent 20,000 HTTP requests for PHP pages in 1 hour
    Hmm... maybe that's what is causing the hiccup I talked about in this thread:

    http://www.linuxforums.org/forum/topic-45213.html

  10. #10
    Linux Guru
    Join Date
    Apr 2003
    Location
    London, UK
    Posts
    3,284
    It's certainly possible.

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •