Find the answer to your Linux question:
Results 1 to 3 of 3
hey guys! new to this forum. can anyone help me find a good opensource webcrawler. we are trying to inventory our site and its huge. and manual check is not ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Nov 2008
    Posts
    2

    Opensource crawlers


    hey guys!
    new to this forum.

    can anyone help me find a good opensource webcrawler.
    we are trying to inventory our site and its huge. and manual check is not an option.


    thanks a bunch

    Shahin

  2. #2
    Linux Guru
    Join Date
    Nov 2007
    Posts
    1,752
    Google "crawl website wget" => The wget command can be used for this.

  3. #3
    Just Joined!
    Join Date
    Nov 2008
    Posts
    2
    wow thanks!.. saved me alot of work

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •