Webcrawler such as teleport pro or superbot for linux?
Does anyone know about a Web Crawler for Linux such as the counterparts for windows (Webwhacker, websnake, teleport pro, black widow and superbot)
I need one, tried these in wine and they crash alot.
Not sure what a webcrawler is?
They're a program that can do something like
# Download all or part of a website to your computer, enabling you to browse the site directly from your hard disk at much greater speeds than if you were to browse the site online
# Create an exact duplicate, or mirror of a website, complete with subdirectory structure and all required files
# Search a website for files of a certain type and size
# Download a list of files at known addresses
# Explore every website linked from a central website
# Search a website for keywords
# Make a list of all pages and files on a website
thanks alot for any replys