Find the answer to your Linux question:
Results 1 to 3 of 3
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1

    Question [SOLVED] Recursive Site Downloader

    Hi, fellow members of

    I have a friend who would like me to edit his website.

    To do this, I would like to recursively download all the pages within the domain, such that their link structure is preserved.

    This would be tedious to do by hand, however.

    As it stands, I could probably use wget for this, but I would prefer something more specially designed for site downloading.

    I have already tried webHTTrack, but found it unsuitable. Perhaps httrack with a particular set of command line parameters would work better? Please advise, if you have anything that could get me started.

    Thanks for reading. Feel free to PM me, also.

  2. #2
    Linux Guru reed9's Avatar
    Join Date
    Feb 2009
    Boston, MA
    wget is pretty straightforward to do this. Why is that not an option for you?

    Downloading an Entire Web Site with wget | Linux Journal
    Create a mirror of a website with Wget | FOSSwire
    Using wget to mirror a website | townx

    I would just create an alias for the command you want to use to make it easy. Something in ~/.bashrc like

    alias grab='wget -mk'
    Then you could just type

  3. #3

    Thumbs up wget OR httrack

    Someone pm-ed me about httrack's wizard. Used that with the N0 option, and successfully got the site.

    Just read your top link about wget. That would work perfectly, and I'll probably just use that in the future bloat-free.

    Never mind the stuff below:

    The main reason is that I am going to need to maintain the directory structure, etc., and I think that wget does not do this.

    Doesn't wget just download the target's index page?
    Or does it get the whole domain or subdomain if you do not pass a full URL as its parameter?
    Last edited by Slekvic; 04-23-2011 at 06:12 AM. Reason: just read something that made my reply pointless.

  4. $spacer_open

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts