Find the answer to your Linux question:
Results 1 to 5 of 5
Hello, I am attempting to write a shell script that accesses a web page with a specified log-in and parses said page for download links to installation packages. Can anyone ...
  1. #1
    Just Joined!
    Join Date
    Mar 2007
    Posts
    20

    Parsing of links from authenticated HTTPS page

    Hello,

    I am attempting to write a shell script that accesses a web page with a specified log-in and parses said page for download links to installation packages. Can anyone advise on the possible (and best) ways to go about this? Perl? wget? curl? I haven't been able to find any clear-cut documentation so far concerning the aforementioned -- if anyone has some favorite tutorials on parsing web data that would also be useful.

    Thanks in advance!

  2. #2
    Just Joined!
    Join Date
    Oct 2010
    Location
    Reston, VA
    Posts
    26
    The big difference is the login. Can you login via get? Something like:

    https://example.com?user=me&password=foo

    I wouldn't think so though.

    wget won't post data, so that leaves either curl or perl. Unless you're a perl guy I'd look into curl first as its much easier to use.

    -Gooney0

  3. #3
    Just Joined!
    Join Date
    Mar 2007
    Posts
    20
    I'm not exactly sure...

    If it helps, the back-end link I'm trying to parse is the following:

  4. #4
    Just Joined!
    Join Date
    Mar 2007
    Posts
    20

  5. #5
    Just Joined!
    Join Date
    Oct 2010
    Location
    Reston, VA
    Posts
    26
    Sure enough that requires POST. Try curl or perl.

    -Gooney0

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
...