Results 1 to 5 of 5
Hello,
I am attempting to write a shell script that accesses a web page with a specified log-in and parses said page for download links to installation packages. Can anyone ...
- 10-22-2010 #1Just Joined!
- Join Date
- Mar 2007
- Posts
- 20
Parsing of links from authenticated HTTPS page
Hello,
I am attempting to write a shell script that accesses a web page with a specified log-in and parses said page for download links to installation packages. Can anyone advise on the possible (and best) ways to go about this? Perl? wget? curl? I haven't been able to find any clear-cut documentation so far concerning the aforementioned -- if anyone has some favorite tutorials on parsing web data that would also be useful.
Thanks in advance!
- 10-23-2010 #2Just Joined!
- Join Date
- Oct 2010
- Location
- Reston, VA
- Posts
- 26
The big difference is the login. Can you login via get? Something like:
https://example.com?user=me&password=foo
I wouldn't think so though.
wget won't post data, so that leaves either curl or perl. Unless you're a perl guy I'd look into curl first as its much easier to use.
-Gooney0
- 10-23-2010 #3Just Joined!
- Join Date
- Mar 2007
- Posts
- 20
I'm not exactly sure...
If it helps, the back-end link I'm trying to parse is the following:
- 10-23-2010 #4Just Joined!
- Join Date
- Mar 2007
- Posts
- 20
https://dist.r1soft.com/s/?action=Do...leRelease=true
The actual log-in page is
https://dist.r1soft.com/download/
- 10-23-2010 #5Just Joined!
- Join Date
- Oct 2010
- Location
- Reston, VA
- Posts
- 26
Sure enough that requires POST. Try curl or perl.
-Gooney0


Reply With Quote