Scripts For Automatic Downloads

[cicero_dhoom]

Hi Team,

I'm new to scripts and I'm looking for a script which will automatically download my PDF's and documents from particular location, which are at a remote Server.
This need to run at my system start up.

Any Ideas on How to start this Stuff. I hope this is a combination and set of
Wget commands in scripts put into cron or Init.d. Can any one please help me on this, Any Useful Links ??

I need some help on this,
Thanks on Advance

[wje_lf]

I'm not particularly specialized in what you want, but nobody else has answered, so I'll get the ball rolling.

Yep, the script should go in init.d somewhere. That varies from distribution to distribution, so either consult your CentOS documentation or just do this at the command line:

find /etc -name '*init*'

and see what file names come up. The files might well be sufficiently self-documenting that they provide you with all you need to know about where your scipt should be called from. Just keep in mind that things like environment variables (particularly PATH) are not defined when the system is first coming up, so you'll have to specify everything explicitly. For example, the line which calls your script is going to have to specify not just your script name, but the full path, like:

/fred/barney/foo/bar/baz/script.sh

As another example, if you wish to use wget, you won't be able to just say

wget

You'll have to know where it is, and specify that. For example, suppose you do this at the command line (and I recommend that you do):

which wget

The output from that (for example, /usr/bin/wget) is how you need to specify wget in your script.

If all this is to happen just when your system comes up (and not, say, every hour or day), then you can skip the cron stuff.

Other than that, try googling this:

bash tutorial

I'm sure that others here will jump in with more specific recommended links.

Hope this helps.

[Franklin52]

A basic explanation that should work on most systems.

Log in as root, create a file in "/etc/init.d/" and save it as "ftp.sh":

#!/bin/sh

ftp /i /n <<EOF
open <hostname>
user <username> <password>
get <filename>
quit
EOF

Make the file executable:

chmod a+rx /etc/init.d/ftp.sh

In /etc/inittab you can check the default runlevel, something like: "id:2:initdefault".
In this case the default runlevel is 2, now create a symbolic link to that file in "/etc/rc2.d"

cd /etc/rc2.d
ln -s ../init.d/ftp.sh S19ftp.sh

Now the script should run when you start up your machine.


Regards

[wje_lf]

Quoth Franklin52:

In /etc/inittab you can check the default runlevel, something like: "id:2:initdefault".
In this case the default runlevel is 2, now create a symbolic link to that file in "/etc/rc2.d"

Except that I note in other posts by cicero_dhoom that he's running CentOS. According to this, he wants to place a symbolic link not in directory (for example) /etc/rc2.d, but in (for example) /etc/rc.d/rc2.d.

Further, if cicero_dhoom wishes to use ftp as in Franklin52's example, he should remember to use the full pathname of ftp, obtained by doing this at the command line:

which ftp

Other than that, Franklin52's advice will send cicero_dhoom well on the road to success.

[cicero_dhoom]

Thanks Guys,

That was very Much helpful indeed.

The tricky part is to How am I going to download the Files which are generated often.

I have a script which generates regular PDFs on Demand. I need to automatically download, only the PDfs which are the Latest ones stored with Epoh values. I'm planning to use date -d '1970-01-01 UTC <epoh value> sec ' command to convert the epoh value back. I'm using the Mysql DB to store these values on a Table.

Any Ides on these...

[wje_lf]

I'm not sure how you're going to get there from here, but if you decide to use ftp (as in Franklin52's example), maybe you can use ftp's "cd" command to get to the directory where the PDF's are stored,and then ftp's "dir" command to get the names of all the files in that directory. Then exit from ftp, slice'n'dice the output to get the names of the guys you need to download, and get into ftp again if necessary.

Does this help?

[cicero_dhoom]

Thanks Bill,

Most of the searches are with the same Ideas as yours.
The thread was very much helpful indeed.

Thanks again

[devnulled]

I'd do the following, place the latest PDFs in the www dir and echo their names in a file, and then have a while true script which fetches that file, loops thru it and wget's domain/filename while keeping track of which files I've already downloaded locally in a file used to compare the fetched data (if that makes any sense)
I can probably throw something together if you want me to.

[cicero_dhoom]

Thanks for the useful tips,

I will try these out and lets see the out come.

Thanks again