wget+cron = multiple downloads

[stokilo]

Hi

I have a problem with very simple combination of wget+cron on my ubuntu server.
I created file execute.cron as normal user

execute.cron :

16 * * * * wget -t 0 www.page.com > output.log

next i run : crontab execute.cron

Task was invoked properly but i as i result is should get one index.html page. When i run this from command line that is the case. Using cron i get

index.html
index.html.1
index.html.2
and so on

What can be a reason that cron + wget run multiple downloads ?


Best regards
Slawek

[GatekeeperCruelman]

The problem why you don't get the index.html is because "wget -t 0 www.page.com > output.log" say that it want the output in outpot.log.
With a cronjob you to specify the user before you make the command line you want to execute. This can maybe work if you still get errors.
16 * * * * [user] [command]

I hope this will help you.

Greets,
GatekeeperCruelman

[stokilo]

I tried both with user and without output.log. Still i get list of files , it is a big problem for me because i need to download large zip file and wget starts download 5 or 6 times that same.

Is it possible that cron is wrong configured ? If yes please give me info what should be changed. I think this is configuration problem beacuse on my ubuntu server that is placed on other machine on vmware all cron jobs works ok.


thx for reply GatekeeperCruelman

[GatekeeperCruelman]

your welcome.
If you need only one file, give at the wget the full file path. www.page.com/list/test.zip (example)
Wget will only get the files you would normally visit with a browser (including links from on that page)

[stokilo]

Hi

About downloading archives it was not working too.

After long investigation i find out where was the problem. I was testing shell scripts under cron and i was thinking that crontab -r removes job from cron. But is not true. My script used to be wget dependent. After few restarts i have pending tasks under root account but i was working as standart user. When i was executing next script under cron then pendings gets run again and resume downloads , effect of this behaviour was in bunch of files that were created on script startup (index.html, 1 ,2 ..) . After cron restart under root account all problem were gone.

Thanks for response

Best regards
Slawek