[SOLVED] Using WGET to save an Audio stream, and Adding the file to SQL Table

[w2lie]

Hi All,

I need to first state that I am new to the linux scripting world. I do not know a lot of commands, but I have big dreams lol

I currently host a website that streams audio out to the internet. The streams themselves are served on a local computer running Icecast. I would like to run WGET on a local linux box, and save the file as a unique name (preferable stamped date and time.) Once the file has completed, I would like to then upload that file to my webhost. Then, on the webhost, I would like to run a CRON that checks for the file, and than add the info to an SQL Table, so it can be downloaded by my members.

An example of the filename for a file created on May 22, 2010 at 3:00pm would be: 'name_100522.1500.mp3'

An example of the filename for a file created on May 22, 2010 at 3:30pm would be: 'name_100522.1530.mp3'

I've started by writing down what I would like to do on paper, but I just don't know how to turn my notes into BASH files... Any help would be great!



BASH File 1 (local machine running SuSE)

-> wget the audio feed and save the file as 'name_%y%m%d.%H (and for the minutes I would like 00 for the top of the hour, and 30 for the bottom of the hour - you'll see)

-> Find the PID of the wget running, and kill it every 30 minutes

-> Repeat file 1 (I am assuming schedule the CRON to run this bash at 00 and 30)

=========

BASH File 2 (local machine running SuSE)
File should be started at a time after the file recorded in step 1 completed.. I was thinking of kicking CRON to start at 05 and 35 minutes since step 1 should kill the processes at 00 and 30 minutes.

-> check to see that the file recorded in step 1 exists (File name will be stamped Current time-35 minutes) AND see if the file is over 1 Meg

-> IF the file meet the above conditions, upload the file to my remote webhost via FTP.

-> Delete any local files in the working directory that are older than xx days (using file name's time/date stamp)

=========

BASH File 3 (Remote Machine / LINUX Webhost)
File should be started at a time after the file recorded in step 1 completed, and step 2 has uploaded the file to the webhost.. I was thinking of kicking CRON to start at 00 and 30 minutes past each hour, to update the website with the file that was recorded via wget in step 1)

-> if the file name ''name_%y%m%d.%H(and 00 or 30 for minutes)" exists, insert an entry in the SQL table so the file can be downloaded.

Now I don't know if this will work if you subtract an hour, will it be smart enough to subtract the day/month/or year when the calendar changes as well?? If not, I guess we'll have to add IF / ELSE's for the conditions?

-> See if files exist on the server that are older than 8 days (down to the minute, so I'm assuming I would be looking for files older than 192 hours (based on the file name stamp)).

-> Delete those files off the server and delete the row from the SQL table, so that I don't have broken downloads.

====================


So, as you can see, I know what I want to do, I just have no idea on how to really code this into BASH.

I'm sure this could probably all be kicked with one file, but I would like to keep the remote side of the stuff running on the remote host and local stuff running on local for personal reasons (security, and to make sure that files are deleted off the webhost even if the local machine can't talk to it)


Thanks for ANY help that might be out there. This is mostly 5 miles over my head, but I want to learn a bit here as well.. I am as green as it gets - but hopefully one day I can help someone else who is in my boat.

73
Phil / w2lie

[Cabhan]

I'm a bit confused on the format of the file that you will be wgetting.

Is this file essentially an infinitely-long file that you intend on reading for a certain period of time? If so, you may have issues: if the file is an MP3, for instance, does the MP3 format require any sort of tail data? If so, by simply killing wget, you will end up with an unusable file, because the "end" of the file (even if it doesn't actually exist on the server) will never have been written.

Actually, on that note, what protocol are you using to stream? It's presumably not HTTP, but wget only supports HTTP, HTTPS, and FTP.

You will probably want to find some other utility than wget to download your stream. I'm sure there must be some sort of commandline MMS utility somewhere that might even have time-based controls in it.

As for some of your other problems:

If you want to generate a filename based on the current date, it's relatively simple:

filename=name_$(date +%Y%m%d.%H%M).mp3

This would use the formatting options of the date command to produce the format that you want (see "man date" for more details).

Finding files that are older than a certain date is very easy by using the "find" command.

The only other tricky bit is your code to get an hour previous, and how to correctly change the day/month/year if necessary. Fortunately, date is pretty good at this stuff:

date -d '-1 hour'

will give you the date from one hour ago.

[w2lie]

Thank you for the reply.

Yes, it will be an mp3 stream. I could probably just set a second stream on the server that is set to time-out after 30 minutes. This way, the feed closes on, closing the mp3 file.

I will try and do some googles on the find command. But, based on your reply, this sounds like something that will work, right?

Thanks again
Phil / w2lie

[w2lie]

Thanks for the help.. Unfortunately I've discovered a limitation with my shared server, so I need to rethink my plans.. But - to run 1 CRON at a time, everything works..

I'm opening up a new thread for the new direction.

Thanks again