Running More than one script in a bash script and piping results to SSH

[w2lie]

Ok.. Obviously I don't know how to set the subject of this up.. but here is what I am trying to accomplish (please keep in mine, this is only my first month playing with ANY Linux programming):

Background info:
My shared web host limits running 2 CRONS or 2 SSH sessions at one time. I need to run more than that.. So, my solution is to run what I need on my home computer, and then push all the results via SSH to my web server. To keep things timed, I am trying to call 4 bash scripts from inside of 1 bash script... Each bash script has variables I need to export out to the remote (web) server. Being that I can only run 2 SSH or 2 CRON sessions on the remote, it wouldn't do me any good to open up CRON or SSH remotely or locally - either way I'm maxing out. That is why I would like to call 1 final script that takes the output of the 4 bash scripts and does the job.


So - to make this clear as mud

Main bash calls via CRON every 30 minutes:

./script_1 &
./script_2 &
./script_3 &
./script_4 &
#wait for scripts to finish
sleep 31 m

#upload file generated by 'script_1'
#upload file generated by 'script_2'
#upload file generated by 'script_3'
#upload file generated by 'script_4'

#add info into the remote database using SSH
i_don't_know

In EACH script (ex script_1), there is a 'wget' command to capture MP3 audio from a live stream stored as variables that need to be sent to the remote server about the 'wget' capture. The most important includes a SQL Insert command. (saved as $sql_cmd)


So - I guess the goal is:

I need to scp the file saved by wget to the remote server. I also need to pass the SQL statement generated in each script as a command in SSH. I'm lost how to get the info from "script_x" into a string that can be used to SSH - and doing this all inside of ONE SSH command. Would I store the SQL strings in a file and call that in the SSH command line? If so, what is the command to make sure the variable output in the "script_#" file is sent to a file? Can I call the variable from the main Bash Script?

Now - the good news is, I can SSH from my local machine to the remote one.. That is about as far as I got.

Again - I am so new to this that my ears are still wet.. This has been something I have been working on for a while, and I'm just lost at this point..


Thanks again..
73
Phil / w2lie

[Kloschüssel]

you can store the capture output to variables and just reuse them, without storing stuff to the filesystem (obviously this works only if the capture is quite small to not exceed memory limits).

alright. I may give you some clue of commands that may be helpful for you, but you'll have to figure it out on yourself as I have not the resources to do your job.

take a look at:
* basic output redirection (&1>, &2>)
* piping (|)
* tee (which duplicates a stream to multiple other streams)
* mysql (which is a mysql shell client)
* ssh (login to remote computer)
* rsa keys (so your script won't be prompted for passwords)

your script should work just like if you were sitting in front of the computer and typing the commands in. so for example if you have a ssh in your script, it will log you into the remote computer and therefore you MUST have a exit later on in order to close the remote session properly. all commands in between will be executed remotely (well, to be consistent, exit is also being executed remotely).

happy bashing!

[sixdrift]

Any shell script running under a process is essentially operating as a user typing commands. So from that perspective, you are on the right track.

If it were me, I would run my four local scripts as you have done, wait for some synchronizer to know they completed, and then either: A) pack all data into a tarball and "scp" that to the server", or B) "scp" the files one at a time to the server. Then once I have all the files on the server, I would have my local script ssh into the server and run a single script.

Now that single script could either be written and stored ahead of time, or you could generate the script on the fly and send it up with the other data files. In either case, each scp is a single SSH that occurs one at a time. Then when you SSH into the box, again you have a single SSH.

Now the real question. Why not have a single cron job on the server run a script that does all 4 things at once? Essentially you could specify a cron job at the lowest time increment and then the script checks for other things. Much like turning a single timer into a basic multi-use timer.