bash: parallel subprocesses and wait

[mglenney]

I have a script that is used to execute remote commands on servers and return the results. I would like those commands to be run in parallel. I have the following:

hostnum=${#hosts[*]}

for ((i=0;i<$hostnum;i++))
do
  result[$i]=`ssh -o ConnectTimeout=5 -o LogLevel=quiet ${hosts[$i]} $command` &
done
wait

for ((i=0;i<$hostnum;i++))
do
  echo -ne "${hosts[$i]}\t${result[$i]}\n"
done

If $command is set to "hostname" and I want to run it against host1 and host2 I'm getting the following output:

host1
host2

If I remove the "&" and the "wait" and run them serially I get:

host1        host1.example.com
host2        host2.example.com

If I watch my processes I can see that all the subprocesses are getting run but the results are never getting stored. Any ideas?

Thanks,

MG

[mglenney]

Here's a little more information: It looks like the results are returned but since I'm out of the loop by the time they get back they aren't being put into the array properly.

+ hostnum=4
+ (( i=0 ))
+ (( i<4 ))
+ (( i++ ))
+ (( i<4 ))
++ ssh -o ConnectTimeout=5 -o LogLevel=quiet host1 hostname
+ (( i++ ))
+ (( i<4 ))
++ ssh -o ConnectTimeout=5 -o LogLevel=quiet host2 hostname
+ (( i++ ))
+ (( i<4 ))
++ ssh -o ConnectTimeout=5 -o LogLevel=quiet host3 hostname
+ (( i++ ))
+ (( i<4 ))
+ wait
++ ssh -o ConnectTimeout=5 -o LogLevel=quiet host4 hostname
+ result[$i]=host4.example.com
+ result[$i]=host1.example.com
+ result[$i]=host3.example.com
+ result[$i]=host2.example.com
+ (( i=0 ))
+ (( i<4 ))
+ echo -ne 'host1\t\n'
host1
+ (( i++ ))
+ (( i<4 ))
+ echo -ne 'host2\t\n'
host2
+ (( i++ ))
+ (( i<4 ))
+ echo -ne 'host3\t\n'
host3
+ (( i++ ))
+ (( i<4 ))
+ echo -ne 'host4\t\n'
host4
+ (( i++ ))
+ (( i<4 ))
+ exit 0

I hope this is relevant information. It certainly didn't help me figure this problem out but maybe it will for you.

[mglenney]

I figured out a work-around with the help of a coworker. Since I couldn't find a way to populate an array with the results I just dump the results from each host into a file named for that host in a temp directory. Then, after all the results are in, I can display the results:

hostnum=${#hosts[*]}

mkdir $tmpdir

for ((i=0;i<$hostnum;i++))
do
  ssh -o ConnectTimeout=5 -o LogLevel=quiet ${hosts[$i]} $command > $tmpdir/${hosts[$i]} &
done
wait

for file in `ls $tmpdir`
do
  numlines=`cat $tmpdir/$file | wc -l`
  if [ "$numlines" -gt "1" ]; then
    echo "-------------------------------------------------------------------"
    echo -ne "$file\n\n"
    echo -ne "`cat $tmpdir/$file`\n\n"
  else
    echo -ne "$file\t"
    echo -ne "`cat $tmpdir/$file`\n"
  fi
done

rm -r $tmpdir

Once nice thing that came from this approach is that I can test to see if the results are multiple lines and adjust my output accordingly.

Just wanted to post the solution in case this comes up for anyone else.

MG