Find the answer to your Linux question:
Results 1 to 4 of 4
I am trying to get a concept where I can create simultaneous running child processes from a parent and able to read their exit status as soon as they come ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Sep 2007
    Posts
    7

    Multithreading in BASH


    I am trying to get a concept where I can create simultaneous running child processes from a parent and able to read their exit status as soon as they come in. Once the exit code is read it will start a new child to take its place. That way there is alway 10 children running at once.

    In the end I am going to be using this for a way to do asynchronous wgets to speed up my wget scripting. So if you have an alternative, let me know. Thats also why I set a randomized timer on the example child process to simulate wgets and so I can check if it doesnt do things in order or not.

    Here is the child process I am threading, just for reference:

    Code:
    number=0
    let "number = $RANDOM % 20"
    time=$(echo "1+$number*0.1" | bc)
    sleep $time
    echo "I got $1 ($time)"
    exit $1

    I first started out with this job script I found. I realized that it didnt get the exit codes fast enough. I wanted the exit code retrieved as soon as the process is exited.

    Code:
    #!/bin/bash
    
    MAXJOBS=10
    
    function jobidfromstring()
    {
            local STRING;
            local RET;
     
            STRING=$1;
            RET="$(echo $STRING | sed 's/^[^0-9]*//' | sed 's/[^0-9].*$//')"
     
            echo $RET;
    }
     
    function clearToSpawn
    {
        local JOBCOUNT="$(jobs -r | grep -c .)"
        if [ $JOBCOUNT -lt $MAXJOBS ] ; then
            echo 1;
            return 1;
        fi
     
        echo 0;
        return 0;
    }
     
    JOBLIST=""
     
    for (( i=0; i<=20; i++ ));
    do
            while [ `clearToSpawn` -ne 1 ] ; do
                    sleep 0.5
            done   
            ./child $i &
            LASTJOB=`jobidfromstring $(jobs %%)`
           JOBLIST="$JOBLIST $LASTJOB"
    done
    
    for JOB in $JOBLIST ; do
            wait %$JOB
            echo "Job $JOB exited with status $?"
    done
    echo "Done."

    Then I decided to tryout the SIGCHLD handler and see how that worked. It turned out that I wasn't able to define the handler in a way that would capture every child for some reason.

    Code:
    #!/bin/bash
    
    iNext=0
    
    function ChildReturned() {
            wait $1
            echo "$1 was returned. Its exits status was: $?"
            iNext=$((iNext+1))
            StartChild $iNext
    }
    
    
    function StartChild {
            ./child $1 &
            pid=$!
            echo "Started: $1  PID= $pid"
            trap 'ChildReturned $pid' SIGCHLD
    }
    
    
    set -bm
    
    for (( iNext=0; iNext<=10; iNext++ ));
    do
            StartChild $iNext
    done
    
    wait

    Finally I tried just starting the children and looping through a while statement that would check when each one died, retrieve the exit code, and then start a new child. However I realized that a child that terminates (SIGCHLD) still exists as a process until you get the exit code with wait(). Therefore using kill -0 wasnt going to work.

    Code:
    #!/bin/bash
    
    
    throttle=10
    iNext=0
    JobIDArray[0]=""
    
    function jobidfromstring()
    {
            local STRING;
            local RET;
    
            STRING=$1;
            RET="$(echo $STRING | sed 's/^[^0-9]*//' | sed 's/[^0-9].*$//')"
            echo $RET;
    }
    
    function AnyAlive {
            for (( i=1; i<=throttle; i++ )); do
                    if kill -0 %${JobIDArray[i]} > /dev/null 2>&1; then
                            echo 1
                            return 1
                    fi
            done
            echo 0
            return 0
    }
    
    for (( iNext=1; iNext<=throttle; iNext++ )); do
            ./child $iNext &
            JobIDArray[iNext]=`jobidfromstring $(jobs %%)`
            echo "          JobIDArray[$iNext] = ${JobIDArray[iNext]}"
    done
    
    
    while [[ `AnyAlive` -eq 1 ]] ; do
            for (( i=1; i<=throttle; i++ )); do
                    jID=${JobIDArray[i]}
                    if ! kill -0 %$jID; then
                            wait %$jID
                            return=$?
                            iNext=$((iNext+1))
                            ./child $iNext &
                            JobIDArray[i]=`jobidfromstring $(jobs %%)`
                            echo "$jID was returned. Its exits status was: $return   |   JobIDArray[$i] = ${JobIDArray[i]}"
                    fi
            done
    done


    ---------------------------

    I am pretty much stuck at where to begin next. I am very new to bash and I don't even know if its possible to do what I want. My third script would work fine if I just had a way to identify if a job was terminated or not. I know that if you try to kill -s SIGCHLD the job when its terminated, it will give "no job exists" or something. I can't seem to know how I can get that to work in an IF statement. And if I did, would wait() still get the exit code.

  2. #2
    Linux Enthusiast likwid's Avatar
    Join Date
    Dec 2006
    Location
    MA
    Posts
    649
    Well, this would actually be a lot easier in Python IMO. But, you can use trap to do things when a signal is received. IDK if that helps in any way.

  3. #3
    Just Joined!
    Join Date
    Sep 2007
    Posts
    7
    I know it will be a ton easier in another language, I havent dove into compiling on linux yet so I want to atleast get this concept in bash if possible. I also know about trap, i used it in my 2nd example.

  4. $spacer_open
    $spacer_close
  5. #4
    Linux Enthusiast likwid's Avatar
    Join Date
    Dec 2006
    Location
    MA
    Posts
    649
    Ah, didn't notice that. Python is an interpreted language btw. I can't offer much help because I've never attempted to do something like this in bash. Good luck

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •