Understanding pthread_cond_wait()

[hjazz6]

Hi all,

I would like to know if my understanding on how the program handles pthread_cond_wait() and pthread_cond_signal() is correct.

Given the code below,

// Thread A
while (1)
{
do_Awork();
signal = 1;
pthread_cond_signal(&condition);
signal = 0;
}

// Thread B, C, D
while (1)
{
pthread_mutex_lock(&mutex);
while (!signal)
pthread_cond_wait(&condition, &mutex);
do_BCDwork();
pthread_mutex_unlock(&mutex);
}

"mutex" and"condition" are global variables that have been initialized. "signal" is also a global variable that has been initialized to 0. The program should only execute do_BCDwork() in Thread B, C and D when Thread A finishes do_Awork() and changes signal to 1. Then Thread A signals either Thread B, C or D, sets signal to 0 again and repeat the whole process.

I would like to understand the flow and logic of the above code, and please tell me if I misunderstood anything.

I'll start with say, Thread B that enters the while(1) loop, locks mutex and enters the while (!signal) loop, which is true and so it calls pthread_cond_wait(), at the same time unlocking the mutex. Now Thread C goes into the while(1) loop, locks the mutex, enters the while (!signal) loop, which is true and it calls pthread_cond_wait(), at the same time unlocking the mutex. Sane thing happens to Thread D. So we now have all three threads waiting at pthread_cond_wait().

Say, at this point Thread A enters its while(1) thread, runs do_Awork(), then sets signal = 1. It then calls pthread_cond_signal() and say, Thread C gets woken up. (Any of the three threads can get signaled, right? It need not be in the order they called pthread_cond_wait()?)

Thread C, after waking up, returns from pthread_cond_wait() and checks the while(!signal) loop, which is now false and so it goes out of that while loop, locks the mutex and runs do_BCDwork(). Thread A sets signal back to 0. (One question, could Thread A set signal = 0 faster than Thread C checking the while(!signal) loop, hence, resulting in Thread C calling pthread_cond_wait() again since the while-loop condition is not met? If so, should I put a usleep(100) before signal = 0 so that this won't happen?)

After running do_BCDwork(), Thread C unlocks the mutex. Then it locks the mutex again since it has to stay in the while(1) loop, checks that while(!signal) is true and calls pthread_cond_wait(). This process repeats itself, only either Thread, B, C or D can be woken up each time Thread A sets signal = 1.

Is my understanding of the logic correct? What other things should I look out for?

Both do_Awork() and do_BCDwork() access different parts of a global queue (array of pointers). Basically, it's a producer(Thread A)-Consumer(Thread B, C, D) relationship. Thread A puts an item into the queue, Thread B, C, D take it out of the queue. I tried to achieve that by using pthread_cond_wait, so Thread B, C and D wait till Thread A puts something in the queue before trying to access that item. Initially I put a mutex around the queue, but it means that at any one time, only one thread can access the queue, and it slows down my program, so I was thinking if there was a way to have several threads simultaneously access different parts of the queue (via do_Awork() and do_BCDwork()). That is, Thread B, C, D first simultaneously access items already in the queue. When the queue is empty, they wait till Thread A puts an item in the queue. Then one thread gets woken up and accesses that item via do_BCDwork(). During this time, Thread A might have put another item in the queue, and another thread can be woken up by Thread A and it carries out do_BCDwork() too. However, I realize that if this were to happen, say Thread C gets woken up first, it would lock the mutex while it runs do_BCDwork(). If Thread B were also to get woken up at this time, it would also try to lock the mutex that Thread C is holding, in order to execute do_BCDwork(), which would give me an error (This might be why I keep getting "Segmentation fault" in my program). Is there some way to workaround this?

Thank you.

Regards,
Rayne

[wje_lf]

Any of the three threads can get signaled, right? It need not be in the order they called pthread_cond_wait()?

That is correct.

could Thread A set signal = 0 faster than Thread C checking the while(!signal) loop, hence, resulting in Thread C calling pthread_cond_wait() again since the while-loop condition is not met?

Yes.

should I put a usleep(100) before signal = 0 so that this won't happen?

No.

Is my understanding of the logic correct?

No.

What other things should I look out for?

You have here a global queue, an array of pointers. Presumably you have a mechanism that when you reach the end of the array, you can re-use the members at the beginning of the array if they're no longer needed.

You want A to fill the queue as fast as it can, but if the queue is completely full, you want it to wait until at least one member of the queue no longer contains interesting data before filling that member and then checking again whether there's an empty member of the queue.

You want B, C, and D to empty the queue as fast as they can, but if the queue is empty, each should wait until at least one member of the queue contains interesting data.

What you want to protect is not all the data in the queue, but only that mechanism which shows which members of the queue contain interesting data, along with related indicators, like which member of the queue contains the next item to be consumed, and which member of the queue is to receive the next produced item. (I'm oversimplifying this because you have more than one consumer.)

There are two rules of interest with respect to POSIX threads programming.

1. Don't make any assumptions as to the relative speed of execution of various threads. Assume that the threads will execute at the worst imaginable relative speeds so as to expose errors in synchronization.
2. Don't try to "fix" problems by making certain threads run more slowly. This hides bugs in pthreads programming, but doesn't fix them.

So have one mutex which protects the data which shows which members of the queue have active data. Have two condition variables: one indicating whether the queue is empty (I'll call it cond_empty), and one indicating whether the queue is full (cond_full). Have each member of the queue be in one of four states:

1. not containing data and available to be filled;
2. in the process of being filled by the producer;
3. containing valid dat which is available to be emptied; and
4. in the process of being emptied by a consumer.

There's nothing in your code which keeps the producer thread from filling up the queue and ... keeping right on going, filling it up some more.

So you want your producer thread to do something like this:

while(1)
{
  status=pthread_mutex_lock(&mutex);

  if(status!=0)
  {
    blow_up();
  }

  while(queue is full)
  {
    status=pthread_cond_wait(&cond_full,&mutex);

    if(status!=0)
    {
      blow_up();
    }
  }

  /* We emerge from the loop with the mutex still locked, because
   * pthread_cond_wait() will leave it that way.
   */

  mark the queue member as being in use, and being filled with data;

  status=pthread_mutex_unlock(&mutex);

  if(status!=0)
  {
    blow_up();
  }

  produce the data for the queue member;

  status=pthread_mutex_lock(&mutex);

  if(status!=0)
  {
    blow_up();
  }

  mark the queue member as being full of data ready to be consumed;

  status=pthread_mutex_unlock(&mutex);

  if(status!=0)
  {
    blow_up();
  }

  /* The queue is at this point not empty. */

  status=pthread_cond_signal(&cond_empty);

  if(status!=0)
  {
    blow_up();
  }
}

You want your consumer code to do the same thing, reversing which condition variables you test and signal, which states you set in the queue member of interest, and which test you make on the queue (whether empty, not full).

Disclaimer: I have not tested the above code.

For more details (essential ones) on POSIX programming, run (don't walk) to your nearest bookseller and order David R. Butenhof's fine book Programming with POSIX Threads, published by Addison Wesley. My copy of the O'Reilly book on the same subject somehow manages to be complete, but doesn't warn of many potential gotchas in using it. Stick with Addison Wesley here.

Hope this helps.