Posix unnamed semaphore - undefined behavior

[xelag]

Hi,
I have a problem with Posix unnamed semaphore used to synchronize threads. I want to first to explain the context of my system...

My threads allow to activate some sub-routines which are each one related to a flag. The latter are tested in a infinite loop executed in the thread which is blocked at each loop by a semaphore (sem_wait()). With same code, it'll be clearer, I hope :

//function executed separately than the main execution flow of the program
void myThreadedFunction()
{
  while(true)
  {
    if(mySubRoutine1_activated)
    {
      mySubRoutine1_activated = false;
      mySubRoutine1_Private();
    }
    else if(mySubRoutine2_activated)
    {
      mySubRoutine2_activated = false;
      mySubRoutine2_Private();
    }
    //...
    
    //stop the thread until the next activation
    sem_wait(&mySemaphore);
  }
}
 
//Public function called from the main execution flow of the program
void MySubRoutine1_Public()
{
  mySubRoutine1_activated = true;
  //activate the loop in myThreadedFunction
  sem_post(&mySemaphore);
}
 
//Private function, called from myThreadedFunction
void mySubRoutine1_Private()
{
  //process code
}

I have to interrupt this thread asynchronously.

I can manage to do it because when I initialize it, I setup the interruptions with :

pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);

And I interrupt it with a call to the function :

pthread_cancel(threadRef);

The thread interrupts with success.
When I reinitialize the thread, first, I reinit the activation flags related to sub-routines, as well as the semaphore used to block/activate the loop by the following way:

//destroy the old semaphore
sem_destroy (&mySemaphore);
//initialization of the new semaphore
//(initialize the semaphore to 0, so at first sem_wait(), the thread will wait until
//that one activates one of the sub-routine and that a sem_post() is done).
sem_init(&mySemaphore, 0, 0);

Next, initialization of the thread :

// The thread is detached
pthread_attr_setdetachstate(&pthread_attr_t,PTHREAD_CREATE_DETACHED);
//initialize the thread and run it into the function myThreadedFunction
pthread_t threadRef;
pthread_create(&threadRef, &pthread_attr_t, myThreadedFunction)

And now, here is the fact..
I manage to interrupt my threads, to reinitialize and to run them again. But, sometimes with an unexpected way, after have restart my threads, when I do a sem_post() over the semaphore mySemaphore to activate the loop, the semaphore increments well (I checked it out with the function sem_getvalue()), but the thread waiting on the sem_wait() in the infinite loop doesn’t unblock!

I read in the doc MAN, that to do a sem_destroy(&mySemaphore) although this semaphore got threads waiting, can bring undefined behavior.

“Destroying a semaphore that other processes or threads are currently blocked on (in sem_wait(3)) produces undefined behaviour.”

Linux MAN page


Could it be the cause of my problem? However, I have the impression that I reinitialize everything properly and twice on three, it’s working perfectly..

I thank you already for your help, I need it much!
Alex

PS: sorry for my English, it’s not my mother tongue…

[wje_lf]

sorry for my English, it’s not my mother tongue

Actually, you're doing quite well.

It's difficult to tell exactly what's going on with your code, but the chief problem is that you're canceling a thread asynchronously. This means that it can be canceled between any two hardware instructions. In turn, that means that you shouldn't do anything in this thread that you would need to run to completion. This means that your thread should only do compute-bound things, and should avoid calling just about any system or library function. Let's hear it from David R. Butenhof's excellent book Programming with POSIX Threads:

... you should never call any function while asynchronous cancellation is enabled unless the function is documented as "async-cancel safe." The only functions required to be async safe by Pthreads are pthread_cancel, pthread_setcancelstate, and pthread_setcanceltype. ... No other POSIX or ANSI C functions need be async-cancel safe, and you should never call them with asynchronous cancelability enabled.

...

The consequences of asynchronous cancellation in a function that is not async-cancel safe can be severe. And worse, the effects are sensitive to timing -- so a function that appears to be async-cancel safe during experimentation may in fact cause all sorts of problems later when it ends up being canceled in a slightly different place.

This explains your observation:

twice on three, it’s working perfectly

You're actually quite fortunate that it failed one time out of three. It could succeed 99 times out of 100, and then when it absolutely needs to run correctly, it fails.

There are many, many things that can get messed up if you interrupt them in the middle. Examples:

1. malloc() or free(), because you can trash your heap;
2. pthread_mutex_lock, because interruption within that function can cause the mutex to be left in an inconsistent, illegal state; aaaaaand:
3. sem_wait()!

Why aren't you using deferred cancellation instead of asynchronous cancellation? Mr. Butenhof gives us this advice:

Avoid asynchronous cancellation!
It is difficult to use correctly and is rarely useful.

[xelag]

Why aren't you using deferred cancellation instead of asynchronous cancellation? Mr. Butenhof gives us this advice:

Because I wanted to be able to cancel my threads instantly without to add cancellation points into different parts of my code. But, that's what I'm going to do now.. I didn't know that asynchronous cancellation was so dangerous..

Thank you very much for your support!
Alex

[xelag]

sorry for my English, it’s not my mother tongue

Actually, you're doing quite well.

Thank you!

I have a question about how to manage deferred cancellations.
I have several points in my threads where they can be blocked (semaphore, mutexes...). How to manage a deferred cancellation if the threads are blocked on one of those sync tools?
Should I put run a function unlocking all those sync tools potentially blocking my threads and creating a cancellation point after each sem_wait() or pthread_mutex_unlock() with a call to pthread_testcancel()?

Thank you for your help,
Alexandre

[wje_lf]

1. If the thread which can be canceled obtains any resources, such as heap space (via, for example, malloc()) or locked mutexes, you need to make sure that all this is cleaned up when you cancel a thread. For that purpose, you need to use pthread_cleanup_push() to declare a thread cleanup function, and pthread_cleanup_pop() to indicate that cleanup is no longer needed.
2. If you use pthread_cond_wait() or pthread_cond_timedwait() for synchronization and the thread is canceled, then the thread will be woken up with the associated mutex locked. pthread_cond_wait() and pthread_cond_timedwait() are cancellation points, so you don't need to introduce an additional cancellation point at that place in the code. But your thread cleanup function will have to unlock that mutex, as well as return any other resources the thread is holding. It may also have to fix up any synchronization bookkeeping you do with variables.
3. I don't know where POSIX semaphores fit into this. In particular, I don't know whether cancelling a thread will cause it to wake up from waiting from a POSIX semaphore. If it's possible to cancel a thread, I would advise that you use a mutex and a condition variable instead of a POSIX semaphore for synchronization. It'll just be cleaner in the long run.

[xelag]

I found on this web site that sem_wait was also a cancellation point.

So I tried just to change the cancellation type to deferred cancellation, and it seemed to work until I got the same exact problem with a sem_wait which doesn't wake up after a sem_post...

Afterwards, I tried to implement my own system of interruption with exceptions.
I keep a trace of the semaphore into which my thread is blocked. When I want to interrupt my thread, if the latter is blocked, I do a sem_post on the semaphore and at the exit of sem_wait, I raise an exception which make the thread leaving properly. But it's still not working... ((

I spent hours on this problem. I'm quite stuck right now. I thought the brute-force asynchronous was the reason of my problem.. some hope, and.. disillusion..

Help...

[wje_lf]

This won't give you instant help, but it might be healthful in the long run.

Since you're serious about POSIX threads programming, I recommend that you run (don't walk) to your nearest bookseller and order David R. Butenhof's excellent book Programming with POSIX Threads, published by Addison Wesley. Don't buy the O'Reilly book on the same subject; at least my copy of the O'Reilly book, while apparently technically complete and accurate, doesn't convey with sufficient clarity the philosophy behind the design of POSIX threads, or many of the gotchas that lurk behind every turn.

Then take a weekend, curl up, and read pretty much the whole book.

Then take a couple of days to let that knowledge bake in your brain.

Then go back to your code and review everything in it.

I sense that's the best I can do here for you. Sorry. :(