Terminating a Pthread program

[hjazz6]

Hi all,
I'm running Linux Red Hat Enterprise 5, and I'm writing a program using Pthreads. I am looking for a way to correctly terminate the program using signals.

Say I have 5 consumer threads + 1 producer thread running. I have the producer thread running pcap_loop and the other 5 consumer threads processing the packets. At some point, I want to terminate the program by pressing Ctrl-C. I have the following:

signal(SIGINT, terminate_process);

where terminate_process(int sig) is the function I want to run when the program terminates. One of the things I want to do when the program terminates is print the number of packets received, but that involves a counter that's only updated in the producer thread.

I have a series of statements, which are correctly printed after I
press Ctrl-C, but then the program just hangs there. If I press Ctrl-C
again get the error "*** glibc detected *** ./a.out: double free or
corruption (!prev): 0x000000000098abd10 ***"

Here is my terminate_process function:

void terminate_process(int sig)
{
   /* several printf statements */
   pcap_breakloop(handle);
   pcap_close(handle);
}

"handle" is a global variable for pcap_loop.

I'm thinking that everytime I interrupt the program by pressing Ctrl-
C, I'm interrupting either the producer thread which runs the
pcap_loop function, or one of the consumer threads which process the
packets. Depending on which thread I interrupt, the other threads
might not get terminated and so they are left hanging waiting for the
next packet to process (consumer) or receive (producer).

My question is how to I correctly terminate the program?

I also want to mention that previously, I had a few MPI functions in
the program, so I compiled and ran it using mpicc and mpiexec. So when
I terminated the program by pressing Ctrl-C, it killed the entire
process and the terminate_process() worked correctly.

Thank you.

Regards,
Rayne

[wje_lf]

The best book by far on POSIX threads programming is the one published by Addison Wesley, Programming with POSIX Threads, by David R. Butenhof. He has a whole section on using signals in pthreads programs. It begins with the following ominous quote from Lewis Carrol:

Beware the Jammerwock, my son!
The jaws that bite, the claws that catch!
Beware the Jubjub bird, and shun
The frumious Bandersnatch!

He lays some background, and then says:

It is always best to avoid using signals in conjunction with threads. At the same time, it is often not possible or practical to keep them separate.

He does have several pieces of advice if you must mix signals and POSIX threads. Here are some of them:

1. Don't use sigprocmask().
2. Use just one thread to catch all signals.
3. In your main thread, use pthread_sigmask() to mask off all signals. That mask will be inherited by all threads you create.

Within that framework, there are two ways to handle signals. Here's the first way.

1. In the thread intended to catch signals, use pthread_sigmask() to unmask those signals you wish to handle.
2. Don't do any more work than necessary in the signal handler.

Here's the second way, which is probably cleaner.

1. Devote a thread to nothing but handling signals. Do not use pthread_sigmask() to unmask those signals, or any others. Keep all signals masked in this thread, just as in all the other threads.
2. This thread should be a loop containing a call to sigwait(). That function will return when a signal is to be handled. Handle it, and then repeat the loop.
3. Do not use sigwaitinfo() or sigtimedwait(). They add burdensome subtleties which are a result of two standards (POSIX realtime and POSIX threads) being finished in an unfortunate order.

Better yet, throw all this away. Don't handle signals. Instead, use tcgetattr() and tcsetattr() to disable the mechanism that translates ^C into a signal, and just do the necessary when the ^C is input. This is more complicated than it looks at first, but it's not rocket science.

If you use either of these approaches (using signals correctly in POSIX threads, or not using signals at all), you're more likely to eliminate the symptom variation you noticed when you introduced MPI.

Hope this helps.