3K threads

[jfranzoy]

Hi. I am facing a problem I never imagined and I was asking if somebody there can help me.

I am a writer and supporter of a c++ library. The library is compiled as an .so/.dll (depending on the platform). I offer, among other things, a class named Thread. The Linux implementation of this class uses pthread.

My library will be used in a new big project that need a program to start 3000 threads. There is no time to change it to use less thredas.

The problem is that the default stack size in Linux is 10MB. I would need 10MB x 3000 = 10GB of virtual space, which is impossible in IA32/Linux as far as I know.

I think that the solution is to make the stack size configurable in my Thread class.

There is a pthread_attr_setstacksize function. But this function just set the "minimum" stack size. It continues taking 10MB per stack.

There is a pthread_attr_setstack function that enables you to set the stack address and size. All was fine in the toy samples, but I get an SIGSEGV when I tried to use it in my library.

I came up with a minimal code that reproduce the problem. It's in the attachment. The failing-example consist of:
- d.cpp that calls my_pthread_create in c.so
- a shared library called c.so that exports 'my_pthread_create( void *(fn)(void *) )' function. This function:
- initialize the pthread_attr_t
- set the PTHREAD_CREATE_JOINABLE ATTRIBUTES
- allocate 1MB + a page with malloc
- call pthread_attr_setstack, setting the new allocated memory
- call pthread_create where it crash

If there c.cpp is not compiled as a shared library, there is no signal 11.

What is the problem? Am I doing something wrong?

Thans

[kyku]

Doesn't the problem have anything in common with the compilation of program? I compiled it as follows:

g++ -Wall -shared -g -o c.so c.cpp -fPIC -DPIC -pthread
g++ -Wall -g -o d d.cpp c.so -pthread
LD_LIBRARY_PATH=. ./d

And it seems to be working:

pthread_attr_init done
pthread_setdetachstate done
allocated '1028096' bytes for stack on 0xb7be0008
pthread_attr_setstack done
new thread created tid=3083705232
new thread tid=3083705232
join done.

Please provide some feedback as the problem looks interesting, and I'd like to know the reason.