Error with socket shared between functions

[striker07]

Hello!

Let's show the problem: I have a socked connection (in a fd var). In certain moment, I need to use the same socket in two threads. Lets see in the example:

int main(void)
{
   int fd; // the socket
   pthread_y myThread;
   
   // Performs connection and another routines
   
   pthread_create(&myThread, NULL, anotherThread, (void*) &fd);
   while (1)
   {
      // Some processing
      send(fd, "Some data", strlen("Some data"), 0);
   }
}

void *anotherThread(void *fd)
{
   int *socket;
   char aBuffer[1024];

   socket = (int*) fd;

   while (1)
   {   
      // some processing
      send(*socket, aBuffer, strlen(aBuffer), 0);
   }
}

Well, the problem is that, in the thread (anotherThread function), I can't send data. The send() function returns -1 (error signal).

What I can do to solve this?

[Rubberman]

The simple answer is "you can't do that". Implement a linked list that contains buffers to send. Each thread will add their buffer to the tail of the list. Another thread will send the buffers it finds on the list. To do this you will need to implement a mutex or other semaphor to guard access to the list. In any case, you want at most one reader and one writer thread on a single socket.

Alternatively, you can create your own function that does the actual send, and inside that function, guard access to the actual send. That way, only one thread can effectively "send" on that socket at any one time. This may be easier to implement than the multi-thread and linked-list type of technique.

[striker07]

Well, this isn't so easy then. Let's remember queue implementation in C...

Thank you, Rubberman!

[tetsujin]

The simple answer is "you can't do that".

Really? That seems strange. I mean, I understand there's the practical problem, that if you want the messages you send to be formatted properly, you need to synchronize when you write data to that fd...

But assuming that the programmer has dealt with that issue: POSIX threads are supposed to share file descriptors. So why should the send() call in the thread fail? (I guess getting errno would provide some clue...)

[Rubberman]

A lot of functions are not reintrant. IE, two threads cannot use them at the same time because they may use local static variables and other similar constructs that preclude such usage. There are often reintrant versions of the same functions that end in _r, so if there was a reintrant version of send(), it would be send_r() most likely. Even if the function were reintrant, at best you could expect the messages to be mixed together at the receiving end. So, basically what I am saying is even if it worked, it would not be a good situation. Read the man pages to see what it says about this stuff. In any case, if one of my engineers did this, I would give them a solid whack upside the head and then route them to tech support so when customers called with problems such constructs caused, they would have the joy of debugging them...

[tetsujin]

In any case, if one of my engineers did this, I would give them a solid whack upside the head and then route them to tech support so when customers called with problems such constructs caused, they would have the joy of debugging them...

Well, I won't argue with that. I don't figure there are many cases where this would be what you want. But in some cases it could be (i.e. lock a semaphore, write a full, self-contained message, release the semaphore)

I'm mostly interested in the technical reasons for why this wouldn't work - as opposed to the various reasons it's not generally a good idea.

[Rubberman]

Well, I won't argue with that. I don't figure there are many cases where this would be what you want. But in some cases it could be (i.e. lock a semaphore, write a full, self-contained message, release the semaphore)

I'm mostly interested in the technical reasons for why this wouldn't work - as opposed to the various reasons it's not generally a good idea.

From my original reply to the thread, I mentioned that you could have two threads writing to the same socket provided that the writes were properly guarded, as per your comment above. However, in such as case, the threads are NOT writing simultaneously as one will be waiting on the semaphore or mutex while the other is writing the data to the socket. That's fine, and should be clearly commented that such is the intention in the source code.

As for the technical reasons why the send() won't work from two threads simultaneously, that depends entirely on the implementation of the function. A review of the code may be illustrative of the reasons why, but it may lay deeper in the user-space source, or even in the kernel-space code. Right at this moment, I do not know the answer to that. The last time I implemented the suite of Berkeley Sockets functions was around 1990 for a real-time operating system, and since we didn't support fine-grained threading (processes only), this wasn't a real issue.

[xuancong]

I'm tried to do this on Windows before, and it works, so there are 2 possibilities:

1. Your 2nd send fails because your 1st send has not finished yet, in either case, you must call select(NULL, &write_fds, NULL) with infinite waiting time first before attempt to write to any thread-shared sockets. Bear in mind that modern computer systems are much faster than what you think, it takes several miliseconds to send a packet of any size while it takes only a few microseconds or even less to start a new thread and reaching your 2nd send function.

2. The implementation of your OS is inferior than Window, OOPs, so bad, it just doesn't work.

Take note that even in Windows, I have to call select before send if using shared socket. For single thread, you don't need to wait because by default, send function returns only when it finishes sending, so you can call next send immediately.

Xuancong

[unlimitedscolobb]

Let's show the problem: I have a socked connection (in a fd var). In certain moment, I need to use the same socket in two threads. Lets see in the example:

Let me throw in my twopence. If you want to use a socket in two threads, you absolutely must set it up before spawning the second thread. Otherwise you get an ugly race condition (Race condition - Wikipedia, the free encyclopedia).

And the generalizing remark: always guard your shared mutable state (state = variables) with a mutex, semaphore, or something else.

[striker07]

Wow, this discussion is very interesting!

Well, the project that I'm developing is running in an embedded system running a limited version of Linux (that I don't know the exactly version...).
In that system, I must listen a socket connection, process the data, and send it to a serial port. At the same time, I listen the serial port, process the data, and send it to the socket. Like a bridge, but with some processing in the middle of the way.

After the answer of Rubberman, I plan to do that:

3 Threads
3 Queues

1. A thread to receive and send data to the socket. The received data, is enqueued in a queue of data waiting for processing. This thread reads another queue with data waiting to be sent.
2. The second thread will read data from the serial port, add a little header, and put this data into the queue of data waiting to be sent. This threads reads data from the queue of processed data.
3. The last thread, read data from the queue of unprocessed data, process them and put it, or in the queue of data ready to be sent, or in the queue of data waiting to go to the serial port.

This problem of sharing a socket, or a file descryptor, is present here two times. This idea showed above is valid?

PS: Sorry for my poor English!