Related to handling of Named Pipes.

[applegrew]

I need to make a program which can write data to multiple named pipes, but if any of the named pipes closes then can I trap the SIGPIPE signal and just stop sending data there. Also I need to write to the pipes in non-blocking mode so that data to other pipes donot get interrupted, but what do I need to do re-syncronise the data to the pipe which recently paused (blocked)?

[applegrew]

Bump................

[wje_lf]

what do I need to do re-syncronise the data to the pipe which recently paused (blocked)?

This question is difficult to understand. Can you phrase it in different words? Perhaps be more detailed about what you want to do?

[applegrew]

This question is difficult to understand. Can you phrase it in different words? Perhaps be more detailed about what you want to do?

oops. OK.
From the time I posted this thread I have figured out most of the things. The portion that you have quoted means that when I open the fifo for wrting in non-blocking mode then what happens when I try to write, but the reader process is still busy; simply speaking it is not the right time to write. Well as I have figured out that I will receive a EAGAIN error. So, I am now planning to keep checking for that error. If that occurs then I will write the pending data to a temporary file. When the reader process becomes ready then I will read the data out of the temporary file. Does this scheme sounds ok?

Ok I do have some new issues. I can't understand the meaning of the parameters of pthread_create() and how do I get reference to the currently running thread, actually the main() thread?

[wje_lf]

The standard way of handling multiple streams of input and output, reading and writing as the occasion arises, is to use select(). Do this at the command line:

man select

If you use select(), you won't have to use threads at all.

POSIX threads can be a nightmare to debug if you don't understand them thoroughly (or even if you do).

Any particular reason you're using POSIX threads?

[applegrew]

The standard way of handling multiple streams of input and output, reading and writing as the occasion arises, is to use select(). Do this at the command line:

man select

If you use select(), you won't have to use threads at all.

POSIX threads can be a nightmare to debug if you don't understand them thoroughly (or even if you do).

Any particular reason you're using POSIX threads?

I am using POSIX threads only because I didn't know about select(). In fact I am just now getting the hang of programming for Linux. But, I think select() is not the proper choice for me. Let me describe what I really need to do.

I am trying to create a program that can read data (television broadcast video stream) from a fifo and will send that video stream to other (1,2 or more) fifos. My scheme is that a thread will read data from the in-fifo and there will be threads for writing to each out-fifo. Now, suppose at one out-fifo I have made mencoder read the output stream and record (save) the video to harddisk. If at anytime I need to pause the video recording then I will just suspend this thread. Now at the other out-fifo I have made mplayer play the video stream, but I need to be able to pause the live feed say for at max 1/2 an hour. My scheme is to create a temporary buffer (which will be saved to hard disk) required for buffering the lag of 1/2 an hour of video.

I hope I have made this clear enough. Can you suggest a good scheme or there is already a ready-made software for this?

[wje_lf]

This sounds like an interesting project.

On the one hand, I don't fully understand the project description you just gave. If I wanted to understand it fully, I would have several questions at this point. Unfortunately, I have another interesting project of my own at this point that requires my attention, so I won't be asking those questions. (grin)

On the other hand, you've given me enough information to focus on one of the questions in your original post.

So at this point, my comments are these:

Yes, it looks as though POSIX threads might be a good idea for your program. But applications using POSIX threads can be difficult to debug. The main source of difficulty is that all the threads share the same heap space. If a particular thread dies (or worse) because of bad data in the heap, it might be some other thread that corrupted the data. So you can't just single step through the thread in question and watch the data go bad.

The point here is: Keep the application as simple as possible. This is very important.

If you're going to use POSIX threads, run (don't walk) to your nearest bookseller and get David R. Butenhof's excellent book Programming with POSIX Threads. It's published by Addison-Wesley. Then take a weekend and read pretty much all of it. You will easily save much more time than that with the knowledge you gain. Butenhof shows not just surface knowledge, but an inner understanding of the complications and potential dangers of using POSIX threads.

There's another book about POSIX threads published by O'Reilly. My copy of that book doesn't have the deep underlying understanding of POSIX threads that the Butenhof book does, so I wouldn't recommend the O'Reilly book.

A problem presents itself, but it has a simple solution. The problem is that signals are global to the whole process, and cannot be associated with a particular thread.

The solution is to ignore SIGPIPE signals and check for an EPIPE error when writing.

I just wrote and briefly checked out the following two toy programs: a producer and a consumer. Start the consumer first. Then, in a separate window, start the producer. After watching the action for a while, kill the consumer. Watch the producer recover gracefully without blowing up.

Here's the producer:

#include <sys/select.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <errno.h>
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
  int     phyle;
  int     select_value;

  char    tiny_buffer[4];

  ssize_t byte_count;
  ssize_t write_value;

  fd_set  write_set;

  signal(SIGPIPE,SIG_IGN);

  phyle=open("named_FIFO",
             O_WRONLY | O_NONBLOCK
            );

  if(phyle<0)
  {
    if(errno==ENXIO)
    {
      fprintf(stderr,
              "consumer has not opened the pipe yet\n"
             );
    }
    else
    {
      perror("open(\"named_FIFO\")");
    }

    exit(1);
  }

  byte_count=0;

  for(;;)
  {
    FD_ZERO(&write_set);
    FD_SET(phyle,&write_set);

    select_value=select(phyle+1,
                        NULL,
                        &write_set,
                        NULL,
                        NULL
                       );

    if(select_value!=1)
    {
      fprintf(stderr,
              "bad select value %d\n",
              select_value
             );

      exit(1);
    }

    if(!FD_ISSET(phyle,&write_set))
    {
      fprintf(stderr,
              "select() did not set write bit\n"
             );

      exit(1);
    }

    write_value=write(phyle,
                      tiny_buffer,
                      sizeof(tiny_buffer)
                     );

    if(write_value>0)
    {
      byte_count+=write_value;

      printf("wrote %d bytes (%d total)\n",
             write_value,
             byte_count
            );
    }
    else
    if(write_value==0)
    {
      fprintf(stderr,
              "we wrote zero bytes? this should not happen\n"
             );

      exit(1);
    }
    else
    if(errno==EPIPE)
    {
      printf("normal closing of pipe by consumer\n");

      break;   /* <--------- */
    }
    else
    {
      perror("abnormal error on writing to pipe");

      exit(1);
    }
  }

  return 0;

} /* main() */

Here's the consumer:

#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
  int  phyle;

  char tiny_buffer[4];

  umask(0077);

  unlink("named_FIFO");

  if(mkfifo("named_FIFO",0700)<0)
  {
    perror("mkfifo()");

    exit(1);
  }

  phyle=open("named_FIFO",
             O_RDONLY
            );

  if(phyle<0)
  {
    perror("open(\"named_FIFO\")");

    exit(1);
  }

  for(;;)
  {
    printf("waiting for a byte\n");

    read(phyle,
         tiny_buffer,
         1
        );

    sleep(3);
  }

  return 0;   /* never gets here */

} /* main() */

Hope this helps.

[applegrew]

hey thanks for the codes. I will study them and BTW it is too late now for me to run to a bookshop. Till then I will read an see this ->POSIX Threads Programming , and will trouble u here . Furthermore if have time then I have another interesting data structure related problem related to this project. I have posted it here CodeGuru Forums - Need your opinion about this data storage structure. .

[applegrew]

hey I checked ur code. Does unlink also deletes regular files? And I have seen unmask before too but have not been able to understand what it is.

[wje_lf]

I'll get to your question in just a moment, but first I need to say something more general.

I highly recommend you become familiar with the man pages. ("man" is short for "manual"). They are called pages not to remind us of web pages, but because they used to be stored in binders, as paper. (I still have a set, from Interactive UNIX, which was bought out long ago by Sun.) They were divided into physical sections. The man pages are now "divided" the same way.

Some command names are represented in more than one section, because the name in question may actually name (for example) both a program and a function.

To get the man page for a particular program or function, do this at the command line:

man whatever

The man page will come up in less, so you can browse it as you would normally browse a text file using less (even though these are not straight, legible text files).

That command will get you one man page. You will get no warning if there is more than one man page for this name. To get them all, do this:

man -a whatever

You will see one man page. When you're done looking at that one, do

q

as you would normally do to get out of less. The next command page, if any, will spring up.

Lather, rinse, repeat.

To get a general idea of what's in each section, do this:

man -a intro

I highly recommend that you do this:

man man

You can get more information on man pages here.

And now for your question:

Does unlink also deletes regular files? And I have seen unmask before too but have not been able to understand what it is.

It isn't unmask, it's umask.

Do this at the command line:

man -a unlink
man -a umask

and all will be revealed.