Warning: fetch_template() calls should be replaced by the vB_Template class. Template name: right_column in ....includes/functions.php on line 4597

Warning: array_merge() [function.array-merge]: Argument #1 is not an array in ....includes/class_bootstrap.php(1419) : eval()'d code on line 1

Warning: array_merge() [function.array-merge]: Argument #1 is not an array in ....includes/class_bootstrap.php(1419) : eval()'d code on line 2

Warning: array_merge() [function.array-merge]: Argument #1 is not an array in ....includes/class_bootstrap.php(1419) : eval()'d code on line 4

Warning: array_merge() [function.array-merge]: Argument #1 is not an array in ....includes/class_bootstrap.php(1419) : eval()'d code on line 5
Buffered vs Unbuffered I/O Linux article
Welcome to Linux Forums! With a comprehensive Linux Forum, information on various types of Linux software and many Linux Reviews articles, we have all the knowledge you need a click away, or accessible via our knowledgeable members.
Find the answer to your Linux question:
Site Navigation
Linux Forums
Linux Articles
Product Showcase
Linux Downloads
Linux Hosting
Free Magazines
Job Board
IRC Chat
RSS Feeds
Free Publications


When migrating C programs from systems like Solaris to systems like Debian (Linux), there's more than BIG-ENDIAN to LITTLE-ENDIAN differences to care about. Buffered vs Unbuffered I/O is also an important consideration, especially when the program being ported is a multi-user-access database system.

I recently had to migrate a large multi-user-access database system from Solaris to Linux.  I took care of big/little-endian by means of C++ data-types called xlong/xshort (See: http://www.stanford.edu/dept/its/support/uspires/xlong/ ).  But buffered I/O no longer worked as it did in Solaris, where buffers where maintained at the system level.  Any changes made to records by one user were immediately known to other users.  But in Linux, buffers were maintained locally in each user's workspace.  So if user A altered a control block in A's buffer, it was NOT known in B's buffer.  Each time B tried to read a control block, B got the same (old) information.  None of A's changes appeared in B's block.

The solution was to recode the file I/O to use "read" and "write" and "lseek" instead of "fread" and "fwrite" and "ftell", along with a few other functional differences.  The files could still be opened and closed with "fopen" and "fclose", yielding a FILE di (device identifier), but I/O had to be done with a file identifier:  fid = fileno(di).

What follows is a summary of all the if-tested logic added to the program to allow it to work on both Solaris (buffered) and Linux (unbuffered):


/*- Use -DEMFBUFF to define buffered I/O everywhere -*/

#define Fsync(x) fsync(fileno(x))

---

    if ((mydi = curDev->di) != 0) /*- device assigned -*/

    {  /*- Determine where file ends -*/

#ifdef EMFBUFF

       if (!fseek(mydi, 0L, SEEK_END))  /*- Position -*/

          llen = ftell(mydi);  /*- Get EOF position -*/

#else

       lseekpos = lseek(fileno(mydi), 0L, SEEK_END);

       if (lseekpos > 0) llen = (long)lseekpos;  /*- Position -*/

#endif

       clearerr(mydi);  /*- Everything is error free -*/

    }

---

 {  /*--- Process READ and WRITE for files ---*/

#ifdef EMFBUFF

    long initpos;   /*- Initial position -*/

    long position;  /*- Position in file -*/

#else

    off_t initpos;   /*- Initial position -*/

    off_t position;  /*- Position in file -*/

    off_t lseekpos;  /*- lseek position -*/

#endif

    long blklen;  /*- length of I/O data -*/

    long blknum;  /*- block to read/write -*/

    byte *blkloc;  /*- location of buffer -*/

#ifdef EMFBUFF

    int excess;

#else

    ssize_t excess;

    int fid;  /*- Unbuffered file identifier -*/

#endif

    int blksiz, loop;

    FILE *mydi;  /*- Buffered file stream -*/

---

#ifndef EMFBUFF

    fid = fileno(mydi);

#endif

---

#ifdef EMFBUFF

       if (fseek(mydi, position, SEEK_SET))

#else

       if (lseek(fid, position, SEEK_SET) < 0)

#endif

---

#ifdef EMFBUFF

          {  if ((excess = fread(blkloc, 1, blksiz, mydi)) == 0)

#else

          {  if ((excess = read(fid, blkloc, (size_t)blksiz)) <= 0)

#endif

---

#ifdef EMFBUFF

          {  excess = fwrite(blkloc, 1, blksiz, mydi);

             if (excess == 0)

#else

          {  excess = write(fid, blkloc, (size_t)blksiz);

             if (excess <= 0)

#endif

---

#ifdef EMFBUFF

             if (fflush(mydi))

             {  setError(curDev);

                   return;

             }

#else

             Fsync(mydi);

#endif

---

#ifdef EMFBUFF

    clearerr(mydi);  /*- Clear all errors -*/

    blklen = ftell(mydi) - initpos;

#else

    if (code != 247) Fsync(mydi);

    clearerr(mydi);  /*- Clear all errors -*/

    lseekpos = lseek(fid, 0L, SEEK_CUR);

    if (lseekpos <= 0) blklen = 0;

    else blklen = (long)(lseekpos - initpos);

#endif



===========

I've supplied this if-tested code to give everyone an opportunity to see the transformation.  There are subtle differences in how things like "fread" and "read" operate, "fwrite" and "write" operate, and how seeking and flushing blocks operate.  Buffered I/O still works fine for non-shared files, like standard input and output associated with each individual user.  But shares files require Unbuffered I/O, which appears to force any "buffers" to be maintained at the system-level where everyone has immediate access to any changes.

If you're having troubles with buffered I/O because of simultaneous access by multiple users, then switch to unbuffered I/O.



 
Rate This Article: poor excellent
 
Comments about this article
Use (inline) function or macro to implement comp
writen by: robheus on 2010-04-20 19:15:06
I would code this as inline functions or preprocessor macro's (MyRead, MyWrite, MySeek). Would make coding a lot easier!
RE: Use (inline) function or macro to implement comp written by robheus:

Comment title: * please do not put your response text here