Intermittent FIFO Pipe Failures

[djl]

The script below succeeds 100% when run in a directory on a locally-mounted disk volume, but exhibits a failure rate between .05% and .5% when run in a directory mounted on any of our NFS volumes.

It appears the mkfifo pipe intermittently behaves like a file, in that when the script backgrounds the write, the subsequent read receives no data.

#!/bin/ksh

failcount=0
nreps=1000

awk 'BEGIN{for(i=0;i<1000;++i){print i}}' >datafile

docat="cat datafile"
for i in {1 .. 100} ; do
	docat="$docat && cat datafile"
done

for reps in {1..$nreps} ; do
	mkfifo mypipe
	eval "$docat" | awk '{print}' > mypipe &
	cat mypipe >myfile
	[[ -z $(head myfile) ]] && failcount=$(( $failcount + 1 ))
	echo "failrate ($failcount/$reps)" >status
	rm -f mypipe myfile
done

cat status
rm -f status datafile

We have a hunch this is related to how NFS is configured (e.g. for syncronous vs asynchronous writes), and we're interested to know if anyone else can reproduce the same behavior. Any additional insight would also be greatly appreciated.

Note: It is trivial to make these failures go away by modifying the script. The point is not to make the script work, but to find out why the failures are occurring in the first place, as we have other scripts which are exhibiting similar behavior (although far less frequently).

Version info:

$ uname -a
Linux myhost.mydomain.com 2.6.18-194.32.1.el5 #1 SMP Mon Dec 20 10:52:42 EST 2010 x86_64 x86_64 x86_64 GNU/Linux

$ /bin/ksh
$ echo ${.sh.version}
Version M 93s+ 2008-01-31

$/bin/awk --version
GNU Awk 3.1.5

$ /usr/bin/mkfifo --version
mkfifo (GNU coreutils) 5.97

$ yum --version nfs
3.2.22

[scm]

Would you need a small delay before you cat the fifo to allow the data to flow into the remote pipe? If the cat starts while the pipe is empty it may think it's terminated.