Shell programming - Child process and stdin

[szoftveres]

Hi All,

During development of a simple program, I've faced an issue that can be represented through this example:

Suppose a simple text file:

# ===============
bash
echo level 1 create
echo $$    # This prints the PID of the current shell
echo

bash
echo level 2 create
echo $$
echo

bash
echo level 3 create
echo $$
echo

# ===============

echo level 3 destroy
echo $$
echo
exit

echo level 2 destroy
echo $$
echo
exit

echo level 1 destroy
echo $$
echo
exit

# ===============

After redirecting this file into a shell, the following happens:

$ bash < myfile.txt
level 1 create
7396

level 2 create
7397

level 3 create
7398

level 3 destroy

Actually the last child shell exits, and afterwards, nothing happens. I expect the parent shells also write their things at the corresponding destroy sections.
Why don't they continue reading their standard inputs? If I enter the contents of the text file directly from terminal, everything just works fine. What is the difference?

Thanks in advance for your answers!

[tetsujin]

Why don't they continue reading their standard inputs? If I enter the contents of the text file directly from terminal, everything just works fine. What is the difference?

I'm not entirely sure. The basic difference is that when you type the commands into the shell, STDIN (file descriptor 0) is your TTY - but if you put the commands in a file and redirect that file to be the shell's STDIN, then STDIN for that process is a file on the filesystem. The child shell processes then inherit that file descriptor - and when the first one terminates, it closes STDIN, meaning it closes the file. IIRC the C standard library doesn't close STDIN on exit if STDIN is a TTY.

What's confusing me is that when you exec() a new process, the copy of the file descriptor obtained by the new process should be independent of the one in the parent process... All the child processes should have their own copy of the FD with their own read position within the source file, and closing the file in one process shouldn't affect the others... right?

As a related test, I have a program that reads in a single line from a file:

#include <iostream>

int main() {
   std::string line;
   getline(std::cin, line);
   std::cout << line << std::endl;
}

I have another program that prints the result of ftell(stdin):

#include <stdio.h>

int main() {
   printf("%d\n", ftell(stdin));
}

So I tried this:

$ exec 5<./test.sh
$ ./getline <&5
echo "1 -> $$"
$ read x <&5; echo $x

$ ./ftell_stdin <&5
109

So for some reason, some programs are seeking to the end of STDIN on exit when STDIN is not a TTY. Others are more well-behaved:

$ exec 5<./test.sh
$ head -1 <&5
echo "1 -> $$"
$ ./ftell_stdin <&5
15

It appears that bash is doing this C Stdlib treatment of STDIN, seeking to the end on exit. I also tested Korn Shell and it doesn't do this.

(EDIT): Ugh, I am thick. I think I just figured it out:

C Standard library does buffering of I/O, and some of this is contingent upon whether the file descriptors for standard I/O are TTY's or something else (files, pipes, network sockets, etc.) So what is likely happening in these "ill-behaved" programs is that when STDIN is a file, the programs are using the standard C buffering, which means that on the first call to read data, it pulls in as much as it can to fill its input buffer. The buffer's size is probably measured in kilobytes, meaning that it gobbles up the entire 116 byte test file of mine with no effort. Thus, when the program terminates, the file descriptor points to the end of the file ('cause the whole thing was read in by C Stdlib in that child process). If you had a much larger input file, you'd probably see the file position jump up to some multiple of 1024:

$ find .. > ./files
$ ls -l ./files
-rw-r--r-- 1 tetsujin tetsujin 7754681 Jul 11 19:20 files
$ exec 5<./files
$  ./getline <&5
..
$ ./ftell_stdin <&5
4096

Sure enough...

So yeah, this is arguably an oversight in the implementation of Bash - it (arguably) shouldn't be slurping up 4kiB of input when its input is a file and it reads a line of input... And ksh's behavior of making sure the file pointer corresponds to the position after the last command it executed from the input is (arguably) better. (Though I think one could also make the argument that when you hand a file descriptor to a program, you shouldn't necessarily assume you know what you're gonna get back...) If you really need this to behave the way you expect, ksh might be a good way to go (I tend to prefer the "mksh" version when I use Korn Shell)

(EDIT): Also, there's another alternative: turn that input file into an input pipe:

$ cat ./test.sh | bash    #instead of "bash < ./test.sh"
1 -> 26222
2 -> 26223
3 -> 26224
26224
26223
26222

Why does this work? C Stdlib sees that STDIN is a pipe, rather than a file, and makes the decision to not read more than what the caller requests. It's kind of an annoying irregularity in C Stdlib's behavior, honestly...