[SOLVED] Odd quirk in /bin/cat?

[Miven]

This might seem like a newbie question, but I've been using linux for 15 years and never noticed this quirk of /bin/cat:

I'll explain what I'm trying to do, and then I'll explain the problem.

I have a cute little C application that I'm working on that makes heavy use of a configuration file that may or may not change on the fly. When the program is first run, it creates a default FILE.rc that the user is expected to tweak to his/her liking, even while the program is running.

Now, because the program is still under development, the default FILE.rc keeps changing everytime I make changes to the program. At some point it won't, but I still have many features to implement.

The default FILE.rc is a char* defined in a header, rc.h, compiled in, and then written to disk whenever the FILE.rc doesn't exist/gets deleted. This rc.h is a pain to edit everytime I make changes. You have to make sure you put a \n at the end of every line and quote everything, and other little pains.

So what I've done is create DEFAULT.rc, an ordinary text file that I can tweak/re-format/whatever very easily, and then parse it into the header rc.h everytime I change it. This automagically adds quotes and \n's and other little things so I don't have to.

Here's an example test.rc

# Here's a comment

# Here's another comment after a blank line
# that I want to preserve.

If I use this command:

for line in `cat test.rc`; do echo "-> '$line'"; done

I get this:

-> '#'
-> 'Here's'
-> 'a'
-> 'comment'
-> '#'
-> 'Here's'
-> 'another'
-> 'comment'
-> 'after'
-> 'a'
-> 'blank'
-> 'line'
-> '#'
-> 'that'
-> 'I'
-> 'want'
-> 'to'
-> 'preserve.'

Blank line is missing, and of course it's using IFS=$' \t\n' so it parses everything by word.

If I use the command:

IFS=$'\n'; for line in `cat test.rc`; do echo "-> '$line'"; done

I get something more appropriate:

-> '# Here's a comment'
-> '# Here's another comment after a blank line'
-> '# that I want to preserve.'

But the blank line is still missing. Shouldn't it be:

-> '# Here's a comment'
-> ''
-> '# Here's another comment after a blank line'
-> '# that I want to preserve.'

If I go:

for line in `cat -E test.rc`; do echo "-> '$line'"; done

I get this:

-> '# Here's a comment$'
-> '$'
-> '# Here's another comment after a blank line$'
-> '# that I want to preserve.$'

And I have the blank line. Why is a double newline treated as nothing by default? Should it not be an empty string?

Is this a quirk in my version of /bin/cat (cat (coreutils) 5.1.2), or is this to be expected? Maybe it's an issue with bash? I've never noticed it before. I did dramatically upgrade bash from 2-something to 4.1.0 about a year ago.

Obviously there's hackish workarounds, but I'm just puzzled by this because I seem to recall having to parse blank lines out in situations like this when I didn't want them.

[atreyu]

Hi,

try this while loop, which does not make use of a subshell, but does require a somewhat recent version of Bash:

while read line; do echo "-> '$line'";done < <(cat test.rc)

[Miven]

Wacky. Cool. Thank you very much, atreyu. Deep magic.

I started to think it was bash doing something funny. Why do you need the space in the <_<(cat test.rc)? Odd. Shouldn't it be <<(cat test.rc)? It's not normal redirection? And the second < has to be right beside the bracket? No space. Odd. Very odd. Strangely enough, it works.

Thank you again. I'll use it. Maybe I'll even RTFM

[atreyu]

In Bash, it is known as process substitution, you can read up on it more here.

FYI, if you consider this thread to be solved to your satisfaction, you can mark it so using the Thread Tools link at the top of the page.

[Miven]

Yes. I will do that.

Good link too, I think I'll poke around there for a while, and BTW it has to be #!/bin/bash. #!/bin/sh just doesn't cut the mustard.