Shell expansion and double quotes

[marc_h]

Hello Everyone,

Here is a small shell script written to do various automated tasks. Basically, the "try" function will print the job title and attempt to execute the rest of the arguments as being a command line, exiting if the said command line return an error (non-zero) value:

#!/bin/bash
function prt_status {
echo
echo ": $*"
echo
}

function quit {
prt_status "Non-Zero exit code returned, exiting."
exit 1
}

function try {
prt_status $1
shift
$* || quit
}

try "Job 1" echo Hello World
try "Job 2" ls -l "A file.txt"

If you execute the script you will likely get the following output:

: Job 1

Hello World

: Job 2

ls: cannot access A: No such file or directory
ls: cannot access file.txt: No such file or directory

: Non-Zero exit code returned, exiting

The problem is that shell expansion will actually remove any double quotes.

As a matter of fact, 'ls' will always be called this way:

ls -l A file.txt

Instead of

ls -l "A file.txt"

I tried a lot workaround, using any known escape character, as well as various combination of $* and $(at sign), but nothing seems to work this far...

So the question is... Is there some way to make it work the way i expect?

Thanks
Paul M.

[Franklin52]

Change the IFS, replace your function try with:

function try {
  OIFS=$IFS
  IFS=\n
  prt_status $1
  shift
  $* || quit
  IFS=$OIFS
}

[marc_h]

Thank you for this quick answer Franklin, it seems to works perfectly!

I bumbed into the IFS variable before, but the man page let me think that it was only used when $* is enclosed with double quotes:

When the expansion occurs within double quotes, it expands to a single word with the value of each parameter separated by the first character of the IFS special variable. That is, "$*" is equivalent to "$1c$2c...", where c is the first character of the value of the IFS variable. If IFS is unset, the parameters are separated by spaces. If IFS is null, the parameters are joined without intervening separators.

But it seems that the IFS variable is in fact used when expanding all literals enclosed by double quotes, am i right?

I guess that the important thing here is not to set IFS=\n, but to prevent the shell to use the default space character as being the field separator... still correct?

Alternatively setting IFS to a null string (IFS="") seems also to work...

Is there any negative side effect that we could expect from playing with IFS? Or I could I expect most script to run flawlessly after setting it to a non-default value?

These are details... But thank you again for your help
Paul

[marc_h]

Hum... setting IFS=\n do not seems to work flawlessly after all...

If I replace:

try "Job 2" ls -l "A file.txt"

by:

try "Job 2" ls -l "Another file.txt"

I will get as result:

ls: cannot access A: No such file or directory
ls: cannot access other file.txt: No such file or directory

For some reason, IFS don't seems to get the escaped characters right...

IFS="" still works however, but only when invoking explicitly bash. Invoking sh will result in the following:

: Job 1

test: 28: echoHelloWorld: not found

It looks like bash and sh handle the IFS variable differently...

[jippie]

The trick is pretty much as mentioned above. Just pick a smart field separator and use it in the try command line parameter list.

#!/bin/bash

IFS="|"

function prt_status {
echo
echo ": $*"
echo
}

function quit {
prt_status "Non-Zero exit code returned, exiting."
exit 1
}

function try {
prt_status $1
shift
$* || quit
}

try "Job 1"|echo "Hello World"
try "Job 2"|ls -l "A file"

[achaplot]

Edit your function "try"

function try {
prt_status $1
shift
"$@" || quit
}

--- this will work as $@ is the variable for keeping the arguments as one string.

ankur c

[marc_h]

First of all, thank you for all these feedbacks...

To jippie:

The trick is pretty much as mentioned above. Just pick a smart field separator and use it in the try command line parameter list.

#!/bin/bash

IFS="|"

function prt_status {
echo
echo ": $*"
echo
}

function quit {
prt_status "Non-Zero exit code returned, exiting."
exit 1
}

function try {
prt_status $1
shift
$* || quit
}

try "Job 1"|echo "Hello World"
try "Job 2"|ls -l "A file"

Hum... the pipe character may be problematic as separator...

If I use something else such as:

IFS=_
try "Job 2"_ls -l "A file"

I will actually get the following:

-l: command not found

This is because of the "shift" keyword: it will shift parameters according to the way their were parsed, not on the way they are expanded (and we have only three parameters here, '"Job 2"_ls', '-l' and '"Another file"' ).

To achaplot:

function try {
prt_status $1
shift
"$@" || quit
}

Using "$@" seems indeed to preserve the initial parameters splitting by enclosing every one of them with double quotes... And this works with both bash and sh as interpreter, thanks a lot

Just for the records:
- Concerning the fourth post: we could actually set the IFS to an escape character by prefixing the string with a dollar sign (this could be used to instanciate any escaped character, but this is not supported by every interperter...).
- The difference of behavior i've noticed between bash and sh hold in fact that Dash (my sh interpreter) expand $* as a single field when IFS is null, even if not enclosed within double quotes. The man page say nothing about that, Ksh and Bash (even with the --posix option) will not do this. On the other hand Zsh with the SH_WORD_SPLIT option enabled have this same strange behaviour... (Any insight on the matter?)
- Zsh will not split words by default if there are quoted (which could be, indeed, considered much more intuitive).