First script queries

[Hatrick]

Hi,

I am old in years, barely toddling in Linux and an infant muling and puking in his mother's arms as far as scripting is concerned. This is my first: it has taken 10 days, with excursions into cat, cut, echo, grep, ls, sed, sort and tr, and visits to many,many web pages to reach fruition, so please restrain your mirth.

The task set was to extract essential information, e.g. principal ingredient or type of meal, from several hundred recipes having names like 'A wonderful family recipe from XXX, handed down by her great grandmother etc, ad nauseam', then copy them into a classified folder. I know this information could have been extracted easily on the fly just using 'find', but the purpose of the exercise was principally to start to familiarise myself with Linux and its' tools, while appearing to do something useful.

It works as intended but I have several questions about the script which, I hope, someone with expertise and the ability to explain things simply will be prepared to answer.

#!/bin/bash
# This is intended to extract essential info from unusually named files, to classify it and copy to a new #location.
#
for i in beef cabbage cheese pie fish
do
if
[ ! -e /mnt/sdb7/tmp2/$i ]; then
mkdir /mnt/sdb7/tmp2/$i
fi
find /mnt/sdb7/recipes -iname *$i* -type f -exec cp -fiuRt /mnt/sdb7/tmp2/$i '{}' +
done

1) Is there a limit to the number of items allowed in the 'i' list? There are obviously many more categories to be included.

2)There is no folder called recipes; it is Recipes. Can I, therefore, infer that '-iname' applies equally to the source folder and its' contents?

3)In my limited experience with 'find' I have always found it necessary to quote the wildcards - in this case *$i* - but the script doesn't work if I do it here. Why?

4)I don't understand what is happening in the latter part of the 'find' command -after 'fiuRt'. I visualised it as the output of 'find' being held momentarily in '{}' before being copied to its' final destination but, if that were the case, it would be more logical for it to follow 'type f' or, perhaps, 'fiuRt', so I'm obviously wrong. Kate (bless her lovely colours) prints it in red, but it doesn't work anywhere other than its' present position. Can anyone clarify, please?

5)Is the omission of the semi-colon from the end of the 'find' command acceptable simply because no other command follows?

6)Finally, is the code as 'tight' as it could be - within the limits of my experience? If not, how could it be improved?

Thank you for any help you can offer. Spelling, by the way, is British.

[wje_lf]

Quoth Hatrick:

please restrain your mirth.

We have, apparently, restrained more than our mirth. It's been 14 hours.

Oh. And. In the future, would you be so kind as to go through these steps when copying and pasting code to where we can see it?

1. Do the copy and paste.
2. Highlight the code with your mouse.
3. Click the octothorpe (you know, the #).

That makes it easier to read, like this:

#!/bin/bash

exit

But I digress. Harrumph.

for i in beef cabbage cheese pie fish

You didn't ask this, but you may wish to avoid single-letter variable names. Why? Well, suppose you wish to search your script for each occurrence of that name. You're going to stop at all these other places as well:

for i in beef cabbage cheese pie fish
do
if

und so weiter.

1) Is there a limit to the number of items allowed in the 'i' list? There are obviously many more categories to be included.

I hope strawberry jam is one of the categories. I met an Irish lass once who loved her spuds. I picked her up at the side of the road and we went to Yosemite for the day. I told her that I enjoyed potatoes in a way that she had probably never imagined. I was correct: it was to bake a potato, split it open, mash the inside, and add (while the potato was still steaming) cold strawberry jam.

Again I digress. Harrumph. Let's try to answer the question. Play with this script a little. It experiments with different limits to the number of items you can plug into, uh, i:

#!/bin/bash

for limit in 10 20 30 100 200 300 1000 2000 3000
do
  echo '#!/bin/bash' > test1.sh
  echo 'for iterator in \' >> test1.sh

  for(( iterator=1; $iterator <= $limit; iterator++ ))
  do
    echo -n " $iterator" >> test1.sh
  done

  echo                    >> test1.sh
  echo 'do'               >> test1.sh
  echo '  echo $iterator' >> test1.sh
  echo 'done'             >> test1.sh

  chmod 700 test1.sh
  ./test1.sh
done

2)There is no folder called recipes; it is Recipes. Can I, therefore, infer that '-iname' applies equally to the source folder and its' contents?

No. You may infer that the find command won't work properly. Try this:

wally:~/monday/1$ ls
1.sh  t1  test1.sh
wally:~/monday/1$ mkdir Recipes
wally:~/monday/1$ touch Recipes/a1
wally:~/monday/1$ touch Recipes/A2
wally:~/monday/1$ touch Recipes/b3
wally:~/monday/1$ touch Recipes/B4
wally:~/monday/1$ find recipes -iname 'a*'
find: recipes: No such file or directory
wally:~/monday/1$

If the command works for you, then there is something going on in your environment that we haven't heard about yet.

3)In my limited experience with 'find' I have always found it necessary to quote the wildcards - in this case *$i* - but the script doesn't work if I do it here. Why?

It depends on whether you use single or double quotation marks. Single quotion marks won't substitute the value of $i; it will use, literally, $i.

Kate (bless her lovely colours) prints it in red

Well, I'm quite satisfied with Susan's lovely blush, and I'll leave you to admire Kate's. If you're as mature as you claim, you'd fall in love with . But I have no idea how to tackle the overall question. Could you please shift into a lower gear and take this hill again?

Is the omission of the semi-colon from the end of the 'find' command acceptable simply because no other command follows?

I'm not sure that it's ever acceptable if there's a -exec option to the find command. Consider:

wally:~/monday/1$ find . -exec echo {}
find: missing argument to `-exec'
wally:~/monday/1$ find . -exec echo {} \;
.
./1.sh
./test1.sh
./t1
./Recipes
./Recipes/a1
./Recipes/A2
./Recipes/b3
./Recipes/B4
wally:~/monday/1$

6)Finally, is the code as 'tight' as it could be - within the limits of my experience? If not, how could it be improved?

I could tell you, but then I'd have to kill you.

The script looks pretty good to me. I'd put double quotion marks around the *$i*, though.

Spelling, by the way, is British.

Um, no, its' not.

[Hatrick]

Thanks Bill. The formatting tip will, hopefully, be adopted in future. Octothorpe indeed. I think Mr MacPherson was underemployed if he had time to coin that. I'll stick to hash.

Point taken about single letter variables.

Should never have asked the question about the number of permitted variables; too easy to test. But the format of the piece of code you provided is not something I have encountered on my journeys through Bash scripting tutorials, so I shall play.

Still struggling with single and double quotes, backticks, braces and what have you, but will get it sorted eventually. I think the core of the problem is lack of understanding about shells, but the 'type' command may help here.

The query regarding the semi-colon was just curiosity. Bash is so precise in its' demands, which I am trying to fulfill, that I feel offended when it lets me get away with something although, clearly, if nothing follows, the command has ended.

Two weeks ago I thought that the keyboard was a typically modern, over elaborate crumb catcher, and with, statistically, about 3 more years before the clock stops ticking, I don't have time to waste learning to type. Vi, I merely stared at, Nano was a little better but, with kate, I can see what's happening. And lots of lovely pointing and clicking . Unfortunately, root doesn't have direct access but, presumably, I can do my thing as user and chown it afterwards.

Many hours of googling and ploughing through man pages have not managed to clarify, to me, precisely what is happening in the exec command with regard to {}. I'll look again and, if unsuccessful, will try to reformulate the question to post.

Thanks again.

Tony.

[wje_lf]

Quoth Hatrick:

I shall play.

Those are the three most important words in the open source world.

Please remember also that the following words of Isaac Asimov, speaking of scientific discovery, can also be used by the careful explorer of open source:

The most exciting phrase to hear in science, the one that heralds new discoveries, is not 'Eureka!' but 'That's funny ...'

Quoth Hatrick:

single and double quotes

This script might clarify these a little:

#!/bin/bash

set -o verbose

# Remove all files whose names begin with wje.

rm -f wje*

touch wjeaaafred1
touch wjeaaafred2
touch wjebbbfred1
touch wjebbbfred2

ls -l wje*

# Avoid one-letter variable names.

eye=fred

echo eye
echo $eye

echo wjeaaa$eye*
echo "wjeaaa$eye*"
echo 'wjeaaa$eye*'

Bash is so precise in its' demands

True.

I feel offended when it lets me get away with something

If you sense that it's letting you get away with something, there's a good chance that all is not as it seems. You should say "That's funny", and get thee to not a nunnery but a linuxforums programming thread, preferably with details and simple scripts which demonstrate the problem.

lots of lovely pointing and clicking

Keyboard versus mouse is a religious issue (<--- click there).

[Hatrick]

It's 4 a.m. in California. You must sleep less than I do.

[matonb]

Hatrick,

Best of luck in you endevours, I'm sure all here will help when and where they can.


wje_lf : Your post is most amusing. I frequently use the "That's Funny..." quote, which must not be confused with either "That's not supposed to happen" or "Oops"

[wje_lf]

It's 4 a.m. in California. You must sleep less than I do.

I'm very proud of myself. I'm only 59 and already I have the bladder of a ninety-year-old. Mayhap you can guess what this does for sleep habits.

[Hatrick]

Thanks for your good wishes matonb.

wje-lf. Hope you got my message.

Looking back over one of the pieces of code you contributed earlier I notice that you created a directory with mkdir then used 'touch' for the sub- directories. Any reason?
- mkdir works just as well.

Tony.

[wje_lf]

mkdir creates a directory. touch creates a regular data file with zero length if that file does not exist already.

[Cabhan]

Aha, wje_lf missed one of your questions! O how the might have fallen! Bwahahahahahaha.

I digress.

4)I don't understand what is happening in the latter part of the 'find' command -after 'fiuRt'. I visualised it as the output of 'find' being held momentarily in '{}' before being copied to its' final destination but, if that were the case, it would be more logical for it to follow 'type f' or, perhaps, 'fiuRt', so I'm obviously wrong. Kate (bless her lovely colours) prints it in red, but it doesn't work anywhere other than its' present position. Can anyone clarify, please?

The '{}' syntax is specific to the -exec option, and does exactly what you think: that is, it stores the value of each file found by find. From find's man page:

       -exec command ;
              Execute command; true if 0 status is  returned.   All  following
              arguments to find are taken to be arguments to the command until
              an argument consisting of `;' is encountered.  The  string  `{}'
              is  replaced by the current file name being processed everywhere
              it occurs in the arguments to the command, not just in arguments
              where  it  is alone, as in some versions of find.  Both of these
              constructions might need to be escaped (with a `\') or quoted to
              protect them from expansion by the shell.  See the EXAMPLES sec-
              tion for examples of the use of the `-exec' option.  The  speci-
              fied  command is run once for each matched file.  The command is
              executed in the  starting  directory.    There  are  unavoidable
              security  problems  surrounding  use  of  the  -exec option; you
              should use the -execdir option instead.

Even if it were usable outside of -exec, '-type f' tells find to only report actual files, not directories, fifos, links, etc. You don't need to test anything yourself, as find will only report things that match that test.


As for the quoting and pattern-matching, you're in a funky position. Double quotes interpret all variables inside of them and expand all patterns, while single quotes do neither. So you should quote your pattern as:

'*'"$i"'*'

That is, single-*-single-double-$i-double-single-*-single. This will prevent the '*'s from being expanded, but will allow $i to be interpreted.

A great reference for Bash scripting is:
http://www.tldp.org/LDP/abs/html/

Hope that helps.