probably a simple awk / grep / sed question

[mcbenus]

Hi,

I am relatively new to scripting. I have a directory with many files. All the files have the string 'Score' in the second line, followed by a number (e.g. the actual numeric score). What I want to do is two things:
1) I want to make one file that lists all the file names and the scores sorted from lowest to highest.
2) I want to make one file that has the lowest 1000 files written one after another, sorted from lowest to highest based on the 'score'. All the files end with string 'ROOT', so the end-of-file can be easily identified.

For 1) I have somthing like:

foreach f (ZINC*) # all my filenames start with ZINC
echo $f >>summary.txt # writes into summary.txt the name of the file
echo `awk /Score/'{print $3$4}' $f` >> summary.out # writes into summary.txt the score (field 4)
end

This works, but the problem is that I have a new line between the name and the Score+number, and I want them to be in the same line for each file. I am also not sure yet how to sort it, but I belive it is possible using the sort command.

I just learned about awk and it seems like it is the right command but I am not sure how to use it for that. grep or sed may also work, but I am not familiar with them either.

Thanks for any help!

[khafa]

Hi,

For the new line you have to remember that by default echo appends it.
So you have two choices
1) you can use

 echo -n

or
2) use one echo command instead of two as follows

echo $f  `awk /Score/'{print $3$4}' $f` >> summary.txt

Note: I am assuming here that there is a typo above and you meant summary.txt when you wrote summary.out in the second echo.

For the sorting just sort it numerically by the score_value column.
You can check how to do this with the man pages of echo and sort

[mcbenus]

Thanks, it worked! (and yes, the .out was a typo).

I created the file, and sorted it using the sort command and now I have a list of all the names and the associated scores ranked.

I am trying now to create another file that will have all the content of the original files in the same order as my ranked list.

Something like:
cat (files in order) >> combined_file.txt

where the (files in order) are actually written into a file. Can I use a file containing the names of the files as an input to cat?

it sounds like it should exist, I just can't find the syntax. Thanks for the help!

Hi,

For the new line you have to remember that by default echo appends it.
So you have two choices
1) you can use

 echo -n

or
2) use one echo command instead of two as follows

echo $f  `awk /Score/'{print $3$4}' $f` >> summary.txt

Note: I am assuming here that there is a typo above and you meant summary.txt when you wrote summary.out in the second echo.

For the sorting just sort it numerically by the score_value column.
You can check how to do this with the man pages of echo and sort

[khafa]

Hi,

Unfortunately cat is not that smart AFAIK.

You can do as follows:

while read line
do
   cat `awk '{print $1}'` >> combined_file.txt
done < summary.txt

[mcbenus]

Thanks khafa, it works. thanks for your help.

Hi,

Unfortunately cat is not that smart AFAIK.

You can do as follows:

while read line
do
   cat `awk '{print $1}'` >> combined_file.txt
done < summary.txt

[khafa]

You are very welcome

[ghostdog74]

Hi,

I am relatively new to scripting. I have a directory with many files. All the files have the string 'Score' in the second line, followed by a number (e.g. the actual numeric score). What I want to do is two things:
1) I want to make one file that lists all the file names and the scores sorted from lowest to highest.
2) I want to make one file that has the lowest 1000 files written one after another, sorted from lowest to highest based on the 'score'. All the files end with string 'ROOT', so the end-of-file can be easily identified.

For 1) I have somthing like:

foreach f (ZINC*) # all my filenames start with ZINC
echo $f >>summary.txt # writes into summary.txt the name of the file
echo `awk /Score/'{print $3$4}' $f` >> summary.out # writes into summary.txt the score (field 4)
end

This works, but the problem is that I have a new line between the name and the Score+number, and I want them to be in the same line for each file. I am also not sure yet how to sort it, but I belive it is possible using the sort command.

I just learned about awk and it seems like it is the right command but I am not sure how to use it for that. grep or sed may also work, but I am not familiar with them either.

Thanks for any help!

you can do this with the shell

for file in ZINC*
do
    while read -r a b c d e
    do
       case "$a $b $c $d $e" in
         *Score* ) echo "$c $d" >> summary.txt
       esac        
    done < "$file"
 done

[ghostdog74]

Hi,

Unfortunately cat is not that smart AFAIK.

You can do as follows:

while read line
do
   cat `awk '{print $1}'` >> combined_file.txt
done < summary.txt

this is sloppy. What's with all the backticks , and the useless cat?? don't even need while loop when you use awk

awk '{print $1}' summary.txt >> combined_file.txt

[khafa]

this is sloppy. What's with all the backticks , and the useless cat?? don't even need while loop when you use awk

awk '{print $1}' summary.txt >> combined_file.txt

I think you got the question wrong. The thread owner does not want to only list the file names but rather put the content of those files in one file if I got it right.

[ghostdog74]

I think you got the question wrong. The thread owner does not want to only list the file names but rather put the content of those files in one file if I got it right.

then its just this

awk '{print $1}' ZINC* >> combined_file.txt