Extract data from multiple files and merge as rows

[fuxin]

Hi all,

I have several directories from 01 to 50 where each has some file with important data (two numbers). These data I'm trying to extract with the following script:

for file in ??
do
 grep "value" $file/*.dat | tail -n 2 | awk '{printf "%16s\t",$6}' >> new.dat
done

Unfortunately it writes all the number after each other separated by a tab, i.e.:

100 101 102 103 104 105 ....

But I want to have the data of each file in a separate row, i.e.:

100 101
102 103
104 105
....

Also I would like to know how to number each line to make it look like:

1 100 101
2 102 103
3 104 105
....

Maybe the $file variable could be used but I don't know how.

Thanks in advance!

[tpl]

with awk's "printf" you add the newlines yourself;
you might try "print" instead

number the lines with "nl"

[fuxin]

Thanks tpl!

"nl" works fine, I didn't know it before.

About the new line with printf I have to add that the "grep" command finds always four matches for "value" per file. By using "tail -n 2" I get the lowest two. So if I put "\n" for new line I will get a new line after each value. But I want a new line per each file. So that two values from one file will be in one row.

I have tried "print" instead of "printf". The result is that each value is in a new line.

Is it possible to put a new line only once per file?

[tpl]

suppose "new" is the file containing 100 101 102 103 104 105
where the numbers are separated by tabs--then:

sed 's/\t/\n/g' <new >A
sed -n '1~2p' <A >odd
sed -n '2~2p' <A >even
paste odd even >result

should give file "result"

100 101
102 103
104 105

[drl]

Hi.

The command xargs can read a specified number of strings from a line:

#!/usr/bin/env bash

# @(#) s1	Demonstrate extracting pairs of strings, xargs.

# Infrastructure details, environment, commands for forum posts. 
set +o nounset
LC_ALL=C ; LANG=C ; export LC_ALL LANG
echo ; echo "Environment: LC_ALL = $LC_ALL, LANG = $LANG"
echo "(Versions displayed with local utility \"version\")"
c=$( ps | grep $$ | awk '{print $NF}' )
version >/dev/null 2>&1 && s=$(_eat $0 $1) || s=""
[ "$c" = "$s" ] && p="$s" || p="$c"
version >/dev/null 2>&1 && version "=o" $p seq xargs nl edges
set -o nounset
echo

# Create data file if not present.

FILE=${1-data1}
[ ! -f $FILE ] && seq -s "	" 101 1 200 > $FILE

l=$( wc -l < $FILE )
w=$( wc -w < $FILE )
echo " Sample of data file $FILE, $w words in $l lines:"
cut -f1-8  $FILE

echo
echo " Sample of results:"
xargs -a $FILE -d "\t" -n 2 |
grep -v '^$' |
nl |
edges -n -l2

exit 0

producing:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0 
GNU bash 3.2.39
seq (GNU coreutils) 6.10
xargs (GNU findutils) 4.4.0
nl (GNU coreutils) 6.10
edges (local) 307

 Sample of data file data1, 100 words in 1 lines:
101	102	103	104	105	106	107	108

 Sample of results:
     1	101 102
     2	103 104
   ...
    49	197 198
    50	199 200

See man pages for details ... cheers, drl