extracting the particular line on the file

[yat]

I have this kind of file which is the same form the simulation (i just changed the name according to the configuration:

file: msg100x100.txt

Message stats for scenario My scenario
sim_time: 10000.0000
created: 337
started: 327
relayed: 326
aborted: 1
dropped: 82
removed: 0
delivered: 326
delivery_prob: 0.9674
response_prob: 0.0000
overhead_ratio: 0.0000
latency_avg: 121.6696
latency_med: 7.9000
hopcount_avg: 1.0000
hopcount_med: 1
buffertime_avg: 5626.9841
buffertime_med: 5490.0000
rtt_avg: NaN
rtt_med: NaN


file: msg200x200.txt

Message stats for scenario My scenario
sim_time: 10000.0000
created: 337
started: 329
relayed: 329
aborted: 0
dropped: 80
removed: 0
delivered: 329
delivery_prob: 0.9763
response_prob: 0.0000
overhead_ratio: 0.0000
latency_avg: 162.3188
latency_med: 28.2000
hopcount_avg: 1.0000
hopcount_med: 1
buffertime_avg: 5631.9963
buffertime_med: 5516.0000
rtt_avg: NaN
rtt_med: NaN

i have until msgN00xN00.txt file

i just interested in let say, delivery_prob, so that the new file will become like this:

delivery_prob.txt

0.9674
0.9763
.
.
.
.

thx in advanced

[Roxoff]

You're doing this from the command line, probably Bash, is that right?

You can use the 'grep' command (check it's man page) to lift particular lines out of one file, and you can use bash's '>>' invocation to append that text to another file.

[yat]

i want only the numbers, not the string...

yup, i'm doing it from bash command....i don;t klnow, grep seems not that powerful, maybe some awk script...

[atreyu]

Yeah, I'd use awk (and loops and arrays, oh my!). here's my stab:

#!/bin/bash

# store the msg files found in the cwd in an array
declare -a msgFiles
msgFiles=($(find . -type f -name 'msg[0-9]*x[0-9]*.txt'))
echo "Found ${#msgFiles[*]} msg file(s)"

# loop thru all the msg files found
for file in ${msgFiles[*]}; do
  printf "Processing file \`$file' ... "

  # get a list of all fields in the file (i.e., the first column)
  fields=$(awk '$1 ~ /:$/{print $1}' $file|sed -e 's|:$||')

  # loop thru all fields
  for field in $fields; do

    # generate the text file name that corresponds to the field name
    txtFile="${field}.txt"

    # get the value of the field from the file
    value=$(awk "\$1 ~ /^$field:/{print \$2}" $file)

    # append the value on a new line to the text file
    echo $value >> $txtFile
  done
  echo "done"
done

It will look in the current working dir for the msg*.txt files.

[Irithori]

sed 's/^delivery_prob: \(.*\)$/\1/' msg?00x?00.txt > delivery_prob.txt

[Irithori]

Ah sorry. I should test before post

sed -n 's/^delivery_prob: \(.*\)$/\1/p' msg?00x?00.txt > delivery_prob.txt

[drl]

Hi.

With a short pipeline using GNU grep:

#!/usr/bin/env bash

# @(#) s1	Demonstrate two-step grep.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C grep

# For production:
# FILES="msg?00x?00.txt"

# For testing:
FILES="data*"

KEY="delivery_prob"
pl " Results:"
grep "$KEY" $FILES |
grep -o "[0-9][0-9]*\.[0-9][0-9]*"

exit 0

producing:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
GNU bash 3.2.39
GNU grep 2.5.3

-----
 Results:
0.96
0.97

See man grep for details. Best wishes ... cheers, drl

[yat]

Ah sorry. I should test before post

sed -n 's/^delivery_prob: \(.*\)$/\1/p' msg?00x?00.txt > delivery_prob.txt

thx, i use sed (grep seems impossible) and its work...err, mind telling me the meaning of the syntax:

i know it will search the beginning of 'delivery_prob' string, but i lost on \(.*\)$/\1/p, if it's not too troublesome for you to explain..

[Irithori]

It should be possible with grep and also with only one call, as grep can also use regex, but I am more familiar with sed.

Short explanation:
- -n tells sed to be quiet, ie: not print the whole pattern space
- the single quotes protect the regex against expansion by the shell
- the regex itself is a substitute, like this: s/foo/bar/ == replace foo with bar
- the p in the end tells sed to print the result. This is needed, because otherwise we wouldnt have any output due to the -n
- msg?00x?00.txt and > delivery_prob.txt are just shell globbing and redirection

Now for the regex:
- It matches lines, that begin "^" with "delivery_prob: "
- then a group "()" contains simply the rest of the line ".*" until the end "$"
- on the right side of the substitution, this group is then referenced and used with \1

As for the "\(" and "\)"
This is a bit confusing.
By default, gnu sed works in basic regex mode.
Which means, that a "(" or ")" would be taken as a literal "(" or ")".
To indicate a group, the \ needs to be used.

Alternatively -and probably more clear to read- is to set extended regex mode with -r:

sed -rn 's/^delivery_prob: (.*)$/\1/p' msg?00x?00.txt > delivery_prob.txt

Then the behaviour is just the opposite: Parenthesis are interpreted as group definitions by default
and if you would want the literal sign, you would need to escape that.


Additional note:
For closer matching and input verification purposes, it might be good to replace the "Match everything until the end of the line"
with a character class, that matches only numbers (with or without decimal point).
But this is left as an exercise to the reader