AWK - challenge for guru

[rapidgeo]

Hi,

I am ok with AWK and use it a bit.

I am trying to solve a problem where i have a file of fixed field and 9 columns but it doesnt have any line breaks - ie the whole file is on one line.

I am trying to write an AWK line to read the file and write out the fixed field 9 columns before creating a line break and doing the same for the next 9 and so on.

I have had two attempts that work for line one but dont continue. Attempt1 is just one line of AWK and Attempt2 is a bit of AWK with Bash Sh and basic 'bc' program to do the math - neither work properly.

any help will be greatly appreciated.

Attempt1:
-------
#!/bin/sh
DATA=FILE1_FIXED.txt

awk ' BEGIN { FIELDWIDTHS = "12 10 10 10 10 7 7 7 7" } { for (i=0; $(1+i) != ""; i=i+9) { print $(1+i), $(2+i), $(3+i), $(4+i), $(5+i), $(6+i), $(7+i), $(8+i), $(9+i) } }' $DATA

Attempt2:
-------
#!/bin/sh
DATA=FILE1_FIXED.txt

I=0
LINE="0"
while [ $LINE != "" ]; do
column1=`echo 1 + $I | bc -l`
column2=`echo 2 + $I | bc -l`
column3=`echo 3 + $I | bc -l`
column4=`echo 4 + $I | bc -l`
column5=`echo 5 + $I | bc -l`
column6=`echo 6 + $I | bc -l`
column7=`echo 7 + $I | bc -l`
column8=`echo 8 + $I | bc -l`
column9=`echo 9 + $I | bc -l`

awk ' BEGIN { FIELDWIDTHS = "12 10 10 10 10 7 7 7 7" } { print $'$column1', $'$column2', $'$column3', $'$column4', $'$column5', $'$column6', $'$column7', $'$column8', $'$column9'} ' $DATA4


let I=I+9
done

[tpl]

welcome to the forum

between each of the fields, what character have you--space, tab, or what?

please give us a sample line about 30 fields long

[rapidgeo]

Thanks TPL,

It is fixed field data - so where the columns contain small numbers it is space delimetered but when there are large numbers the fields are right up against the next column so the only way to separate the columns is by the FIELDWIDTHS = "12 10 10 10 10 7 7 7 7" part of the AWK.

EX:
11711312.00 -3.927695 55.000778 278003 9565594 67.0 -171.5-8888.0-8888.0 11711300.66 -3.925732 54.999115 277818 9565811 67.8 -164.2-8888.0-8888.0 11711275.55 -3.921168 54.995911 277462 9566315 68.0 -150.5-8888.0

you will note that the -8888.0 is right up against -150.5 as the last two columns but both are 7 characters wide.

[tpl]

well, this isn't perfect, but maybe it'll help

awk 'BEGIN { FS = "0-8888.0 " }

{ i = 1
while ( i <= NF ) {
print ($i "0-8888.0")
i++
}
}' <EX

[rapidgeo]

Hi again TPL,

thanks again for your help and attention.

I haven't tried that code yet - sitting in an airport lounge at the moment - but it looks like it is relying on the last column to be the -8888.0 which it isn't always.

The data is geophysical data where that last column of -8888.0 is a flag for a null value in the column.

Thanks again

[drl]

Hi.

A possible solution:

#!/usr/bin/env bash

# @(#) s1   Demonstrate folding of long lines.

# Utility functions: print-as-echo, print-line-with-visual-space.
pe() { for i;do printf "%s" "$i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }

FILE=${1-data1}
pl " Characteristics of datafile $FILE before fold:"
wc $FILE

pl " Characteristics of datafile t1 after  fold:"
fold -w 80 $FILE > t1
wc t1

exit 0

producing:

% ./s1

-----
 Characteristics of datafile data1 before fold:
  1  21 212 data1

-----
 Characteristics of datafile t1 after  fold:
  3  23 214 t1

See man fold for details ... cheers, drl

For this solution:

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0 
GNU bash 3.2.39
printf - is a shell builtin [bash]
fold (GNU coreutils) 6.10

[rapidgeo]

wow - thanks drl!

so easy - i will be using this a hell of a lot so thanks for your assistance.

Simple:

#!/bin/sh
fold -w 80 $DATA > $DATA_FIXED.txt
unix2dos $DATA_FIXED.txt

[drl]

Hi, rapidgeo.

You're welcome. I noticed a small error in the comment naming the output file. It was "data1" and should have been "t1". It was in the comment, the counting code (with wc) was correct. I modified the post to show the correct name ... cheers, drl