Find the answer to your Linux question:
Results 1 to 8 of 8
Hi, I am ok with AWK and use it a bit. I am trying to solve a problem where i have a file of fixed field and 9 columns but ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Sep 2010
    Posts
    4

    AWK - challenge for guru


    Hi,

    I am ok with AWK and use it a bit.

    I am trying to solve a problem where i have a file of fixed field and 9 columns but it doesnt have any line breaks - ie the whole file is on one line.

    I am trying to write an AWK line to read the file and write out the fixed field 9 columns before creating a line break and doing the same for the next 9 and so on.

    I have had two attempts that work for line one but dont continue. Attempt1 is just one line of AWK and Attempt2 is a bit of AWK with Bash Sh and basic 'bc' program to do the math - neither work properly.

    any help will be greatly appreciated.

    Attempt1:
    -------
    #!/bin/sh
    DATA=FILE1_FIXED.txt

    awk ' BEGIN { FIELDWIDTHS = "12 10 10 10 10 7 7 7 7" } { for (i=0; $(1+i) != ""; i=i+9) { print $(1+i), $(2+i), $(3+i), $(4+i), $(5+i), $(6+i), $(7+i), $(8+i), $(9+i) } }' $DATA

    Attempt2:
    -------
    #!/bin/sh
    DATA=FILE1_FIXED.txt

    I=0
    LINE="0"
    while [ $LINE != "" ]; do
    column1=`echo 1 + $I | bc -l`
    column2=`echo 2 + $I | bc -l`
    column3=`echo 3 + $I | bc -l`
    column4=`echo 4 + $I | bc -l`
    column5=`echo 5 + $I | bc -l`
    column6=`echo 6 + $I | bc -l`
    column7=`echo 7 + $I | bc -l`
    column8=`echo 8 + $I | bc -l`
    column9=`echo 9 + $I | bc -l`

    awk ' BEGIN { FIELDWIDTHS = "12 10 10 10 10 7 7 7 7" } { print $'$column1', $'$column2', $'$column3', $'$column4', $'$column5', $'$column6', $'$column7', $'$column8', $'$column9'} ' $DATA4


    let I=I+9
    done

  2. #2
    tpl
    tpl is offline
    Linux User
    Join Date
    Jan 2007
    Location
    cleveland
    Posts
    477
    welcome to the forum

    between each of the fields, what character have you--space, tab, or what?

    please give us a sample line about 30 fields long
    the sun is new every day (heraclitus)

  3. #3
    Just Joined!
    Join Date
    Sep 2010
    Posts
    4
    Thanks TPL,

    It is fixed field data - so where the columns contain small numbers it is space delimetered but when there are large numbers the fields are right up against the next column so the only way to separate the columns is by the FIELDWIDTHS = "12 10 10 10 10 7 7 7 7" part of the AWK.

    EX:
    11711312.00 -3.927695 55.000778 278003 9565594 67.0 -171.5-8888.0-8888.0 11711300.66 -3.925732 54.999115 277818 9565811 67.8 -164.2-8888.0-8888.0 11711275.55 -3.921168 54.995911 277462 9566315 68.0 -150.5-8888.0

    you will note that the -8888.0 is right up against -150.5 as the last two columns but both are 7 characters wide.

  4. #4
    tpl
    tpl is offline
    Linux User
    Join Date
    Jan 2007
    Location
    cleveland
    Posts
    477
    well, this isn't perfect, but maybe it'll help

    awk 'BEGIN { FS = "0-8888.0 " }

    { i = 1
    while ( i <= NF ) {
    print ($i "0-8888.0")
    i++
    }
    }' <EX
    the sun is new every day (heraclitus)

  5. #5
    Just Joined!
    Join Date
    Sep 2010
    Posts
    4
    Hi again TPL,

    thanks again for your help and attention.

    I haven't tried that code yet - sitting in an airport lounge at the moment - but it looks like it is relying on the last column to be the -8888.0 which it isn't always.

    The data is geophysical data where that last column of -8888.0 is a flag for a null value in the column.

    Thanks again

  6. #6
    drl
    drl is offline
    Linux Engineer drl's Avatar
    Join Date
    Apr 2006
    Location
    Saint Paul, MN, USA / CentOS, Debian, Slackware, {Free, Open, Net}BSD, Solaris
    Posts
    1,283
    Hi.

    A possible solution:
    Code:
    #!/usr/bin/env bash
    
    # @(#) s1   Demonstrate folding of long lines.
    
    # Utility functions: print-as-echo, print-line-with-visual-space.
    pe() { for i;do printf "%s" "$i";done; printf "\n"; }
    pl() { pe;pe "-----" ;pe "$*"; }
    
    FILE=${1-data1}
    pl " Characteristics of datafile $FILE before fold:"
    wc $FILE
    
    pl " Characteristics of datafile t1 after  fold:"
    fold -w 80 $FILE > t1
    wc t1
    
    exit 0
    producing:
    Code:
    % ./s1
    
    -----
     Characteristics of datafile data1 before fold:
      1  21 212 data1
    
    -----
     Characteristics of datafile t1 after  fold:
      3  23 214 t1
    See man fold for details ... cheers, drl

    For this solution:
    Code:
    Environment: LC_ALL = C, LANG = C
    (Versions displayed with local utility "version")
    OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
    Distribution        : Debian GNU/Linux 5.0 
    GNU bash 3.2.39
    printf - is a shell builtin [bash]
    fold (GNU coreutils) 6.10
    Last edited by drl; 09-19-2010 at 09:50 AM. Reason: Corrected comment for output file being "t1" rather than "data1"
    Welcome - get the most out of the forum by reading forum basics and guidelines: click here.
    90% of questions can be answered by using man pages, Quick Search, Advanced Search, Google search, Wikipedia.
    We look forward to helping you with the challenge of the other 10%.
    ( Mn, 2.6.n, AMD-64 3000+, ASUS A8V Deluxe, 1 GB, SATA + IDE, Matrox G400 AGP )

  7. #7
    Just Joined!
    Join Date
    Sep 2010
    Posts
    4
    wow - thanks drl!

    so easy - i will be using this a hell of a lot so thanks for your assistance.

    Simple:

    #!/bin/sh
    fold -w 80 $DATA > $DATA_FIXED.txt
    unix2dos $DATA_FIXED.txt

  8. #8
    drl
    drl is offline
    Linux Engineer drl's Avatar
    Join Date
    Apr 2006
    Location
    Saint Paul, MN, USA / CentOS, Debian, Slackware, {Free, Open, Net}BSD, Solaris
    Posts
    1,283
    Hi, rapidgeo.

    You're welcome. I noticed a small error in the comment naming the output file. It was "data1" and should have been "t1". It was in the comment, the counting code (with wc) was correct. I modified the post to show the correct name ... cheers, drl
    Welcome - get the most out of the forum by reading forum basics and guidelines: click here.
    90% of questions can be answered by using man pages, Quick Search, Advanced Search, Google search, Wikipedia.
    We look forward to helping you with the challenge of the other 10%.
    ( Mn, 2.6.n, AMD-64 3000+, ASUS A8V Deluxe, 1 GB, SATA + IDE, Matrox G400 AGP )

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •