Find the answer to your Linux question:
Results 1 to 10 of 10
All, I have a file which contains the below data . 00085052511050739000 : 00085052511050902016 : 00085052511050944045 :457,487,499,497,490 00085052511051100000 : 00085052511051645012 :210,112,242,236,274 00085052511052739019 :262,268,228 00085052511053934007 :224,121 00085052511054936009 :218,242,311,228 00085052511060244008 :251,267,236 00085052511061342021 ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Jun 2009
    Posts
    3

    awk in shell script


    All, I have a file which contains the below data .

    00085052511050739000 :
    00085052511050902016 :
    00085052511050944045 :457,487,499,497,490
    00085052511051100000 :
    00085052511051645012 :210,112,242,236,274
    00085052511052739019 :262,268,228
    00085052511053934007 :224,121
    00085052511054936009 :218,242,311,228
    00085052511060244008 :251,267,236
    00085052511061342021 :317,278,271,224,231

    When I execute the below command in shell i got desired output, the complete line details
    awk -F\: '{ print $0}' /tmp/DataList.txt

    But when I use the same command inside a shell script I am not getting the exact result . below myshell script

    #!/usr/local/bin/ksh93

    for iList in `awk -F \: '{ print $0}' /tmp/DataList.txt`
    do

    echo $iList

    done

    When I run the script the line is breaking at each space. Any idea?

  2. #2
    Just Joined!
    Join Date
    Apr 2011
    Posts
    8
    You are using : as a field separator, so awk is passing the fields separately to the 'for' loop.
    Any reason why you can't just use

    #!/usr/local/bin/ksh93
    cat /tmp/DataList.txt

    which just outputs the data, as awk is doing when used on the command line?

  3. #3
    Just Joined!
    Join Date
    Jun 2007
    Posts
    17
    first of all, whether in command line or in script, if you use $0 you don't need the "-F \:" -
    secondly, if you want the output to be the same with command line and script, you don't need the for loop.

  4. $spacer_open
    $spacer_close
  5. #4
    Just Joined!
    Join Date
    Sep 2007
    Location
    Silver Spring, MD
    Posts
    95

    Modify 4 loop file

    Quote Originally Posted by ansge24 View Post
    All, I have a file which contains the below data .

    00085052511050739000 :
    00085052511050902016 :
    00085052511050944045 :457,487,499,497,490
    00085052511051100000 :
    00085052511051645012 :210,112,242,236,274
    00085052511052739019 :262,268,228
    00085052511053934007 :224,121
    00085052511054936009 :218,242,311,228
    00085052511060244008 :251,267,236
    00085052511061342021 :317,278,271,224,231

    When I execute the below command in shell i got desired output, the complete line details
    awk -F\: '{ print $0}' /tmp/DataList.txt

    But when I use the same command inside a shell script I am not getting the exact result . below myshell script

    #!/usr/local/bin/ksh93

    for iList in `awk -F \: '{ print $0}' /tmp/DataList.txt`
    do

    echo $iList

    done

    When I run the script the line is breaking at each space. Any idea?
    ====================

    Why not write the program in this way.

    #!/bin/bash

    iList=/tmp/DataList.txt

    awk -F":" '{print $0}' $iList

    I am just curious as to why you are using the for loop to do this type of work.

    There might be something that I am not aware in your discussion.

    Todd

  6. #5
    Just Joined!
    Join Date
    Jun 2009
    Posts
    3
    All,

    My actual intention is to parse the fields and process based on this value . But when i use the cat in for loop the line is breaking at every "space"


    for iList in `cat /tmp/DataList.txt`
    do
    echo $iList
    shiftID=`echo ${iList} | awk -F\:'{print $1}'`
    tmpCList=`echo ${iList} | awk -F\:'{print $2}'`
    processDetails(shiftID,tmpCList);
    done

  7. #6
    Just Joined!
    Join Date
    Sep 2008
    Posts
    22

    Smile

    Quote Originally Posted by ansge24 View Post
    All,

    My actual intention is to parse the fields and process based on this value . But when i use the cat in for loop the line is breaking at every "space"


    for iList in `cat /tmp/DataList.txt`
    do
    echo $iList
    shiftID=`echo ${iList} | awk -F\:'{print $1}'`
    tmpCList=`echo ${iList} | awk -F\:'{print $2}'`
    processDetails(shiftID,tmpCList);
    done
    Hi ansge,
    The reason it doesn't work for, is quite simple: it's just the "for" loop in the shell that splits field in space before it to be processed by awk command (because IFS environment variable is evidently set by default with *spaces/tabs/linefeed/etc.*). So you have to pass every single line - w/o splitting it - to awk command to have the desired output.

    I suggest you to change the source in this way:

    Code:
    cat datalist.txt | while read iList
    do
      echo $iList
      shiftID=`echo ${iList} | awk -F":" '{print $1}'`
      tmpCList=`echo ${iList} | awk -F":" '{print $2}'`
      processDetails(shiftID,tmpCList);
    done
    I'm pretty sure that in this way it will work out.
    In this way I redirect output of 'cat' command in pipe of the "while read" loop. In this way I'm sure the 'read' command reads the whole line (with no space splitting) until EOL. So awk command can process the entire line, as you wish.

    Hope this help.

    Gabo

  8. #7
    Just Joined!
    Join Date
    Sep 2007
    Location
    Silver Spring, MD
    Posts
    95

    Tdsan's awk response

    Quote Originally Posted by ansge24 View Post
    All,

    My actual intention is to parse the fields and process based on this value . But when i use the cat in for loop the line is breaking at every "space"


    for iList in `cat /tmp/DataList.txt`
    do
    echo $iList
    shiftID=`echo ${iList} | awk -F\:'{print $1}'`
    tmpCList=`echo ${iList} | awk -F\:'{print $2}'`
    processDetails(shiftID,tmpCList);
    done
    ====================

    I am getting an error when I run your script

    ./data1.sh: line 11: syntax error near unexpected token `shiftID,tmpCList'
    ./data1.sh: line 11: ` processDetails(shiftID,tmpCList);'

    How do you want the data to look when you run your program, I think that might help me and the rest of the group.

    Tdsan

  9. #8
    Just Joined!
    Join Date
    Sep 2008
    Posts
    22
    Quote Originally Posted by tdsan View Post
    ====================

    I am getting an error when I run your script

    ./data1.sh: line 11: syntax error near unexpected token `shiftID,tmpCList'
    ./data1.sh: line 11: ` processDetails(shiftID,tmpCList);'

    How do you want the data to look when you run your program, I think that might help me and the rest of the group.

    Tdsan
    Sure, "processDetails" is a user function (that ansge24 didn't include in his original post), but shiftID and tmpCList are variable to be passed as parameters in this function, and they must be prefixed by '$', otherwise they are treated as unknown tokens.

    So, change the line to read:

    Code:
     processDetails($shiftID,$tmpCList)
    (ps. cut off also ending semicolon, that is not need in bash script).

    read you next.

  10. #9
    Just Joined!
    Join Date
    Sep 2007
    Location
    Silver Spring, MD
    Posts
    95

    Result of the data

    Quote Originally Posted by gabolander View Post
    Sure, "processDetails" is a user function (that ansge24 didn't include in his original post), but shiftID and tmpCList are variable to be passed as parameters in this function, and they must be prefixed by '$', otherwise they are treated as unknown tokens.

    So, change the line to read:

    Code:
     processDetails($shiftID,$tmpCList)
    (ps. cut off also ending semicolon, that is not need in bash script).

    read you next.
    ======================

    I was more interested in the result of the data and how it looks at the end.

    Can you provide how the data should look when the person has run the script, the end result I should say.

    Sorry for the confusion.

    Todd

  11. #10
    Linux Enthusiast
    Join Date
    Jan 2005
    Location
    Saint Paul, MN
    Posts
    668
    Quote Originally Posted by ansge24 View Post
    All,

    My actual intention is to parse the fields and process based on this value . But when i use the cat in for loop the line is breaking at every "space"


    for iList in `cat /tmp/DataList.txt`
    do
    echo $iList
    shiftID=`echo ${iList} | awk -F\:'{print $1}'`
    tmpCList=`echo ${iList} | awk -F\:'{print $2}'`
    processDetails(shiftID,tmpCList);
    done
    The line "processDetails(shiftID,tmpCList)" does not seem to fit in what you are doing. It is not "ksh" as it would look like:
    Code:
    processDetails $shiftID $tmpCList
    Why have "awk" in the picture at all?
    Code:
    cat /tmp/DataList.txt | while read line; do
        shiftID="${line%:*}"
        tmpCList="${line#*:}"2s
        processDetails ${shiftID},${tmpCList}
    done

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •