awk in shell script

[ansge24]

All, I have a file which contains the below data .

00085052511050739000 :
00085052511050902016 :
00085052511050944045 :457,487,499,497,490
00085052511051100000 :
00085052511051645012 :210,112,242,236,274
00085052511052739019 :262,268,228
00085052511053934007 :224,121
00085052511054936009 :218,242,311,228
00085052511060244008 :251,267,236
00085052511061342021 :317,278,271,224,231

When I execute the below command in shell i got desired output, the complete line details
awk -F\: '{ print $0}' /tmp/DataList.txt

But when I use the same command inside a shell script I am not getting the exact result . below myshell script

#!/usr/local/bin/ksh93

for iList in `awk -F \: '{ print $0}' /tmp/DataList.txt`
do

echo $iList

done

When I run the script the line is breaking at each space. Any idea?

[del2]

You are using : as a field separator, so awk is passing the fields separately to the 'for' loop.
Any reason why you can't just use

#!/usr/local/bin/ksh93
cat /tmp/DataList.txt

which just outputs the data, as awk is doing when used on the command line?

[tueglee]

first of all, whether in command line or in script, if you use $0 you don't need the "-F \:" -
secondly, if you want the output to be the same with command line and script, you don't need the for loop.

[tdsan]

All, I have a file which contains the below data .

00085052511050739000 :
00085052511050902016 :
00085052511050944045 :457,487,499,497,490
00085052511051100000 :
00085052511051645012 :210,112,242,236,274
00085052511052739019 :262,268,228
00085052511053934007 :224,121
00085052511054936009 :218,242,311,228
00085052511060244008 :251,267,236
00085052511061342021 :317,278,271,224,231

When I execute the below command in shell i got desired output, the complete line details
awk -F\: '{ print $0}' /tmp/DataList.txt

But when I use the same command inside a shell script I am not getting the exact result . below myshell script

#!/usr/local/bin/ksh93

for iList in `awk -F \: '{ print $0}' /tmp/DataList.txt`
do

echo $iList

done

When I run the script the line is breaking at each space. Any idea?

====================

Why not write the program in this way.

#!/bin/bash

iList=/tmp/DataList.txt

awk -F":" '{print $0}' $iList

I am just curious as to why you are using the for loop to do this type of work.

There might be something that I am not aware in your discussion.

Todd

[ansge24]

All,

My actual intention is to parse the fields and process based on this value . But when i use the cat in for loop the line is breaking at every "space"


for iList in `cat /tmp/DataList.txt`
do
echo $iList
shiftID=`echo ${iList} | awk -F\:'{print $1}'`
tmpCList=`echo ${iList} | awk -F\:'{print $2}'`
processDetails(shiftID,tmpCList);
done

[gabolander]

All,

My actual intention is to parse the fields and process based on this value . But when i use the cat in for loop the line is breaking at every "space"


for iList in `cat /tmp/DataList.txt`
do
echo $iList
shiftID=`echo ${iList} | awk -F\:'{print $1}'`
tmpCList=`echo ${iList} | awk -F\:'{print $2}'`
processDetails(shiftID,tmpCList);
done

Hi ansge,
The reason it doesn't work for, is quite simple: it's just the "for" loop in the shell that splits field in space before it to be processed by awk command (because IFS environment variable is evidently set by default with *spaces/tabs/linefeed/etc.*). So you have to pass every single line - w/o splitting it - to awk command to have the desired output.

I suggest you to change the source in this way:

cat datalist.txt | while read iList
do
  echo $iList
  shiftID=`echo ${iList} | awk -F":" '{print $1}'`
  tmpCList=`echo ${iList} | awk -F":" '{print $2}'`
  processDetails(shiftID,tmpCList);
done

I'm pretty sure that in this way it will work out.
In this way I redirect output of 'cat' command in pipe of the "while read" loop. In this way I'm sure the 'read' command reads the whole line (with no space splitting) until EOL. So awk command can process the entire line, as you wish.

Hope this help.

Gabo

[tdsan]

All,

My actual intention is to parse the fields and process based on this value . But when i use the cat in for loop the line is breaking at every "space"


for iList in `cat /tmp/DataList.txt`
do
echo $iList
shiftID=`echo ${iList} | awk -F\:'{print $1}'`
tmpCList=`echo ${iList} | awk -F\:'{print $2}'`
processDetails(shiftID,tmpCList);
done

====================

I am getting an error when I run your script

./data1.sh: line 11: syntax error near unexpected token `shiftID,tmpCList'
./data1.sh: line 11: ` processDetails(shiftID,tmpCList);'

How do you want the data to look when you run your program, I think that might help me and the rest of the group.

Tdsan

[gabolander]

====================

I am getting an error when I run your script

./data1.sh: line 11: syntax error near unexpected token `shiftID,tmpCList'
./data1.sh: line 11: ` processDetails(shiftID,tmpCList);'

How do you want the data to look when you run your program, I think that might help me and the rest of the group.

Tdsan

Sure, "processDetails" is a user function (that ansge24 didn't include in his original post), but shiftID and tmpCList are variable to be passed as parameters in this function, and they must be prefixed by '$', otherwise they are treated as unknown tokens.

So, change the line to read:

 processDetails($shiftID,$tmpCList)

(ps. cut off also ending semicolon, that is not need in bash script).

read you next.

[tdsan]

Sure, "processDetails" is a user function (that ansge24 didn't include in his original post), but shiftID and tmpCList are variable to be passed as parameters in this function, and they must be prefixed by '$', otherwise they are treated as unknown tokens.

So, change the line to read:

 processDetails($shiftID,$tmpCList)

(ps. cut off also ending semicolon, that is not need in bash script).

read you next.

======================

I was more interested in the result of the data and how it looks at the end.

Can you provide how the data should look when the person has run the script, the end result I should say.

Sorry for the confusion.

Todd

[alf55]

All,

My actual intention is to parse the fields and process based on this value . But when i use the cat in for loop the line is breaking at every "space"


for iList in `cat /tmp/DataList.txt`
do
echo $iList
shiftID=`echo ${iList} | awk -F\:'{print $1}'`
tmpCList=`echo ${iList} | awk -F\:'{print $2}'`
processDetails(shiftID,tmpCList);
done

The line "processDetails(shiftID,tmpCList)" does not seem to fit in what you are doing. It is not "ksh" as it would look like:

processDetails $shiftID $tmpCList

Why have "awk" in the picture at all?

cat /tmp/DataList.txt | while read line; do
    shiftID="${line%:*}"
    tmpCList="${line#*:}"2s
    processDetails ${shiftID},${tmpCList}
done