Text data grabbed from /dev/ttyS0 - encoding issues

[musther]

Hi,

I have a spectrophotometer attached to a PC via RS232. I've successfully created some scripts to control it and receive data.
The data is dumped to a file one line at a time. When finished, it looks like the attached file.

All the data is there, as is evident when you 'cat' the file. But it's behaving like a binary file. I don't know much about text encoding, so I don't know how to get it into simple ASCII, which will be viewable with a text editor, spreadsheet etc.

The data file is attached.

While I'm asking about this, I also want to be able to split the two columns from the attached file, into two separate files (column 1 into one file, column 2 into another).

Any help would be greatly appreciated!

Thanks

[drl]

Hi.

Looks like you have embedded null bytes and an EOT (the EOT: \004 is near the end). Here's how to fix that:

#!/usr/bin/env bash

# @(#) s1	Demonstrate fix for embedded null bytes.

# Utility functions: print-as-echo, print-line-with-visual-space.
pe() { for i;do printf "%s" "$i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }

FILE=${1-data1}

# Clean original file, if no clean file found.

if [ ! -f $FILE ]
then
  pl " Sample line before for \"data\", type $(file data):"
  od -bc data | head -4
  ./p1 data >$FILE
  pl " Sample line after for $FILE, type $(file $FILE):"
  od -bc $FILE | head -4
  pl " perl code for transformation:"
  cat p1
fi

pl " Sample of $FILE:"
head -5 $FILE

exit 0

which produces:

% ./s1

-----
 Sample line before for "data", type data: data:
0000000 066 060 060 056 060 040 040 060 056 060 061 060 000 012 065 071
          6   0   0   .   0           0   .   0   1   0  \0  \n   5   9
0000020 070 056 060 040 040 060 056 060 061 060 000 012 065 071 066 056
          8   .   0           0   .   0   1   0  \0  \n   5   9   6   .

-----
 Sample line after for data1, type data1: ASCII text:
0000000 066 060 060 056 060 040 040 060 056 060 061 060 012 065 071 070
          6   0   0   .   0           0   .   0   1   0  \n   5   9   8
0000020 056 060 040 040 060 056 060 061 060 012 065 071 066 056 060 040
          .   0           0   .   0   1   0  \n   5   9   6   .   0    

-----
 perl code for transformation:
#!/usr/bin/env perl

# @(#) p1	Demonstrate removal of null bytes.

use warnings;
use strict;

while ( <> ) {
  s/\000//g;
  s/\004//g;
  print;
}

-----
 Sample of data1:
600.0  0.010
598.0  0.010
596.0  0.010
594.0  0.010
592.0  0.010

For the column splitting, you should be able to use command cut, so look at man cut. (It would be easy for me to put that in the perl script or the shell script, but by doing work yourself, you'll learn more.)

Best wishes ... cheers, drl

[musther]

Yes, you're right about the NULs and the EOT, checked it out with og. They're from the serial device.

Thanks for the code, very helpful!

Cheers