grep and sed not working as expected. What's wrong?

**karozans** · 08-12-2011

Hi everyone. I am a noob to linux and I need some help. I am running openSUSE 11.2.

I have a simple text file called "test.txt". Here are the contents.

Code:

asdf
TEST=
TEST="asdf"
TEST= TEST=
asdf

I want to find and replace "TEST=" whole word only.

When I run the command...

Code:

grep -w 'TEST=' test.txt

I get as the output...

Code:

TEST=
TEST="asdf"
TEST= TEST=

When I run the command...

Code:

sed -i -e 's/TEST=/blah=/g' test.txt

I get in the file...

Code:

asdf
blah=
blah="asdf"
blah= blah=
asdf

When I run the command...

Code:

sed -i -e 's/\bTEST=\b/blah=/g' test.txt
or
sed -i -e 's/\<TEST=\>/blah=/g' test.txt

Nothing is changed in the file.

Why doesn't the word boundaries works and the -w for grep?

thanks

**drl** · 08-12-2011

Hi.

1. grep does not do replacement (grep: globally look for regular expression and print),

2. word-constituent characters are letters, digits, and the underscore (excerpt from man grep), which will constrain

Code:

\< \> \b

somewhat.

Consider:

Code:

#!/usr/bin/env bash

# @(#) s1	Demonstrate string substitution.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C sed gres

FILE=${1-data1}

pl " Input data file $FILE:"
cat $FILE

pl " Results, sed, non-global:"
sed 's/TEST=/blah=/' $FILE

pl " Results, sed, global:"
sed 's/TEST=/blah=/g' $FILE

pl " Results, sed, empty string \< \>:"
sed 's/\<TEST\>=/blah=/g' $FILE

pl " Results, sed, empty string \b \b:"
sed 's/\bTEST\b=/blah=/g' $FILE

pl " Results, gres:"
gres 'TEST=' 'blah=' $FILE

exit 0

producing:

Code:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
GNU bash 3.2.39
GNU sed version 4.1.5
gres (local) 1.17

-----
 Input data file data1:
asdf
TEST=
TEST="asdf"
TEST= TEST=
asdf

-----
 Results, sed, non-global:
asdf
blah=
blah="asdf"
blah= TEST=
asdf

-----
 Results, sed, global:
asdf
blah=
blah="asdf"
blah= blah=
asdf

-----
 Results, sed, empty string \< \>:
asdf
blah=
blah="asdf"
blah= blah=
asdf

-----
 Results, sed, empty string \b \b:
asdf
blah=
blah="asdf"
blah= blah=
asdf

-----
 Results, gres:
asdf
blah=
blah="asdf"
blah= blah=
asdf

The sed command was designed to consolidate a number of functions that had been separate commands. One older command, gres, did search and replace (globally look for a regular expression and substitute). It is occasionally handy to have such a work-alike around.

See man pages for details, and experiment, experiment, experiment.

Best wishes ... cheers, drl

**karozans** · 08-12-2011

Hi drl thanks for the reply.

I know that grep doesn't replace. I was demonstrating how the -w doesn't produce the desired results.

What I want is to create a sed command that replaces the whole word only.

So if this is my input file.

Code:

asdf
TEST=
TEST="asdf"
TEST= TEST=
asdf

I want to run a sed command that changes the file to look like this.

Code:

asdf
blah=
TEST="asdf"
blah= blah=
asdf

Notice how the 3rd line is not changed because TEST="asdf" is all a single word?

I want to only change whole words only.

It seems like the /b /< /> don't work like I want them to.

**drl** · 08-12-2011

Hi.

Is the "=" part of the definition of a word?

Look carefully at how your sed calls are different from mine ... cheers, drl

**karozans** · 08-12-2011

In this case yes the "=" is part of the word.

I would like the use of my script to be able to search a file for ANY string to do a find and replace on the whole word only.

**drl** · 08-12-2011

Hi.

Originally Posted by karozans

In this case yes the "=" is part of the word.

I would like the use of my script to be able to search a file for ANY string to do a find and replace on the whole word only.

Yes, I understand that you would like them to be, but they are not in the definition of a word when dealing with regular expressions: 2. word-constituent characters are letters, digits, and the underscore (excerpt from man grep) ...

They can be part of string, but not of a word.

If you look carefully at how I placed them in the demo script, you will see I did not include anything non-alpha-numeric when using

Code:

\< \> \b

.
Best wishes ... cheers, drl

**karozans** · 08-12-2011

Okay after doing some digging it seems as though this might be impossible to do with SED and/or gawk. Even open office and Microsoft word cannot accomplish this.

I would like to be able to do a find and replace on ANY ASCII character word that is a WHOLE word.

For instance what if a user wants to search for...

Code:

()(""%#$'*&!@'

in text that is something like...

Code:

blah blah blah ()(""%#$'*&!@' blah blah blah

and they want this exact whole word.

Basically to me the term "whole word" means a string of any ASCII character allowed that is delimited by a white space.

It doesn't seem like it should be limited to only letters from A-Z and a-z and numbers 0-9

I guess I will have to write a function in c.

Thanks for your help.

**drl** · 08-13-2011

Hi.

I created a perl script to address the issue of substituting general strings without regard to properties of the characters in the context of regular expressions.

This issue not peculiar to SuSE so I posted the solution in thread http://www.linuxforums.org/forum/pro...tml#post856394

The driver script shows the operation of the utility.

Somewhat parenthetically, I should add that this kind of operation could be done with sed and awk, but one would need to escape the characters that have special meaning as regex meta-characters. The perl code seems like an easier solution.

The code itself is longish, although most of it deals with argument processing and modest checks for errors and consistency. The actual heart of the code could be done in a few lines, but I spread things out so that 1) one could see how it operates, and 2) to include a global option like sed has for substitution.

Best wishes ... cheers, drl

**karozans** · 08-16-2011

Hi drl.

I appreciate your help on this.

After checking on this, I found that my linux distro is running BusyBox, that seems to not have GNU SED.

When I run the -r option, there is a complaint about the regex.

Here is a regular expression that I have devised that seems to work on the GNU SED.

Take a look at this and see what you think.

Consider the input file. Only spaces are used for white spaces.

Code:

test=
test="something"
test= test=
      test=
                  test=               test=

Now consider the regex

Code:

test=((?!.)|(?=\s)+)

This seems to work for me. Do you see any problems?

**drl** · 08-16-2011

Hi.

Originally Posted by karozans

Code:

test=((?!.)|(?=\s)+)

This seems to work for me.

If it works for you on samples of your regular data, then you have a solution -- good work.

Best wishes ... cheers, drl

Thread Tools

Display

grep and sed not working as expected. What's wrong?

Posting Permissions