Need help with sed to escape special characters

[majose]

Hello Everyone,

I need to read an encrypted password from the user and update that value in an xml file. I am trying to use "sed" for searching the appropriate tag and replacing this new value that get from the user. Since the encrypted password can contain special characters(like /,\,&,etc), the "sed" command is coming garbled. Could somebody please help to find an efficient solution for this problem?

Is there a way to instruct sed command not to consider any special characters in the replacement string?

Something like s/${key}/"what ever string"/

Or

Any efficient way to put escape sequence for the special characters?

I tried something like 's/\|/\\|/g' , 's|\\|\\\\|g' to escape individual special characters. But this is not working for special characters like &,/ etc.... Is it possible to write any generic pattern which can handle any special characters.

Very much appreciate any help

Thanks,
Martin

[histrungalot]

This worked for me

$ cat runScript
#!/bin/bash

# Some crazy string value
str="$%^()|||@"

# In-place replace
sed -ie 's/'"$str"'/whatever/' file

$ cat file
this is
test to see if $%^()|||@ get
replaced.

$ ./runScript

$ cat file
this is
test to see if whatever get
replaced.

[majose]

Thank you so much for spending time on my post.

But I faced the problem mainly with /, \, and & when used inside the replacement string. With your solution also it's failing for any of these characters.

Each of these characters has a different meaning in the sed context,

1) "/" is used as the delimiter,
2) "\" will escape the character preceding it, even the actual delimiter if \ is the last character in the search/replacement pattern,
3) & is used to remember the matched string

Thanks,
Martin

[histrungalot]

I'm getting close with

#!/bin/bash
str='/\$%^&/&()|||@'
echo $str
str=${str//\\/\\\\}
str=${str//\//\\/}
#str=${str//&/\\&}
#str=${str//|/\\|}
echo $str
sed -ie 's/'"$str"'/whatever/' file
$ cat file
this is
test to see if /\$%^&/&()|||@ get
replaced.
$ ./runScript
/\$%^&/&()|||@
\/\\$%^&\/&()|||@
$ cat file
this is
test to see if whatever get
replaced.
$

Beware of the difference between "" and ''. Like in my example if I use " " for the instead of ' ' the variable \ is lost.
And I'm not escaping the & or | because this is working without them for me.
So the question is, when you "read an encrypted password from the user" will the \ be suppressed in the string?

Thoughts.

[drl]

Hi.

Here is a minimal perl script to ignore special characteristics of characters in a source string. The script s5 uses the perl script p5:

#!/usr/bin/env bash

# @(#) s5	Demonstrate string replacement, ignore RE, perl.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C perl 

FILE=${1-data1}

pl " Input data file $FILE:"
cat $FILE

pl " Minimalist perl script:"
cat p5

pl " Results:"
./p5 '!@#$%^&*()_+=-[]{}:".,<>\|' 'Hello, world' $FILE

exit 0

producing:

% ./s5

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
GNU bash 3.2.39
perl 5.10.0

-----
 Input data file data1:
What a nice spring day !

This string: !@#$%^&*()_+=-[]{}:".,<>\| will be replaced.

Now is the time
for all good men
to come to the aid
of their country.

-----
 Minimalist perl script:
#!/usr/bin/env perl

# @(#) p5	Demonstrate string replacement, ignore RE metas (minimal).

use strict;
use warnings;

my ( $old, $new, $global, $line, $debug, $t1, $t2 );
$old    = shift;
$new    = shift;
$global = 0;
$debug  = 0;

while (<>) {
  chomp( $line = $_ );
  print " Replacing :$old: with :$new:\n" if $debug;
  while ( ( $t1 = index( $line, $old ) ) >= 0 ) {
    print " Starting position of :$old: in :$line: is $t1\n"
      if $debug;
    $t2 = length($old);
    print " Length of :$old: is $t2\n" if $debug;
    substr( $line, $t1, $t2, $new );
    last if not $global;
  }
  print "$line\n";
}

exit(0);


-----
 Results:
What a nice spring day !

This string: Hello, world will be replaced.

Now is the time
for all good men
to come to the aid
of their country.

The various man pages related to perl and perldoc, e.g. perldoc -f substr, provide details. (This script was adapted from one that did more checking of arguments, etc.)

Best wishes ... cheers, drl

[histrungalot]

So you have to do it as a string and not use a regex?

[drl]

Hi.

So you have to do it as a string and not use a regex?

Is that not what you are doing when you escape the special characters before you use s/old/new/ ? ... cheers, drl

[histrungalot]

$t1 = index( $line, $old )

Is a string operation.
I was thinking of something like

perl -pi -e 's/SomeCrazyEscapeSequence/cat/' userfile

[drl]

Hi.

...

perl -pi -e 's/SomeCrazyEscapeSequence/cat/' userfile

If SomeCrazyEscapeSequence has no RE metacharacters in it or are all escaped, then what does it become?

       The simplest regex is simply a word, or more generally, a string of
       characters. 
...
       Not all characters can be used 'as is' in a match.  Some characters,
       called metacharacters, are reserved for use in regex notation.  The
       metacharacters are

           {}[]()^$.|*+?\

       A metacharacter can be matched by putting a backslash before it ...

Search and replace is performed using "s/regex/replacement/modifiers".

-- excerpts from man perlrequick

Best wishes ... cheers, drl

[majose]

Thanks a ton histrungalot and drl for the wonderful solutions.

I could not use the perl script as I do not have much exposure to perl.

Here is the script that finally worked for me,

##############################################
read -r password

password=${password//\\/\\\\}
password=${password//|/\\|}
#password=${password/\|/\\|}
password=${password//&/\\&}

quote='"'
password_xml="${password_xml} password=${quote}${password}${quote}"
password_xml="${password_xml}/>"

${SED} 's|<PASS_TAG.*>|'"$password_xml"'|' ${CONFIG_XML} > ${CCONFIG_XML}.$$
/bin/mv -f ${CONFIG_XML}.$$ ${CONFIG_XML}
################################################## ########

Thanks again for your kind help.

Thanks,
Martin