Removing lines from text file with Bash

[chuckp123]

I have a bunch of text files containing several lines from which I need to remove a section of lines. The section which needs to be removed always start with a line containing only 'CDirPathAttr' and ending with the line '<<- HomeDir'. The lines are always concurrent.

I am attempting to do this by writing a Bash script which cycles through the file line by line. Once it hits the line with 'CDirPathAttr', it sets a flag to 1. As long as the flag is 1, each following line is deleted, until we hit the line '<<- HomeDir', in which case the flag is set back to zero.

Here is the script which I have come up with. I have been working on it for a while, and cannot get it to work. Can anyone tell where I am going wrong?

#!/bin/bash

for f in $(ls /path/to/dir) ; do
        flag=0
        cat $f | while read line
        do
                if [ '$line' = "CDirPathAttr" ] ; then
                        flag=1
                fi

                if [ $flag -eq 1 ] ; then
                        #sed -i -r 's/$line//'
                        echo $line #test that correct lines are selected
                fi

                if [ '$line' = "<<- HomeDir" ] ; then
                        flag=0
                fi

        done
done

[Kieren]

Try:

#!/bin/bash

for f in $(ls /path/to/dir) ; do
        flag=0
        cat $f | while read line
        do
                if [[ $line = "CDirPathAttr" ]]; then
                        flag=1
                fi

                if [ $flag -eq 1 ] ; then
                        #sed -i -r 's/$line//'
                        echo $line #test that correct lines are selected
                fi

                if [[ $line = "<<- HomeDir" ]] ; then
                        flag=0
                fi

        done
done

You almost got there. The lines in bold are the ones I changed

[chuckp123]

Sorry for the delayed response. Adding the double brackets appears to have worked. Thank you!

However, I am now having an additional problem. The lines that I am pulling from the file appear to have some sort of invisible character at the end of them which kind of ruin my if statements. No problem though, I figured I would just add a wildcard to the end of the line. However, I am also finding that wildcards do not work within quotes. This is not a problem for the first if statement, since there are no spaces or weird characters:

if [[ $line = CDirPathAttr* ]]

However, the second if statement is a different story. If I remove the quotes, the if statement obviously won't work because of the spaces/special characters. What would be the proper way to execute this statement?

if [[ $line = "<<- HomeDir*" ]]

[Kieren]

You can escape the special characters/spaces with back-slashes:

if [[ $line = \<\<\-\ HomeDir* ]]

[chuckp123]

You rock! Thanks for the help!

[KenJackson]

You could also let sed do more of the work.

#!/bin/bash
sed -i '/^CDirPathAttr/,/^<<- HomeDir/d' $(ls /path/to/dir)

BTW, that extra control char at the end of the line might be a carriage-return (^M or '\r') which DOS puts before every newline. If you want to convert the file to a Linux file, you can use dos2unix.

I left the $ out of the addresses so it will work with or without the control char. But it will also match longer lines, if any, like "CDirPathAttrDifferent".