filter the output of a text file using the contents of other text file

[patski]

Hi folks,

I'm trying to reduce a huge text file using contents of a much smaller text file. Here's what I've tried:

cat PIGGY_ALL_REELS.edl | grep `cat list_CLIPS.txt`

Here's a small chunk of the first, huge file:

035 A224C004 V C 18:16:55:22 18:17:02:13 01:01:35:01 01:01:41:17
* FROM CLIP NAME: A224_C004_0724WP.MOV
* COMMENT:
* OPACITY LEVEL AT 18:16:55:22 IS 100.00% (REEL A224C004)
* CLIP FILTER: WIDESCREEN

036 A226C001 V C 19:09:32:07 19:09:41:12 01:01:41:17 01:01:50:22
* FROM CLIP NAME: A226_C001_0724VW.MOV
* COMMENT:
* CLIP FILTER: WIDESCREEN

Here's a small chunk of the second, smaller file:

A123_C001
A125_C001
A127_C001
A128_C002
A130_C002
A132_C001
A134_C004
A135_C002

Any brainy thoughts would be hugely appreciated.

Best,

Patski

[Irithori]

You dont need cat, grep is enough.

grep features the -f flag, look it up in the man page.
You will find it interesting

man grep

[patski]

Ahhhh...that IS interesting. Thanks Irithori!

I'm having a bit of trouble with it however. When I do a grep like this:

cat PIGGY_ALL_REELS.edls | grep -f one_line_of_text.txt

I get the result I'm looking for, however when I do this:

cat PIGGY_ALL_REELS.edls | grep -f many_lines_of_text.txt

it doesn't work.

I'm a bit new to this so any pointers as to what I'm doing wrong would be much appreciated!

Best,

Patski

[Irithori]

Hmm, does the file with the patterns contain windows style line feeds by any chance?
If so, get rid of those with e.g. dos2unix.

Example:

$cat bigfile 
a1a
b2b
c3c
d4d
e5e
f6f

$cat patterns 
1
3
5

$grep -f patterns bigfile 
a1a
c3c
e5e

Edit:
You also want to make sure, that there is no empty line in the patterns file.

[patski]

Hi Irithori,

I don't think so, I created the file using an ls > text.txt command. I'm using bash in Mac OSX.

Thanks anyways for your help, it's much appreciated!

Patski