Extracting website headlines

[jeffrey_seeNJ]

Need help here... which command can i use to extract out the headline from some website using linux konsole???

[smolloy]

Not sure I fully understand you, but maybe wget is the command you're looking for?

wget http://news.bbc.co.uk

[jeffrey_seeNJ]

Not sure I fully understand you, but maybe wget is the command you're looking for?

wget http://news.bbc.co.uk

actually i wget the website already, and i grep the headlines i wan but other than the headlines, there is still other words like , <a href........>, i wan to filter out this <a href> thing... wat command can i use??

[smolloy]

actually i wget the website already, and i grep the headlines i wan but other than the headlines, there is still other words like , <a href........>, i wan to filter out this <a href> thing... wat command can i use??

Google is your friend

I found this solution

smolloy@sabayonx86-64 ~ $ cat testfile.txt
<a href>blah blah</a><br>some more text<a href>more text</a>
smolloy@sabayonx86-64 ~ $ sed 's/<[^<>]*>//g' testfile.txt
blah blahsome more textmore text

[jeffrey_seeNJ]

i got it... thx a lot =)

[jeffrey_seeNJ]

Google is your friend

I found this solution

smolloy@sabayonx86-64 ~ $ cat testfile.txt
<a href>blah blah</a><br>some more text<a href>more text</a>
smolloy@sabayonx86-64 ~ $ sed 's/<[^<>]*>//g' testfile.txt
blah blahsome more textmore text

ya...thx a lot...by the way...can u explain [^<>]*does wat?

[Cabhan]

As you may or may not know, sed often uses regular expressions to do its work. A regular expression is a way to match a pattern, as opposed to a block of literal text. For instance, if I want to match "Fred and Barney" and "Frank and Bob", I might use the regular expression "F[a-z]+ and B[a-z]+", which means 'F', followed by 1 or more lowercase letters, followed by ' and ', followed by 'B', followed by 1 or more lowercase letters.

In a regular expression, anything surrounded by [...] is called a character class. A character class is only one character long in the match, and defines what may be matched in that space. In this particular case, the character class begins with a '^', which means "This character class contains everything EXCEPT what I list". So this character class means "Everything except < and >".

The '*' is code for 0 or more of the preceding.

So in this case, the expression

s/<[^<>]*>//g

means "Find every instance of a <...> surrounding characters that are not '<' or '>', and replace them with nothing.

[Franklin52]

So in this case, the expression
Code:

s/<[^<>]*>//g

means "Find every instance of a <...> surrounding characters that are not '<' or '>', and replace them with nothing.

I don't think here the ">" is necessary in the class [^<>] so this should also do the job:

s/<[^<]*>//g

Regards

[jeffrey_seeNJ]

Google is your friend

I found this solution

smolloy@sabayonx86-64 ~ $ cat testfile.txt
<a href>blah blah</a><br>some more text<a href>more text</a>
smolloy@sabayonx86-64 ~ $ sed 's/<[^<>]*>//g' testfile.txt
blah blahsome more textmore text

i try putting the asterisk in front but hw come it doesnt work at all?? sed 's/<*[^<>]>//g'

[Cabhan]

As I mentioned, '*' means 0 more more of the preceding. So the original regular expression to said match <...>, with anything in between.

By moving the '*', your new regular expression says "Match 0 more '<'s, followed by a single character that is not a '>' or '<', followed by a '>'".

The following match your new expression:

h> # 0 '<'s at the beginning
<h>
<<h>
<<<<<<<<<<<<<<<<h>
etc.

This does not match:

<hello> # you are only allowed to have a single character between the '<' and '>'

Does that make sense?