Hi I'm a newbie with scripting and shell...
I'm playing around with ideas from books I'm reading about building crawlers... and tried to build something simple that would capture the url references from lynx and insert them into an array that I could cycle through and put into a mysql table. built the database/table... even got as far as assigning the urls to a variable... but it just leaves me with a whole ugly mess of text... I'm sure there's better out there, but I'm just kind of playing with this as a way to learn.
Anyone interested in helping me figure this out????

question1 - is it even necessary to get the urls into an array? or is it easier to insert directly into mysql??

question2 - if array is needed... what ways can I go about assigning each url to a particular node??

question3 - don't much about sed and awk at all... but is there away to use them to break up the mess of text that gets assigned to the variable THEN insert into an array or mysql???

any help greatly appreciated.
also if possible, a little blurb about how and why of your code or approach would be great so I can learn why this way is a better approach than that... kind of thing...

thanks