Find the answer to your Linux question:
Results 1 to 8 of 8
Hi I want to create a search engine and I was wondering if anyone could tell me how?...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Banned Richard_The_Lionhearted's Avatar
    Join Date
    Feb 2005
    Location
    Here, where else?
    Posts
    379

    Creating a search engine


    Hi I want to create a search engine and I was wondering if anyone could tell me how?

  2. #2
    Linux Newbie
    Join Date
    Nov 2004
    Posts
    239
    Are you thinking of a search engine to search the entire web? or a local one just for your web site?

  3. #3
    Banned Richard_The_Lionhearted's Avatar
    Join Date
    Feb 2005
    Location
    Here, where else?
    Posts
    379
    The entire web. I know it might sound stupid but its something I wanna do.

  4. $spacer_open
    $spacer_close
  5. #4
    Linux Newbie
    Join Date
    Mar 2005
    Location
    California!
    Posts
    159
    well i know google/yahoo/msn etc search engines work fast because the index the internet. not sure exactly what that means, i just read it somewhere. i believe it's like taking a picture of everything thats out there and storing it in a large database that can be easily searched through.
    Registered Linux User # 392752

    \"pr0n\": An anagram of \"porn,\" possibly indicating the use of pornography.

  6. #5
    Linux Engineer
    Join Date
    Mar 2005
    Posts
    1,431
    Yeah, thats right. I think it's called a crawler. Just like when you type "updatedb" it finds out where the files lie, and when you type "locate test" it searches for "test" only in the database, not in the entire filesystem...

  7. #6
    Linux Engineer
    Join Date
    Mar 2005
    Location
    Where my hat is
    Posts
    766
    Crawler? More like a virus. Each link it finds allows it to spread even further.


    And I don't think the original poster has given much thought on just how much space this is going to take up.
    Registered Linux user #384279
    Vector Linux SOHO 7

  8. #7
    Linux Newbie
    Join Date
    Nov 2004
    Posts
    239
    Yeah

    As far as i know, your average proper search engine uses web crawlers or spiders, traversing links, saving the meta tags, url and a bit of content as they go.

    If you were going to do it, you really would need a massive ammount of storage and loads of bandwidth, and loads of time, and well it might be fun but maybe you would be better applying the same principles to a local site / network!

    Or maybe, write a script that generates words sequentialy, then eventually sentances (think infinite monkeys and infinite time here) and repeatedly search google and save the links in your own server. Maybe after several years you would have a database similar to googles!

  9. #8
    Linux Newbie
    Join Date
    Mar 2005
    Posts
    205
    Quote Originally Posted by Richard_The_Lionhearted
    The entire web. I know it might sound stupid but its something I wanna do.
    Pointless, you should make one for your webserver. (if you've got one of course)

    And I don't know why but Dick's avy pisses me off, that idiot squall in ff8, I mean let go.
    Ma homeboy is Jesus himself.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •