Find the answer to your Linux question:
Results 1 to 8 of 8
Hi I want to create a search engine and I was wondering if anyone could tell me how?...
  1. 04-13-2005 #1
    Richard_The_Lionhearted
    Richard_The_Lionhearted is offline
    Banned Richard_The_Lionhearted's Avatar
    Join Date
    Feb 2005
    Location
    Here, where else?
    Posts
    379

    Creating a search engine



    Hi I want to create a search engine and I was wondering if anyone could tell me how?
    Reply With Quote Reply With Quote
  2. 04-14-2005 #2
    dark_lord_kodd
    dark_lord_kodd is offline
    Linux Newbie
    Join Date
    Nov 2004
    Posts
    239
    Are you thinking of a search engine to search the entire web? or a local one just for your web site?
    Reply With Quote Reply With Quote
  3. 04-15-2005 #3
    Richard_The_Lionhearted
    Richard_The_Lionhearted is offline
    Banned Richard_The_Lionhearted's Avatar
    Join Date
    Feb 2005
    Location
    Here, where else?
    Posts
    379
    The entire web. I know it might sound stupid but its something I wanna do.
    Reply With Quote Reply With Quote
  4. 04-15-2005 #4
    designbydave
    designbydave is offline
    Linux Newbie
    Join Date
    Mar 2005
    Location
    California!
    Posts
    159
    well i know google/yahoo/msn etc search engines work fast because the index the internet. not sure exactly what that means, i just read it somewhere. i believe it's like taking a picture of everything thats out there and storing it in a large database that can be easily searched through.
    Registered Linux User # 392752

    • \"pr0n\": An anagram of \"porn,\" possibly indicating the use of pornography.
    Reply With Quote Reply With Quote
  5. 04-15-2005 #5
    jaboua
    jaboua is offline
    Linux Engineer
    Join Date
    Mar 2005
    Posts
    1,431
    Yeah, thats right. I think it's called a crawler. Just like when you type "updatedb" it finds out where the files lie, and when you type "locate test" it searches for "test" only in the database, not in the entire filesystem...
    Reply With Quote Reply With Quote
  6. 04-15-2005 #6
    retired1af
    retired1af is offline
    Linux Engineer
    Join Date
    Mar 2005
    Location
    Where my hat is
    Posts
    765
    Crawler? More like a virus. Each link it finds allows it to spread even further.


    And I don't think the original poster has given much thought on just how much space this is going to take up.
    Registered Linux user #384279
    Vector Linux SOHO 6 / Vector Linux 7 RC 3.4
    Reply With Quote Reply With Quote
  7. 04-16-2005 #7
    dark_lord_kodd
    dark_lord_kodd is offline
    Linux Newbie
    Join Date
    Nov 2004
    Posts
    239
    Yeah

    As far as i know, your average proper search engine uses web crawlers or spiders, traversing links, saving the meta tags, url and a bit of content as they go.

    If you were going to do it, you really would need a massive ammount of storage and loads of bandwidth, and loads of time, and well it might be fun but maybe you would be better applying the same principles to a local site / network!

    Or maybe, write a script that generates words sequentialy, then eventually sentances (think infinite monkeys and infinite time here) and repeatedly search google and save the links in your own server. Maybe after several years you would have a database similar to googles!
    Reply With Quote Reply With Quote
  8. 04-16-2005 #8
    CoffeeMonster
    CoffeeMonster is offline
    Linux Newbie
    Join Date
    Mar 2005
    Posts
    205
    Quote Originally Posted by Richard_The_Lionhearted
    The entire web. I know it might sound stupid but its something I wanna do.
    Pointless, you should make one for your webserver. (if you've got one of course)

    And I don't know why but Dick's avy pisses me off, that idiot squall in ff8, I mean let go.
    Ma homeboy is Jesus himself.
    Reply With Quote Reply With Quote
« Previous Thread | Next Thread »

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Forum Rules