1. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Rep Power

    Good search engine examples

    I'm tired of using overpriced third party scripts, for things like a local search engine to index the site and allow searches resulting from the page urls. I'm using PHP and MySQL and am capable of coding, I just need to get a good heading on where to start to design my own local search engine.

    The web-site uses mod rewrite to change the parameter ridden URLs to keyword rich URLs containing all the keywords a local search engine would need to allow a user to search the site.

    I need to allow users to search based on 4 column names - Manufacturer, Model, Category, Subcategory and a search in the description column would be nice as well.

    Is there any good tutorials or examples or pointers somebody can throw my way to help me get started on creating my own local search engine for my site?
    Last edited by BitZoid; April 23rd, 2013 at 07:23 PM.
    -- Success achieved from tribulation --
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2012
    Rep Power
    To do this, go to http://phpcrawl.cuab.de/ and download, it's free and can spider your website

    Once you have it uploaded, set up a cron job so that it auto spiders your site while having it ignore any links it's already been to

    Next google up "how to make a search engine in php" and use the results from the spider as your database for your searchers and find a good tutorial for how to set up the search portion

    This could also be usefull, it contains classes which help with Dom parsing http://simplehtmldom.sourceforge.net/

    http://www.phpclasses.org/package/3-PHP-HTTP-client-to-access-Web-site-pages.html which is another very usefull set of classes
    Last edited by jack13580; April 23rd, 2013 at 08:20 PM.
  4. #3
  5. No Profile Picture
    Lost in code
    Devshed Supreme Being (6500+ posts)

    Join Date
    Dec 2004
    Rep Power
    There are sort of two different approaches here.

    You can crawl the pages and generate a search index based on the content on them. This is what major search engines like Google and Bing do, and what most of the PHP based search scripts that you'll find do. However, this method is fairly heavy from a performance standpoint, and it's difficult to get good search results from it. Google is able to do it because they invest millions of dollars every day into improving their algorithms, but obviously you don't have that option.

    Searching the fields in your database directly is a better approach for a single-site search engine. Since this depends on your database structure, there aren't really many pre-built PHP scripts for doing it. However, it is primarily just a matter of generating appropriate SQL queries based on the values provided by the user on the search form.

    For a larger site or for searches that involve a lot of text, you will run into limitations of MySQL as far as the relevancy of the results it returns and its performance. As a replacement for MySQL you can use something like SOLR, but the setup and maintenance work is much greater.

    Writing your own code is pretty much never cheaper than buying code that is already written; but if there isn't code already written, then you don't have a lot of choice about it.

    Originally Posted by Spad
    Ah USB, the only rectangular connector where you have to make 3 attempts before you get it the right way around

IMN logo majestic logo threadwatch logo seochat tools logo