Thread: Search engines

    #1
  1. No Profile Picture
    Junior Member
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2001
    Location
    UK
    Posts
    10
    Rep Power
    0

    Search engines


    Is there a way to stop a page from being ever found by a search engine? I mean can you have a special header or something so that you can only find the page by typing in the URL / following a link? This would be useful as although I have a proper login thing for my admin area, if I do a particular search with google, the login page is found, and I don't want that page advertised.

    thanks!
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2001
    Posts
    5
    Rep Power
    0
    Just don't put any link to your login page since search engines follow links on your html pages recursively.
    If you must deny access with a HTTP_REFERER set, try this:

    RewriteEngine on
    # if request_uri is your login.html
    RewriteCond %{REQUEST_URI} ^/url_path/to/login\.html$
    # and if http_referer is not empty
    # opening your login.html from bookmark or by typing the url has a blank http_referer
    RewriteCond %{HTTP_REFERER} !^$
    # [403] it
    RewriteRule ^(.+) - [F]
  4. #3
  5. No Profile Picture
    Junior Member
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2001
    Location
    UK
    Posts
    10
    Rep Power
    0
    Forgive me for not understanding the code you provided me with, but it is useful to know that if the page has no hyperlink to it, it is not found, so I will replace the hyperlink with a picture of the hyperlink, in a form, that goes to the login page instead.
    Thankyou for your help!
  6. #4
  7. No Profile Picture
    Member
    Devshed Newbie (0 - 499 posts)

    Join Date
    Sep 2000
    Location
    Canada
    Posts
    2
    Rep Power
    0
    Don't know if you've done this already, but you might want to make up a robots.txt file to instruct the indexing robots what files you want left alone. It's not always acknowledged by robots but most of the time it is:
    http://www.searchengineworld.com/rob...s_tutorial.htm

    T

IMN logo majestic logo threadwatch logo seochat tools logo