#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2003
    Posts
    392
    Rep Power
    76

    Restrict subfolder in Robots file


    I want to restrict a sub folder from bots. I have the following.

    User-Agent: *
    Disallow: /images/
    -------------
    But how do you restrict a sub folder yet allow access to the parent folder?
    Disallow: /folder/sub folder/
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Sep 2011
    Posts
    33
    Rep Power
    4
    Originally Posted by mallen
    I want to restrict a sub folder from bots. I have the following.

    User-Agent: *
    Disallow: /images/
    -------------
    But how do you restrict a sub folder yet allow access to the parent folder?
    Disallow: /folder/sub folder/
    Googlebot have Allow for robots.txt, but other search engines follow it or not I am not sure. May be the following will help you:

    User-agent: Googlebot
    Allow: /folder/
    Disallow: /folder/sub folder/

    I am not still sure above will fulfill your need
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2012
    Posts
    24
    Rep Power
    0
    Try this

    User-agent: Googlebot
    Allow: /
    Disallow: /*sub folder/

IMN logo majestic logo threadwatch logo seochat tools logo