FAQ - How do I prevent folders from being indexed?

To prevent the Google Search Appliance (GSA) from crawling a certain directory of your web server, it is best to use a robots.txt file. See The Web Robots Pages for a discussion of robots.txt.

Sometimes you can not change the robots.txt file because you do not administer the web server. If this is the case, you should block the GSA with <META> or googleon/googleoff tags.

Related Links