编辑一份 Robots 文件，另存为robots.txt，存在到服务器的根目录里
Java代码
- User-agent: *
- Disallow: /plus/ad_js.php
- Disallow: /plus/advancedsearch.php
- Disallow: /plus/car.php
- Disallow: /plus/carbuyaction.php
- Disallow: /plus/shops_buyaction.php
- Disallow: /plus/erraddsave.php
- Disallow: /plus/posttocar.php
- Disallow: /plus/disdls.php
- Disallow: /plus/feedback_js.php
- Disallow: /plus/mytag_js.php
- Disallow: /plus/rss.php
- Disallow: /plus/search.php
- Disallow: /plus/recommend.php
- Disallow: /plus/stow.php
- Disallow: /plus/count.php
- Disallow: /include
- Disallow: /templets
效果图：
打开百度站长链接：http://zhanzhang.baidu.com/robots/index
输入域名或子域名
robots.txt文件用法举例
Robots 限制搜索引擎蜘蛛抓取哪些文件
原文：http://my.oschina.net/u/2273085/blog/505174
