| Class | Description |
|---|---|
| RobotExclusionFilterTest | |
| RobotRulesTest |
Tests for
RobotRules
References:
https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt
- referred as "Google/Yahoo/Bing/Ask (GYBA) convention". |
| RobotsDirectiveAggregationTest |
Copyright © 2005–2015 IIPC. All rights reserved.