org.archive.wayback.accesscontrol.robotstxt

Class RobotsDirectiveAggregation

java.lang.Object

org.archive.wayback.accesscontrol.robotstxt.RobotsDirectiveAggregation

public class RobotsDirectiveAggregation
extends Object

Class which acts as an aggregation of RobotsDirectives. If given a host String, will return a list of additional robot URLs that need to be added to the current aggregation. Allows a user to then add new RobotsDirectives for one or more robot URLs. Finally, allows the aggregation to be queried to see if any of the directives block a particular path.

Author:

brad

Constructor Summary

Constructors

Constructor and Description
RobotsDirectiveAggregation()

Method Summary

Methods

Modifier and Type	Method and Description
void	addDirectives(String url, RobotsDirectives directives)
List<String>	getMissingRobotUrls(String host)
boolean	isBlocked(String path)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RobotsDirectiveAggregation

public RobotsDirectiveAggregation()

Method Detail

getMissingRobotUrls

public List<String> getMissingRobotUrls(String host)

addDirectives

public void addDirectives(String url,
                 RobotsDirectives directives)

isBlocked

public boolean isBlocked(String path)