com.esri.gpt.framework.robots
Class BotsParser

java.lang.Object
  com.esri.gpt.framework.robots.BotsParser

public class BotsParser
extends Object

Parser of "robots.txt" file.

Method Summary

boolean	canOverride() Checks if robots.txt enabled flag can be overridden.
static BotsParser	getDefaultInstance() Gets default instance.
String	getUserAgent() Gets user agent.
boolean	isEnabled() Checks if using robots.txt is enabled.
Bots	readRobotsTxt(BotsMode mode, MatchingStrategy matchingStrategy, WinningStrategy winningStrategy, InputStream robotsTxt) Parses robots TXT
Bots	readRobotsTxt(BotsMode mode, MatchingStrategy matchingStrategy, WinningStrategy winningStrategy, String serverUrl) Parses context of the Robots.txt file if available.
Bots	readRobotsTxt(BotsMode mode, MatchingStrategy matchingStrategy, WinningStrategy winningStrategy, URL serverUrl) Parses context of the Robots.txt file if available.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getDefaultInstance

public static BotsParser getDefaultInstance()

Gets default instance.

Returns:

instance

getUserAgent

public String getUserAgent()

Gets user agent.

Returns:

user agent

isEnabled

public boolean isEnabled()

Checks if using robots.txt is enabled.

Returns:

true if using robots.txt is enabled

canOverride

public boolean canOverride()

Checks if robots.txt enabled flag can be overridden.

Returns:

true if robots.txt enabled flag can be overridden

readRobotsTxt

public Bots readRobotsTxt(BotsMode mode,
                          MatchingStrategy matchingStrategy,
                          WinningStrategy winningStrategy,
                          String serverUrl)

Parses context of the Robots.txt file if available.

Parameters:

mode - robots.txt mode

matchingStrategy - matching strategy

winningStrategy - winning strategy

serverUrl - url of the server which is expected to have robots.txt present

Returns:

instance of Bots or null if unable to obtain robots.txt

readRobotsTxt

public Bots readRobotsTxt(BotsMode mode,
                          MatchingStrategy matchingStrategy,
                          WinningStrategy winningStrategy,
                          URL serverUrl)

Parses context of the Robots.txt file if available.

Parameters:

mode - robots.txt mode

matchingStrategy - matching strategy

winningStrategy - winning strategy

serverUrl - url of the server which is expected to have robots.txt present

Returns:

instance of Bots or null if unable to obtain robots.txt

readRobotsTxt

public Bots readRobotsTxt(BotsMode mode,
                          MatchingStrategy matchingStrategy,
                          WinningStrategy winningStrategy,
                          InputStream robotsTxt)

Parses robots TXT

Parameters:

mode - robots.txt mode

matchingStrategy - matching strategy

winningStrategy - winning strategy

robotsTxt - stream of data

Returns:

instance or RobotsTxt or null