com.esri.gpt.framework.robots
Class BotsUtils

java.lang.Object
  com.esri.gpt.framework.robots.BotsUtils

public final class BotsUtils
extends Object

Robots txt utility class/shortcut methods

Constructor Summary

BotsUtils()

Method Summary

static Pattern	compileWildcardPattern(String patternWithWildcards) Compiles wildcard pattern into a regular expression.
static BotsParser	parser() Gets default parser
static Bots	readBots(BotsMode mode, MatchingStrategy matchingStrategy, WinningStrategy winningStrategy, String serverUrl) Reads robots.txt
static Bots	readBots(BotsMode mode, MatchingStrategy matchingStrategy, WinningStrategy winningStrategy, URL serverUrl) Reads robots.txt
static Bots	readBots(BotsMode mode, String serverUrl) Reads robots.txt
static Bots	readBots(BotsMode mode, URL serverUrl) Reads robots.txt
static Access	requestAccess(Bots bots, String path) Request access to the resource.
static String	transformUrl(Bots bots, String url) Replaces host part in given url if "host" directive found in robots.txt

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BotsUtils

public BotsUtils()

Method Detail

parser

public static BotsParser parser()

Gets default parser

Returns:

default parser (never null)

readBots

public static Bots readBots(BotsMode mode,
                            MatchingStrategy matchingStrategy,
                            WinningStrategy winningStrategy,
                            String serverUrl)

Reads robots.txt

Parameters:

mode - robots.txt mode

matchingStrategy - matching strategy

winningStrategy - winning strategy

serverUrl - url of the server which is expected to have robots.txt present

Returns:

instance of Bots or null if unable to obtain robots.txt

readBots

public static Bots readBots(BotsMode mode,
                            MatchingStrategy matchingStrategy,
                            WinningStrategy winningStrategy,
                            URL serverUrl)

Reads robots.txt

Parameters:

mode - robots.txt mode

matchingStrategy - matching strategy

winningStrategy - winning strategy

serverUrl - url of the server which is expected to have robots.txt present

Returns:

instance of Bots or null if unable to obtain robots.txt

readBots

public static Bots readBots(BotsMode mode,
                            String serverUrl)

Reads robots.txt

Parameters:

mode - robots.txt mode

serverUrl - url of the server which is expected to have robots.txt present

Returns:

instance of Bots or null if unable to obtain robots.txt

readBots

public static Bots readBots(BotsMode mode,
                            URL serverUrl)

Reads robots.txt

Parameters:

mode - robots.txt mode

serverUrl - url of the server which is expected to have robots.txt present

Returns:

instance of Bots or null if unable to obtain robots.txt

requestAccess

public static Access requestAccess(Bots bots,
                                   String path)

Request access to the resource.

Parameters:

bots - robots

path - relative path to the resource

Returns:

access (never null

transformUrl

public static String transformUrl(Bots bots,
                                  String url)

Replaces host part in given url if "host" directive found in robots.txt

Parameters:

bots - robots.txt

url - url to transform

Returns:

transformed url

compileWildcardPattern

public static Pattern compileWildcardPattern(String patternWithWildcards)

Compiles wildcard pattern into a regular expression.

Allowed wildcards:

* - matches any sequence of characters
$ - matches end of sequence

Parameters:

patternWithWildcards - pattern with wildcards

Returns:

compiled pattern