org.archive.wayback.resourceindex.ziplines

Class GenericBlockLoader

java.lang.Object

org.archive.wayback.resourceindex.ziplines.GenericBlockLoader

All Implemented Interfaces:

BlockLoader

public class GenericBlockLoader
extends Object
implements BlockLoader

Generic BlockLoader, which may simplify configuration - inspecting each location to attempt to choose the correct BlockLoader: HDFS, HTTP, or LocalFile

Author:

brad

Constructor Summary

Constructors

Constructor and Description
GenericBlockLoader()

Method Summary

Methods

Modifier and Type	Method and Description
byte[]	getBlock(String url, long offset, int length) Fetch a range of bytes from a particular URL.
int	getMaxHostConnections()
int	getMaxTotalConnections()
void	init()
void	setDefaultFSURI(String uri)
void	setMaxHostConnections(int maxHostConnections)
void	setMaxTotalConnections(int maxTotalConnections)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GenericBlockLoader

public GenericBlockLoader()

Method Detail

init

public void init()

getBlock

public byte[] getBlock(String url,
              long offset,
              int length)
                throws IOException

Description copied from interface: BlockLoader

Fetch a range of bytes from a particular URL. Note that the bytes are read into memory all at once, so care should be taken with the length argument.

Specified by:

getBlock in interface BlockLoader

Parameters:

url - String URL to fetch

offset - byte start offset of the desired range

length - number of octets to fetch

Returns:

a new byte[] containing the octets fetched

Throws:

IOException - on Network and protocol failures, as well as Timeouts

setDefaultFSURI

public void setDefaultFSURI(String uri)

getMaxTotalConnections

public int getMaxTotalConnections()

setMaxTotalConnections

public void setMaxTotalConnections(int maxTotalConnections)

getMaxHostConnections

public int getMaxHostConnections()

setMaxHostConnections

public void setMaxHostConnections(int maxHostConnections)