org.archive.wayback.resourcestore.jwat

Class JWATResource

java.lang.Object

java.io.InputStream

org.archive.wayback.core.Resource

org.archive.wayback.resourcestore.jwat.JWATResource

All Implemented Interfaces:

Closeable, AutoCloseable, org.archive.format.ArchiveFileConstants, org.archive.format.warc.WARCConstants

public class JWATResource
extends Resource
implements org.archive.format.warc.WARCConstants

JWATResource -- created by Nick Clarke for interfacing with JWAT ARC/WARC Readers Originally forked from https://bitbucket.org/nclarkekb/jwat-wayback-resourcestore

See Also:

JWATFlexResourceStore

Nested Class Summary

Nested classes/interfaces inherited from interface org.archive.format.warc.WARCConstants

org.archive.format.warc.WARCConstants.WARCRecordType

Field Summary

Fields

Modifier and Type	Field and Description
protected org.jwat.arc.ArcReader	arcReader
protected org.jwat.arc.ArcRecordBase	arcRecord
protected org.jwat.gzip.GzipEntry	gzipEntry
protected org.jwat.gzip.GzipReader	gzipReader
protected Map<String,String>	headers
protected long	length
protected InputStream	payloadStream
protected org.jwat.common.ByteCountingPushBackInputStream	pbin
protected int	status
protected org.jwat.warc.WarcReader	warcReader
protected org.jwat.warc.WarcRecord	warcRecord

Fields inherited from interface org.archive.format.warc.WARCConstants

COLON_SPACE, COMPRESSED_WARC_FILE_EXTENSION, CONTENT_DESCRIPTION, CONTENT_LENGTH, CONTENT_TYPE, DEFAULT_ENCODING, DEFAULT_MAX_WARC_FILE_SIZE, DOT_COMPRESSED_WARC_FILE_EXTENSION, DOT_WARC_FILE_EXTENSION, FTP_CONTROL_CONVERSATION_MIMETYPE, HEADER_FIELD_SEPARATOR, HEADER_KEY_BLOCK_DIGEST, HEADER_KEY_CONCURRENT_TO, HEADER_KEY_DATE, HEADER_KEY_ETAG, HEADER_KEY_FILENAME, HEADER_KEY_ID, HEADER_KEY_IP, HEADER_KEY_LAST_MODIFIED, HEADER_KEY_PAYLOAD_DIGEST, HEADER_KEY_PROFILE, HEADER_KEY_REFERS_TO, HEADER_KEY_REFERS_TO_DATE, HEADER_KEY_REFERS_TO_FILE_OFFSET, HEADER_KEY_REFERS_TO_FILENAME, HEADER_KEY_REFERS_TO_TARGET_URI, HEADER_KEY_TRUNCATED, HEADER_KEY_TYPE, HEADER_KEY_URI, HEADER_LINE_ENCODING, HTTP_REQUEST_MIMETYPE, HTTP_RESPONSE_MIMETYPE, MAX_LINE_LENGTH, MAX_WARC_HEADER_LINE_LENGTH, NAMED_FIELD_CHECKSUM_LABEL, NAMED_FIELD_DESCRIPTION, NAMED_FIELD_FILEDESC, NAMED_FIELD_IP_LABEL, NAMED_FIELD_RELATED_LABEL, NAMED_FIELD_TRUNCATED, NAMED_FIELD_TRUNCATED_VALUE_HEAD, NAMED_FIELD_TRUNCATED_VALUE_LENGTH, NAMED_FIELD_TRUNCATED_VALUE_TIME, NAMED_FIELD_TRUNCATED_VALUE_UNSPECIFIED, NAMED_FIELD_WARCFILENAME, PLACEHOLDER_RECORD_LENGTH_STRING, PROFILE_REVISIT_IDENTICAL_DIGEST, PROFILE_REVISIT_NOT_MODIFIED, PROFILE_REVISIT_URI_AGNOSTIC_IDENTICAL_DIGEST, TRUNCATED_VALUE_UNSPECIFIED, TYPE, WARC_FIELDS_TYPE, WARC_FILE_EXTENSION, WARC_HEADER_ENCODING, WARC_ID, WARC_MAGIC, WARC_VERSION, WSP

Fields inherited from interface org.archive.format.ArchiveFileConstants

ABSOLUTE_OFFSET_KEY, CDX, CDX_FILE, CDX_LINE_BUFFER_SIZE, CRLF, DATE_FIELD_KEY, DEFAULT_DIGEST_METHOD, DOT_COMPRESSED_FILE_EXTENSION, DUMP, GZIP_DUMP, HEADER, INVALID_SUFFIX, LENGTH_FIELD_KEY, MIMETYPE_FIELD_KEY, NOHEAD, OCCUPIED_SUFFIX, READER_IDENTIFIER_FIELD_KEY, RECORD_IDENTIFIER_FIELD_KEY, SINGLE_SPACE, TYPE_FIELD_KEY, URL_FIELD_KEY, VERSION_FIELD_KEY

Constructor Summary

Constructors

Constructor and Description
JWATResource()

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
Map<String,String>	getHttpHeaders() Assumes an HTTP response - return the HTTP headers, not including the HTTP Message header
long	getRecordLength()
String	getRefersToDate() return WARC-Refers-To-Date WARC record header value or equivalent.
String	getRefersToTargetURI() return WARC-Refer-To-Target-URI WARC record header value or equivalent.
static Resource	getResource(InputStream rin, long offset)
int	getStatusCode() Assumes an HTTP resource - return the HTTP response code

Methods inherited from class org.archive.wayback.core.Resource

available, getHeader, mark, markSupported, parseHeaders, read, read, read, reset, setChunkedEncoding, setInputStream, skip

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

pbin

protected org.jwat.common.ByteCountingPushBackInputStream pbin

gzipReader

protected org.jwat.gzip.GzipReader gzipReader

gzipEntry

protected org.jwat.gzip.GzipEntry gzipEntry

arcReader

protected org.jwat.arc.ArcReader arcReader

arcRecord

protected org.jwat.arc.ArcRecordBase arcRecord

warcReader

protected org.jwat.warc.WarcReader warcReader

warcRecord

protected org.jwat.warc.WarcRecord warcRecord

payloadStream

protected InputStream payloadStream

headers

protected Map<String,String> headers

length

protected long length

status

protected int status

Constructor Detail

JWATResource

public JWATResource()

Method Detail

getResource

public static Resource getResource(InputStream rin,
                   long offset)
                            throws IOException,
                                   ResourceNotAvailableException

Throws:

IOException

ResourceNotAvailableException

getHttpHeaders

public Map<String,String> getHttpHeaders()

Description copied from class: Resource

Assumes an HTTP response - return the HTTP headers, not including the HTTP Message header

Specified by:

getHttpHeaders in class Resource

Returns:

key-value Map of HTTP headers

getRecordLength

public long getRecordLength()

Specified by:

getRecordLength in class Resource

Returns:

the size in bytes of the record payload, including HTTP header

getStatusCode

public int getStatusCode()

Description copied from class: Resource

Assumes an HTTP resource - return the HTTP response code

Specified by:

getStatusCode in class Resource

Returns:

the HTTP response code from the HTTP message

getRefersToTargetURI

public String getRefersToTargetURI()

Description copied from class: Resource

return WARC-Refer-To-Target-URI WARC record header value or equivalent. Default implementation returns null.

Overrides:

getRefersToTargetURI in class Resource

Returns:

header value (URI)

getRefersToDate

public String getRefersToDate()

Description copied from class: Resource

return WARC-Refers-To-Date WARC record header value or equivalent. Default implementation returns null

Overrides:

getRefersToDate in class Resource

Returns:

14-digit timestamp string (yyyyMMddHHmmss)

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class Resource

Throws:

IOException