org.archive.cdxserver

Class CDXServer

java.lang.Object

org.archive.cdxserver.BaseCDXServer

org.archive.cdxserver.CDXServer

All Implemented Interfaces:

org.springframework.beans.factory.InitializingBean

public class CDXServer
extends BaseCDXServer

Field Summary

Fields

Modifier and Type	Field and Description
protected String	cdxFormat
protected org.archive.format.cdx.CDXLineFactory	cdxLineFactory
protected org.archive.format.cdx.CDXInputSource	cdxSource
protected boolean	collapseToLast
protected org.archive.format.gzip.zipnum.ZipNumParams	defaultParams
protected int	maxPageSize
protected String[]	noCollapsePrefix
protected int	queryMaxLimit
protected org.archive.format.gzip.zipnum.ZipNumCluster	zipnumSource

Fields inherited from class org.archive.cdxserver.BaseCDXServer

ajaxAccessControl, authChecker, canonicalizer, CDX_AUTH_TOKEN, cookieAuthToken, surtMode, urlSurtRangeComputer

Constructor Summary

Constructors

Constructor and Description
CDXServer()

Method Summary

Methods

Modifier and Type	Method and Description
void	afterPropertiesSet()
protected org.archive.util.iterator.CloseableIterator<String>	createBoundedCdxIterator(String[] startEndUrl, CDXQuery query, org.archive.format.gzip.zipnum.ZipNumIndex.PageResult pageResult, org.archive.util.iterator.CloseableIterator<String> idx)
protected org.archive.util.iterator.CloseableIterator<String>	createPagedCdxIterator(String[] startEndUrl, CDXQuery query, AuthToken authToken, CDXWriter responseWriter)
protected boolean	determineGzip(javax.servlet.http.HttpServletRequest request, CDXQuery query)
org.archive.format.cdx.CDXLine	findLastCapture(String url, String digest, boolean ignoreRobots) Look up the latest (non-revisit) capture of url in the CDX database.
void	getCdx(CDXQuery query, AuthToken authToken, CDXWriter responseWriter)
void	getCdx(javax.servlet.http.HttpServletRequest request, javax.servlet.http.HttpServletResponse response, CDXQuery query)
String	getCdxFormat()
org.archive.format.cdx.CDXInputSource	getCdxSource()
org.archive.format.gzip.zipnum.ZipNumParams	getDefaultParams()
String[]	getNoCollapsePrefix()
int	getPageSize()
int	getQueryMaxLimit()
org.archive.format.gzip.zipnum.ZipNumCluster	getZipnumSource()
boolean	isCollapseToLast()
void	setCdxFormat(String cdxFormat)
void	setCdxSource(org.archive.format.cdx.CDXInputSource cdxSource)
void	setCollapseToLast(boolean collapseToLast) If set to true, timestamp-collapsing writes out the last best capture in the collapse group, instead of the first.
void	setDefaultParams(org.archive.format.gzip.zipnum.ZipNumParams defaultParams)
void	setNoCollapsePrefix(String[] noCollapsePrefix)
void	setPageSize(int pageSize)
void	setQueryMaxLimit(int queryMaxLimit)
void	setZipnumSource(org.archive.format.gzip.zipnum.ZipNumCluster zipnumSource)
protected void	writeCdxResponse(CDXWriter responseWriter, org.archive.util.iterator.CloseableIterator<String> cdx, int readLimit, CDXQuery query, AuthToken authToken, CDXAccessFilter accessChecker)
protected void	writeIdxResponse(CDXWriter responseWriter, org.archive.util.iterator.CloseableIterator<String> iter)

Methods inherited from class org.archive.cdxserver.BaseCDXServer

canonicalize, createAuthToken, extractAuthToken, getAjaxAccessControl, getAuthChecker, getCookieAuthToken, handleAjax, isSurtMode, prepareResponse, setAjaxAccessControl, setAuthChecker, setCookieAuthToken, setSurtMode

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

zipnumSource

protected org.archive.format.gzip.zipnum.ZipNumCluster zipnumSource

cdxSource

protected org.archive.format.cdx.CDXInputSource cdxSource

cdxFormat

protected String cdxFormat

cdxLineFactory

protected org.archive.format.cdx.CDXLineFactory cdxLineFactory

maxPageSize

protected int maxPageSize

queryMaxLimit

protected int queryMaxLimit

noCollapsePrefix

protected String[] noCollapsePrefix

collapseToLast

protected boolean collapseToLast

defaultParams

protected org.archive.format.gzip.zipnum.ZipNumParams defaultParams

Constructor Detail

CDXServer

public CDXServer()

Method Detail

afterPropertiesSet

public void afterPropertiesSet()
                        throws Exception

Specified by:

afterPropertiesSet in interface org.springframework.beans.factory.InitializingBean

Overrides:

afterPropertiesSet in class BaseCDXServer

Throws:

Exception

getZipnumSource

public org.archive.format.gzip.zipnum.ZipNumCluster getZipnumSource()

setZipnumSource

public void setZipnumSource(org.archive.format.gzip.zipnum.ZipNumCluster zipnumSource)

getPageSize

public int getPageSize()

setPageSize

public void setPageSize(int pageSize)

getDefaultParams

public org.archive.format.gzip.zipnum.ZipNumParams getDefaultParams()

setDefaultParams

public void setDefaultParams(org.archive.format.gzip.zipnum.ZipNumParams defaultParams)

getCdxFormat

public String getCdxFormat()

setCdxFormat

public void setCdxFormat(String cdxFormat)

getQueryMaxLimit

public int getQueryMaxLimit()

setQueryMaxLimit

public void setQueryMaxLimit(int queryMaxLimit)

getNoCollapsePrefix

public String[] getNoCollapsePrefix()

setNoCollapsePrefix

public void setNoCollapsePrefix(String[] noCollapsePrefix)

isCollapseToLast

public boolean isCollapseToLast()

Returns:

the collapseToLast

setCollapseToLast

public void setCollapseToLast(boolean collapseToLast)

If set to true, timestamp-collapsing writes out the last best capture in the collapse group, instead of the first.

Initial value is false.

Parameters:

collapseToLast - the collapseToLast to set

See Also:

DupeTimestampLastBestStatusFilter

getCdxSource

public org.archive.format.cdx.CDXInputSource getCdxSource()

setCdxSource

public void setCdxSource(org.archive.format.cdx.CDXInputSource cdxSource)

determineGzip

protected boolean determineGzip(javax.servlet.http.HttpServletRequest request,
                    CDXQuery query)

getCdx

@RequestMapping(value="/cdx")
public void getCdx(javax.servlet.http.HttpServletRequest request,
                         javax.servlet.http.HttpServletResponse response,
                         CDXQuery query)

getCdx

public void getCdx(CDXQuery query,
          AuthToken authToken,
          CDXWriter responseWriter)
            throws IOException

Throws:

IOException

createPagedCdxIterator

protected org.archive.util.iterator.CloseableIterator<String> createPagedCdxIterator(String[] startEndUrl,
                                                                         CDXQuery query,
                                                                         AuthToken authToken,
                                                                         CDXWriter responseWriter)
                                                                              throws IOException

Throws:

IOException

createBoundedCdxIterator

protected org.archive.util.iterator.CloseableIterator<String> createBoundedCdxIterator(String[] startEndUrl,
                                                                           CDXQuery query,
                                                                           org.archive.format.gzip.zipnum.ZipNumIndex.PageResult pageResult,
                                                                           org.archive.util.iterator.CloseableIterator<String> idx)
                                                                                throws IOException

Throws:

IOException

writeIdxResponse

protected void writeIdxResponse(CDXWriter responseWriter,
                    org.archive.util.iterator.CloseableIterator<String> iter)

writeCdxResponse

protected void writeCdxResponse(CDXWriter responseWriter,
                    org.archive.util.iterator.CloseableIterator<String> cdx,
                    int readLimit,
                    CDXQuery query,
                    AuthToken authToken,
                    CDXAccessFilter accessChecker)

findLastCapture

public org.archive.format.cdx.CDXLine findLastCapture(String url,
                                             String digest,
                                             boolean ignoreRobots)

Look up the latest (non-revisit) capture of url in the CDX database. If digest is non-null, return only a capture with identical digest.

Parameters:

url - URL (in regular form) to look for

digest - content digest in the same format as CDX database, or null if any version qualifies.

ignoreRobots - whether robots.txt-excluded captures qualify

Returns:

CDXLine found