Package thredds.catalog

Class InvDatasetScan

java.lang.Object

thredds.catalog.InvDataset

thredds.catalog.InvDatasetImpl

thredds.catalog.InvCatalogRef

thredds.catalog.InvDatasetScan

public class InvDatasetScan
extends InvCatalogRef

Represents server-side information on how to scan a collection of datasets for catalog generation.

Used by the THREDDS Data Server (TDS) to automatically generate catalogs.

Typically built from the information given by a datasetScan element in a TDS config catalog.

Usage notes:

1. The static methods setContext() and setCatalogServletName() should only be called once per web application instance. For instance, in your HttpServlet implementations init() method.
2. The method setScanLocation() should not be used; it is "public by accident".

Should be thread safe except that the above two usage notes are not enforced.

Field Summary

Fields inherited from class thredds.catalog.InvDatasetImpl

mark, ncmlElement, tm, tmi

Fields inherited from class thredds.catalog.InvDataset

access, authorityName, catalog, collectionType, contributors, creators, dataFormatType, datasets, dataType, dates, defaultService, docs, gc, harvest, id, keywords, metadata, name, parent, projects, properties, publishers, restrictAccess, services, tc, variableMapLink, variables

Constructor Summary

Constructors

Constructor	Description
InvDatasetScan(InvCatalogImpl catalog, InvDatasetImpl parent, String name, String path, String scanLocation, String filter, boolean addDatasetSize, String addLatest, boolean sortOrderIncreasing, String datasetNameMatchPattern, String startTimeSubstitutionPattern, String duration)	Constructor.
InvDatasetScan(InvCatalogImpl catalog, InvDatasetImpl parent, String name, String path, String scanLocation, CrawlableDatasetFilter filter, boolean addDatasetSize, String addLatest, boolean sortOrderIncreasing, String datasetNameMatchPattern, String startTimeSubstitutionPattern, String duration)	Constructor.
InvDatasetScan(InvDatasetImpl parent, String name, String path, String scanLocation, String configClassName, Object configObj, CrawlableDatasetFilter filter, CrawlableDatasetLabeler identifier, CrawlableDatasetLabeler namer, boolean addDatasetSize, CrawlableDatasetSorter sorter, Map<String,ProxyDatasetHandler> proxyDatasetHandlers, List<DatasetEnhancer> childEnhancerList, CatalogRefExpander catalogRefExpander)
InvDatasetScan(InvDatasetImpl parent, String name, String path, String scanLocation, String id, InvDatasetScan from)

Method Summary

Modifier and Type	Method	Description
boolean	getAddDatasetSize()
CatalogRefExpander	getCatalogRefExpander()
List<DatasetEnhancer>	getChildEnhancerList()
String	getCrDsClassName()
Object	getCrDsConfigObj()
CrawlableDatasetFilter	getFilter()
CrawlableDatasetLabeler	getIdentifier()
String	getInvalidMessage()
CrawlableDatasetLabeler	getNamer()
String	getPath()
Map<String,ProxyDatasetHandler>	getProxyDatasetHandlers()
String	getScanLocation()
CrawlableDatasetSorter	getSorter()
boolean	isRead()
boolean	isValid()
InvCatalogImpl	makeCatalogForDirectory(String orgPath, URI catURI)	Try to build a catalog for the given path by scanning the location associated with this InvDatasetScan.
InvCatalog	makeLatestCatalogForDirectory(String orgPath, URI baseURI)	Deprecated. Instead use makeProxyDsResolver() which provides more general proxy dataset handling.
InvCatalogImpl	makeProxyDsResolverCatalog(String path, URI baseURI)	Try to build a catalog for the given resolver path by scanning the location associated with this InvDatasetScan.
CrawlableDataset	requestCrawlableDataset(String path)	Return the CrawlableDataset for the given path, null if this InvDatasetScan does not allow (filters out) the requested CrawlableDataset.
static void	setCatalogServletName(String catServletName)
static void	setContext(String c)
void	setScanLocation(String scanLocation)	Resets the location being scanned (DO NOT USE THIS METHOD, "public by accident").
String	translatePathToLocation(String dsPath)	Return the CrawlableDataset path/location that corresponds to the given dataset path.

Methods inherited from class thredds.catalog.InvCatalogRef

equals, findDatasetByName, findProperty, findService, finish, getAccess, getAccess, getAlias, getAuthority, getCalendarDateCoverage, getCollectionType, getContributors, getCreators, getDataFormatType, getDatasets, getDataType, getDates, getDocumentation, getDocumentation, getFullName, getGeospatialCoverage, getID, getKeywords, getMark, getMetadata, getMetadata, getName, getParent, getProjects, getProperties, getProxyDataset, getPublishers, getServiceDefault, getUniqueID, getURI, getUrlPath, getUserProperty, getVariableMapLink, getVariables, getXlinkHref, hasAccess, hashCode, hasNestedDatasets, isHarvest, readAsynch, release, setXlinkHref, useRemoteCatalogService

Methods inherited from class thredds.catalog.InvDatasetImpl

addAccess, addAccess, addDataset, addDataset, addDocumentation, addProperty, addService, canonicalize, dump, getAccessLocal, getDataSize, getLastModifiedDate, getLocalMetadata, getLocalMetadataInheritable, getNcmlElement, getParentReal, getServiceName, getServicesLocal, removeDataset, removeLocalMetadata, removeService, replaceDataset, resolve, setAlias, setAuthority, setCatalog, setCollectionType, setContributors, setDataFormatType, setDataSize, setDataType, setGeospatialCoverage, setHarvest, setID, setKeywords, setLastModifiedDate, setLastModifiedDate, setLocalMetadata, setMark, setName, setNcmlElement, setParent, setProjects, setPublishers, setResourceControl, setServiceName, setServicesLocal, setTimeCoverage, setTimeCoverage, setUrlPath, setUserProperty, toString, transferMetadata, writeHtmlDescription, writeHtmlDescription

Methods inherited from class thredds.catalog.InvDataset

findAccess, getCatalogUrl, getHistory, getParentCatalog, getProcessing, getRestrictAccess, getRights, getSubsetUrl, getSummary, getTimeCoverage, getVariables

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

InvDatasetScan

public InvDatasetScan(InvDatasetImpl parent,
                      String name,
                      String path,
                      String scanLocation,
                      String id,
                      InvDatasetScan from)

InvDatasetScan

public InvDatasetScan(InvCatalogImpl catalog,
                      InvDatasetImpl parent,
                      String name,
                      String path,
                      String scanLocation,
                      String filter,
                      boolean addDatasetSize,
                      String addLatest,
                      boolean sortOrderIncreasing,
                      String datasetNameMatchPattern,
                      String startTimeSubstitutionPattern,
                      String duration)

Constructor. Old stuff.

Parameters:

catalog - parent catalog

parent - parent dataset

name - dataset name

path - url path

scanLocation - scan this directory

filter - RegExp match on name

addDatasetSize - add a size element

addLatest - add a latest element

sortOrderIncreasing - sort

datasetNameMatchPattern - dataset naming

startTimeSubstitutionPattern - time range using the file name

duration - time range using the file name

InvDatasetScan

public InvDatasetScan(InvCatalogImpl catalog,
                      InvDatasetImpl parent,
                      String name,
                      String path,
                      String scanLocation,
                      CrawlableDatasetFilter filter,
                      boolean addDatasetSize,
                      String addLatest,
                      boolean sortOrderIncreasing,
                      String datasetNameMatchPattern,
                      String startTimeSubstitutionPattern,
                      String duration)

Constructor. Used by InvDatasetFeatureCollection

Parameters:

catalog - parent catalog

parent - parent dataset

name - dataset name

path - url path

scanLocation - scan this directory

filter - CrawlableDatasetFilter, may be null

addDatasetSize - add a size element

addLatest - add a latest element

sortOrderIncreasing - sort

datasetNameMatchPattern - dataset naming

startTimeSubstitutionPattern - time range using the file name

duration - time range using the file name

InvDatasetScan

public InvDatasetScan(InvDatasetImpl parent,
                      String name,
                      String path,
                      String scanLocation,
                      String configClassName,
                      Object configObj,
                      CrawlableDatasetFilter filter,
                      CrawlableDatasetLabeler identifier,
                      CrawlableDatasetLabeler namer,
                      boolean addDatasetSize,
                      CrawlableDatasetSorter sorter,
                      Map<String,ProxyDatasetHandler> proxyDatasetHandlers,
                      List<DatasetEnhancer> childEnhancerList,
                      CatalogRefExpander catalogRefExpander)

Method Detail

setContext

public static void setContext(String c)

setCatalogServletName

public static void setCatalogServletName(String catServletName)

getPath

public String getPath()

getScanLocation

public String getScanLocation()

setScanLocation

public void setScanLocation(String scanLocation)

Resets the location being scanned (DO NOT USE THIS METHOD, "public by accident").

Used by DataRootHandler to allow scanning an aliased directory ("content").

Parameters:

scanLocation - the scan location.

getCrDsClassName

public String getCrDsClassName()

getCrDsConfigObj

public Object getCrDsConfigObj()

getFilter

public CrawlableDatasetFilter getFilter()

getIdentifier

public CrawlableDatasetLabeler getIdentifier()

getNamer

public CrawlableDatasetLabeler getNamer()

getSorter

public CrawlableDatasetSorter getSorter()

getProxyDatasetHandlers

public Map<String,ProxyDatasetHandler> getProxyDatasetHandlers()

getAddDatasetSize

public boolean getAddDatasetSize()

getChildEnhancerList

public List<DatasetEnhancer> getChildEnhancerList()

getCatalogRefExpander

public CatalogRefExpander getCatalogRefExpander()

isValid

public boolean isValid()

getInvalidMessage

public String getInvalidMessage()

translatePathToLocation

public String translatePathToLocation(String dsPath)

Return the CrawlableDataset path/location that corresponds to the given dataset path. The given dataset path must start with the datasetScan path for this InvDatasetScan, if not, a null is returned.

Parameters:

dsPath - a datasetScan dsPath that

Returns:

the CrawlableDataset path that corresponds to the given dataset path or null.

requestCrawlableDataset

public CrawlableDataset requestCrawlableDataset(String path)
                                         throws IOException

Return the CrawlableDataset for the given path, null if this InvDatasetScan does not allow (filters out) the requested CrawlableDataset.

This method can handle requests for regular datasets and proxy datasets.

Parameters:

path - the path of the requested CrawlableDataset

Returns:

the CrawlableDataset for the given path or null if the path is not allowed by this InvDatasetScan.

Throws:

IOException - if an I/O error occurs while locating the children datasets.

IllegalArgumentException - if the given path is not a descendant of (or the same as) this InvDatasetScan collection level.

makeCatalogForDirectory

public InvCatalogImpl makeCatalogForDirectory(String orgPath,
                                              URI catURI)

Try to build a catalog for the given path by scanning the location associated with this InvDatasetScan. The given path must start with the path of this InvDatasetScan.

Parameters:

orgPath - the part of the baseURI that is the path

catURI - the base URL for the catalog, used to resolve relative URLs.

Returns:

the catalog for this path (uses version 1.1) or null if build unsuccessful.

makeProxyDsResolverCatalog

public InvCatalogImpl makeProxyDsResolverCatalog(String path,
                                                 URI baseURI)

Try to build a catalog for the given resolver path by scanning the location associated with this InvDatasetScan. The given path must start with the path of this InvDatasetScan and refer to a resolver ProxyDatasetHandler that is part of this InvDatasetScan.

Parameters:

path - the part of the baseURI that is the path

baseURI - the base URL for the catalog, used to resolve relative URLs.

Returns:

the resolver catalog for this path (uses version 1.1) or null if build unsuccessful.

makeLatestCatalogForDirectory

public InvCatalog makeLatestCatalogForDirectory(String orgPath,
                                                URI baseURI)

Deprecated.

Instead use makeProxyDsResolver() which provides more general proxy dataset handling.

Try to build a catalog for the given path by scanning the location associated with this InvDatasetScan. The given path must start with the path of this InvDatasetScan.

Parameters:

orgPath - the part of the baseURI that is the path

baseURI - the base URL for the catalog, used to resolve relative URLs.

Returns:

the catalog for this path (uses version 1.1) or null if build unsuccessful.

isRead

public boolean isRead()

Overrides:

isRead in class InvCatalogRef

Returns:

true the referenced catalog has been read