ucar.nc2.ncml

Class Aggregation

java.lang.Object

ucar.nc2.ncml.Aggregation

Direct Known Subclasses:

AggregationOuterDimension, AggregationTiled, AggregationUnion

public abstract class Aggregation
extends java.lang.Object

Superclass for NcML Aggregation. An Aggregation acts as a ProxyReader for VariableDS. That, is it must implement:

   public Array read(Variable mainv);
   public Array read(Variable mainv, Section section);
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	Aggregation.Dataset Encapsolates a NetcdfFile that is a component of the aggregation.
protected class	Aggregation.DatasetProxyReader
protected static class	Aggregation.Type
protected static class	Aggregation.TypicalDataset

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	cacheDirty
protected MFileCollectionManager	datasetManager
protected java.util.List<Aggregation.Dataset>	datasets
protected java.lang.String	dateFormatMark
protected DateFormatter	dateFormatter
protected static boolean	debug
protected static boolean	debugConvert
protected static boolean	debugDateParse
protected static boolean	debugOpenFile
protected static boolean	debugProxy
protected static boolean	debugRead
protected static boolean	debugSyncDetail
protected java.lang.String	dimName
protected static DiskCache2	diskCache2
protected static java.util.concurrent.Executor	executor
protected java.util.List<Aggregation.Dataset>	explicitDatasets
protected boolean	isDate
protected static org.slf4j.Logger	logger
protected NetcdfDataset	ncDataset
protected java.lang.Object	spiObject
protected Aggregation.Type	type
protected static Aggregation.TypicalDataset	typicalDatasetMode

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Aggregation(NetcdfDataset ncd, java.lang.String dimName, Aggregation.Type type, java.lang.String recheckS) Create an Aggregation for the given NetcdfDataset.

Method Summary

Modifier and Type	Method and Description
void	addCollection(java.lang.String spec, java.lang.String olderThan)
void	addDataset(Aggregation.Dataset nested)
void	addDatasetScan(org.jdom2.Element crawlableDatasetElement, java.lang.String dirName, java.lang.String suffix, java.lang.String regexpPatternString, java.lang.String dateFormatMark, java.util.Set<NetcdfDataset.Enhance> enhanceMode, java.lang.String subdirs, java.lang.String olderThan) Add a dataset scan
void	addExplicitDataset(java.lang.String cacheName, java.lang.String location, java.lang.String id, java.lang.String ncoordS, java.lang.String coordValueS, java.lang.String sectionSpec, FileFactory reader) Add a nested dataset, specified by an explicit netcdf element.
protected abstract void	buildNetcdfDataset(CancelTask cancelTask) Call this to build the dataset objects in the NetcdfDataset
void	close()
protected Variable	findVariable(NetcdfFile ncfile, Variable mainV)
void	finish(CancelTask cancelTask)
java.util.List<Aggregation.Dataset>	getDatasets()
void	getDetailInfo(java.util.Formatter f)
java.lang.String	getDimensionName() Get dimension name to join on
java.lang.String	getFileTypeDescription()
java.lang.String	getFileTypeId()
long	getLastModified()
protected java.lang.String	getLocation()
Aggregation.Type	getType() Get type of aggregation
protected Aggregation.Dataset	getTypicalDataset() Open one of the nested datasets as a template for the aggregation dataset.
protected Aggregation.Dataset	makeDataset(MFile dset)
protected Aggregation.Dataset	makeDataset(java.lang.String cacheName, java.lang.String location, java.lang.String id, java.lang.String ncoordS, java.lang.String coordValueS, java.lang.String sectionSpec, java.util.EnumSet<NetcdfDataset.Enhance> enhance, FileFactory reader) Dataset factory, so subclasses can override
protected void	makeDatasets(CancelTask cancelTask) Make the list of Datasets, from explicit and scans.
protected void	persistRead() read info from the persistent XML file, if it exists; overridden in AggregationExisting
void	persistWrite() Allow information to be make persistent.
protected abstract void	rebuildDataset() Call this when rescan has found changed datasets
protected void	setDatasetAcquireProxy(Aggregation.Dataset typicalDataset, NetcdfDataset newds) All non-agg variables use a proxy to acquire the file before reading.
protected void	setDatasetAcquireProxy(Aggregation.DatasetProxyReader proxy, Group g)
static void	setExecutor(java.util.concurrent.Executor exec)
void	setModifications(org.jdom2.Element ncmlMods)
static void	setPersistenceCache(DiskCache2 dc)
static void	setTypicalDatasetMode(java.lang.String mode)
boolean	syncExtend() Check to see if its time to rescan directory, and if so, rescan and extend dataset if needed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

typicalDatasetMode

protected static Aggregation.TypicalDataset typicalDatasetMode

logger

protected static org.slf4j.Logger logger

diskCache2

protected static DiskCache2 diskCache2

executor

protected static java.util.concurrent.Executor executor

debug

protected static boolean debug

debugOpenFile

protected static boolean debugOpenFile

debugSyncDetail

protected static boolean debugSyncDetail

debugProxy

protected static boolean debugProxy

debugRead

protected static boolean debugRead

debugDateParse

protected static boolean debugDateParse

debugConvert

protected static boolean debugConvert

ncDataset

protected NetcdfDataset ncDataset

type

protected Aggregation.Type type

spiObject

protected java.lang.Object spiObject

explicitDatasets

protected java.util.List<Aggregation.Dataset> explicitDatasets

datasets

protected java.util.List<Aggregation.Dataset> datasets

datasetManager

protected MFileCollectionManager datasetManager

cacheDirty

protected boolean cacheDirty

dimName

protected java.lang.String dimName

dateFormatMark

protected java.lang.String dateFormatMark

isDate

protected boolean isDate

dateFormatter

protected DateFormatter dateFormatter

Constructor Detail

Aggregation

protected Aggregation(NetcdfDataset ncd,
                      java.lang.String dimName,
                      Aggregation.Type type,
                      java.lang.String recheckS)

Create an Aggregation for the given NetcdfDataset. The following addXXXX methods are called, then finish(), before the object is ready for use.

Parameters:

ncd - Aggregation belongs to this NetcdfDataset

dimName - the aggregation dimension name

type - the Aggregation.Type

recheckS - how often to check if files have changes

Method Detail

setPersistenceCache

public static void setPersistenceCache(DiskCache2 dc)

setExecutor

public static void setExecutor(java.util.concurrent.Executor exec)

setTypicalDatasetMode

public static void setTypicalDatasetMode(java.lang.String mode)

addExplicitDataset

public void addExplicitDataset(java.lang.String cacheName,
                               java.lang.String location,
                               java.lang.String id,
                               java.lang.String ncoordS,
                               java.lang.String coordValueS,
                               java.lang.String sectionSpec,
                               FileFactory reader)

Add a nested dataset, specified by an explicit netcdf element. enhance is handled by the reader, so its always false here.

Parameters:

cacheName - a unique name to use for caching

location - attribute "location" on the netcdf element

id - attribute "id" on the netcdf element

ncoordS - attribute "ncoords" on the netcdf element

coordValueS - attribute "coordValue" on the netcdf element

sectionSpec - attribute "section" on the netcdf element

reader - factory for reading this netcdf dataset

addDataset

public void addDataset(Aggregation.Dataset nested)

addDatasetScan

public void addDatasetScan(org.jdom2.Element crawlableDatasetElement,
                           java.lang.String dirName,
                           java.lang.String suffix,
                           java.lang.String regexpPatternString,
                           java.lang.String dateFormatMark,
                           java.util.Set<NetcdfDataset.Enhance> enhanceMode,
                           java.lang.String subdirs,
                           java.lang.String olderThan)

Add a dataset scan

Parameters:

crawlableDatasetElement - defines a CrawlableDataset, or null

dirName - scan this directory

suffix - filter on this suffix (may be null)

regexpPatternString - include if full name matches this regular expression (may be null)

dateFormatMark - create dates from the filename (may be null)

enhanceMode - how should files be enhanced

subdirs - equals "false" if should not descend into subdirectories

olderThan - files must be older than this time (now - lastModified >= olderThan); must be a time unit, may ne bull

addCollection

public void addCollection(java.lang.String spec,
                          java.lang.String olderThan)
                   throws java.io.IOException

Throws:

java.io.IOException

setModifications

public void setModifications(org.jdom2.Element ncmlMods)

getType

public Aggregation.Type getType()

Get type of aggregation

Returns:

type of aggregation

getDimensionName

public java.lang.String getDimensionName()

Get dimension name to join on

Returns:

dimension name or null if type union/tiled

getLocation

protected java.lang.String getLocation()

close

public void close()
           throws java.io.IOException

Throws:

java.io.IOException

syncExtend

public boolean syncExtend()
                   throws java.io.IOException

Check to see if its time to rescan directory, and if so, rescan and extend dataset if needed. Note that this just calls sync(), so structural metadata may be modified (!!)

Returns:

true if directory was rescanned and dataset may have been updated

Throws:

java.io.IOException - on io error

getLastModified

public long getLastModified()

getFileTypeId

public java.lang.String getFileTypeId()

getFileTypeDescription

public java.lang.String getFileTypeDescription()

buildNetcdfDataset

protected abstract void buildNetcdfDataset(CancelTask cancelTask)
                                    throws java.io.IOException

Call this to build the dataset objects in the NetcdfDataset

Parameters:

cancelTask - maybe cancel

Throws:

java.io.IOException - on read error

rebuildDataset

protected abstract void rebuildDataset()
                                throws java.io.IOException

Call this when rescan has found changed datasets

Throws:

java.io.IOException - on read error

persistWrite

public void persistWrite()
                  throws java.io.IOException

Allow information to be make persistent. Overridden in AggregationExisting

Throws:

java.io.IOException - on error

persistRead

protected void persistRead()

read info from the persistent XML file, if it exists; overridden in AggregationExisting

getDetailInfo

public void getDetailInfo(java.util.Formatter f)

finish

public void finish(CancelTask cancelTask)
            throws java.io.IOException

Throws:

java.io.IOException

getDatasets

public java.util.List<Aggregation.Dataset> getDatasets()

makeDatasets

protected void makeDatasets(CancelTask cancelTask)
                     throws java.io.IOException

Make the list of Datasets, from explicit and scans.

Parameters:

cancelTask - user can cancel

Throws:

java.io.IOException - on i/o error

getTypicalDataset

protected Aggregation.Dataset getTypicalDataset()
                                         throws java.io.IOException

Open one of the nested datasets as a template for the aggregation dataset.

Returns:

a typical Dataset

Throws:

java.io.IOException - if there are no datasets

makeDataset

protected Aggregation.Dataset makeDataset(java.lang.String cacheName,
                                          java.lang.String location,
                                          java.lang.String id,
                                          java.lang.String ncoordS,
                                          java.lang.String coordValueS,
                                          java.lang.String sectionSpec,
                                          java.util.EnumSet<NetcdfDataset.Enhance> enhance,
                                          FileFactory reader)

Dataset factory, so subclasses can override

Parameters:

cacheName - a unique name to use for caching

location - attribute "location" on the netcdf element

id - attribute "id" on the netcdf element

ncoordS - attribute "ncoords" on the netcdf element

coordValueS - attribute "coordValue" on the netcdf element

sectionSpec - attribute "sectionSpec" on the netcdf element

enhance - open dataset in enhance mode NOT USED

reader - factory for reading this netcdf dataset

Returns:

a Aggregation.Dataset

makeDataset

protected Aggregation.Dataset makeDataset(MFile dset)

setDatasetAcquireProxy

protected void setDatasetAcquireProxy(Aggregation.Dataset typicalDataset,
                                      NetcdfDataset newds)
                               throws java.io.IOException

All non-agg variables use a proxy to acquire the file before reading. If the variable is caching, read data into cache now. If not caching, VariableEnhanced.setProxyReader() is called.

Parameters:

typicalDataset - read from a "typical dataset"

newds - containing dataset

Throws:

java.io.IOException - on i/o error

setDatasetAcquireProxy

protected void setDatasetAcquireProxy(Aggregation.DatasetProxyReader proxy,
                                      Group g)
                               throws java.io.IOException

Throws:

java.io.IOException

findVariable

protected Variable findVariable(NetcdfFile ncfile,
                                Variable mainV)