Upgrading To TDS 4.5

New Requirements

• Java 8 is now required
• Tomcat 7 (servlet 3.0) is now required
• On the command line when starting up Tomcat, you must specify -Dtds.content.root.path=<content root> where <content root> points to the top of the content directory (This is ${tomcat}/content, not ${tomcat}/content/thredds). For example:

  -Dtds.content.root.path=/opt/tomcat-home/content/

  For example, your {tomcat}/bin/setenv.sh might look like:

  #!/bin/sh
  #
  ulimit -n 2048
  #
  CATALINA_HOME="/opt/tomcat-home"
  export CATALINA_HOME
  CATALINA_BASE="/opt/tomcat-home"
  export CATALINA_BASE
  JAVA_HOME="/opt/jdk"
  export JAVA_HOME
  #
  CONTENT_ROOT="-Dtds.content.root.path=/data/thredds/tds-dev/content"
  NORMAL="-d64 -Xmx4090m -Xms512m -server"
  MAX_PERM_GEN="-XX:MaxPermSize=256m"
  HEADLESS="-Djava.awt.headless=true"
  JAVA_PREFS_SYSTEM_ROOT="-Djava.util.prefs.systemRoot=$CATALINA_BASE/content/thredds/javaUtilPrefs -Djava.util.prefs.userRoot=$CATALINA_BASE/content/thredds/javaUtilPrefs"
  #             
  JAVA_OPTS="$CONTENT_ROOT $NORMAL $MAX_PERM_GEN $HEADLESS $JAVA_PREFS_SYSTEM_ROOT"
  export JAVA_OPTS
  

Troubleshooting:

Don't leave a space after the =

 CONTENT_ROOT="-Dtds.content.root.path=/data/thredds/tds-dev/content"

not

 CONTENT_ROOT= "-Dtds.content.root.path=/data/thredds/tds-dev/content"

If the content root is not correctly set, you will see a message like: "SEVERE: Error listenerStart" in {tomcat}/logs/catalina.out and "Error creating bean with name 'tdsContext'", and "Content directory not a directory" in {$tomcat}/logs/localhost.log.

We recommend that you use an absolute path, like /opt/tomcat/content, instead of a reletive path like "../content".

Note that your catalogs are under ${tomcat}/content/thredds but tds.content.root.path does not include the thredds subdirectory.

 

GRIB Feature Collection Changes

1. You must change your GRIB feature Collection from featureType="GRIB" to featureType="GRIB1" or featureType="GRIB2".
2. The valid options for timePartitions are directory or file.
3. On-the-fly updating of datasets is no longer supported. If you have datasets that change, you must use the TDM.
4. Datasets are automatically partitioned by GRIB reference time. If there are multiple reference times, you have the option of giving access to a TwoD dataset which has two time dimensions, reference time and valid time.

Example 4.3 catalog:

<featureCollection featureType="GRIB" name="GFS-Global_onedeg" path="grib/NCEP/GFS/Global_onedeg">
 <collection spec="/data/ldm/pub/native/grid/NCEP/GFS/Global_onedeg/.*grib2$"
   dateFormatMark="#GFS_Global_onedeg_#yyyyMMdd_HHmm"
   timePartition="1 day"
   olderThan="5 min"/>

 <update startup="nocheck" trigger="allow"/>
 <tdm startup="true" rescan="0 8,23,38,53 * * * ? *" trigger="allow"/>

 <gribConfig datasetTypes="Best LatestFile Files" />
</featureCollection>

For version 4.5 should be:

<featureCollection featureType="GRIB2" name="GFS-Global_onedeg" harvest="true" path="grib/NCEP/GFS/Global_onedeg">
  <collection spec="/data/ldm/pub/native/grid/NCEP/GFS/Global_onedeg/.*grib2$"
    dateFormatMark="#GFS_Global_onedeg_#yyyyMMdd_HHmm"
    timePartition="directory"
    olderThan="5 min"/>
  <update startup="never" trigger="allow"/>
  <tdm rewrite="test" rescan="0 8,23,38,53 * * * ? *" trigger="allow"/>
  <gribConfig datasetTypes="TwoD Best LatestFile Files" />
</featureCollection>

For complete information, see GRIB Collections and TDM.

Netcdf Subset Service Changes

1. You must change the service endpoint for Netcdf Subset Service (NCSS) in your catalogs from:

 <service name="ncss" serviceType="NetcdfSubset" base="/thredds/ncss/grid"/>

to:

 <service name="ncss" serviceType="NetcdfSubset" base="/thredds/ncss/"/>

2. Consequently, access to the NCSS has a different URL path. (If you are accessing NCSS from the HTML form, or are correctly parsing the catalog, then you dont need to do anything different)

An old (4.3) NCSS URL:

 http://thredds.ucar.edu/thredds/ncss/grid/grib/NCEP/GFS/Global_0p5deg/best/dataset.html

Should be changed in version 4.5 to:

 http://thredds.ucar.edu/thredds/ncss/grib/NCEP/GFS/Global_0p5deg/best/dataset.html
  

Note the only difference is the absence of "grid" after "thredds/ncss" in the URL.

3. The REST interface has changed. If you have been using NCSS from a script, you will need to check your URLs and parameters, and possibly change them, see the NCSS REST API. (If you are accessing NCSS from the HTML form, then you dont need to do anything different).

New Features in version 4.5

• TDS
  • Point Feature Collections (point, stations) now handled by Netcdf Subset Service.
  • Automatic Metadata Extraction from the dataset into the thredds catalogs, on point collections only but will extend to other collections.
  • NCSS Station data can output WaterML (beta).
  • NCSS Point can output NetCDF-4/CF.
• NcML
  • updated to implement full extended model.
  • logicalReduce allows you to remove length 1 dimensions [TDS-267].
  • NcML handles multichar separators.
• CDM
  • add Albers Equal Area projection
  • add Geostationary projection (Tom Rink SSEC)
  • CFPointWriter refactored, can write to netCDF-4
  • CFGridWriter2 replaces CFGridWriter
• IOSPs
• Netcdf-4 writing improved, default chunking improved.
• GRIB: yet another rewrite of Grib Collections to allow scaling to very large collections.
• Improved HDF-EOS processing.
• CF 1.6 DSG: bug in handling station profiles, multidimensional representation [TDS-513]
• MSGnavigation scan mode bug
• IOSP for SIGMET radar files (beta)
• GRIB
  • ECMWF DRS 50002 added by jkaehler@meteomatics.com
• ToolsUI
• using new Swing Look and Feel
• table columns can be set by user (upper right button on header line)
• File Chooser improvements

API changes

• Writing CF gridded data: ucar.nc2.dt.grid.CFGridWriter is deprecated in favor of ucar.nc2.dt.grid.CFGridWriter2. The API is a bit different, and simpler. Please switch to it.
• Writing Netcdf-4 files with chunking: The Nc4Chunking object changed from package ucar.nc2.jni to package ucar.nc2.write. It was substantially rewritten, and the set of standard strategies has changed.
• ucar.unidata.geoloc.ProjectionRect no longer extends java.awt.Rectangle2D. This is necessary to remove dependence of the core package on java.awt, which is not supported on Android. If you need to convert ProjectionRect into Rectangle2D, you can use:

 Rectangle2D hr = new Rectangle2D.Double();
 hr.setRect(prect.getX(), prect.getY(), prect.getWidth(), prect.getHeight());

Internal changes

• HttpClient library upgraded to 4.2, previously was 3.
• We are now using log4j2. The configuration file is in {tomcat}/webapps/thredds//WEB-INF/classes/log4j2.xml. See http://logging.apache.org/log4j/2.x/
• Most services have been converted to annotated Spring controllers. Documentation on adding your own service is coming soon.
• Point dataset unit tests upgraded to CF 1.6.
• Unit testing framework now complete; 2700+ unit tests.
• Coverity static analysis now being done. 3000+ defects eliminated, about 1000 to go.
• Code being converted to use Java 7 constructs, eg try-with-resources, generics.

This document is maintained by Unidata and was last updated Aug 2015. Send comments to THREDDS support.