Reading the Zarr data model

Zarr

As of version 5.5.1, the netCDF-Java library provides read-only support for the Zarr v2 data model. Any dataset that adheres to the v2 spec can be read into a NetcdfFile object, as long as the following is true:

• all filters and compressors used by the dataset must be known to the netCDF-Java library (see Filters)
• the underlying storage of the dataset must be a directory store, zip store, or object store

Enabling Zarr support

To use Zarr in the netCDF-Java library, you must include the cdm-zarr module in your netCDF-Java build. See here for more information on including optional modules.

How to read a Zarr dataset

Reading a Zarr dataset is syntactically the same as reading a netCDF file:

// a local file path
NetcdfFile directoryStoreZarr = NetcdfFiles.open(pathToDirectoryStore);
// a local file path + '.zip'
NetcdfFile zipdirectoryStoreZarr = NetcdfFiles.open(pathToZipStore);
// an object store path, sarting with 'cdms3:' and ending with 'delimiter=' + the store delimiter
NetcdfFile objectStoreZarr = NetcdfFiles.open(pathToObjectStore);

If the file is a legal Zarr dataset, the library will map it to a NetcdfFile object for reading. See reading CDM files for more examples on accessing data once the NetcdfFile object is returned.

Filters

As of netCDF-Java version 5.5.1, a ucar.nc2.filter package is included, that provides a suite of implemented filters and compressors, as well as a mechanism for user-supplied filters. This package is used by both the Zarr and HDF5 IOSPs, and is available for public use.

The current list of filters included natively in the netCDF-Java library is:

• Deflate (zlib)
• Shuffle
• 32-bit Checksum (CRC, Fletcher, and Adler)
• ScaleOffset

This list is still expanding, but if the filter you are looking for is not provided at this time, you are able to provide it yourself (See Implementing a Filter) for details.)

Implementing a Filter

To add a user-supplied Filter to the netDF-Java library, you will have to provide two classes:

• A class that extends the ucar.nc2.filter.Filter abstract class
• A class that implements the ucar.nc2.filter.FilterProvider interface

Once implemented, you will need to include these classes as JAR files in your classpath. See here for more information.

Filter implementation

To implement a user-supplied filter, you will need to extend the abstract ucar.nc2.filter.Filter class, and provide implementations for the following methods:

• getName returns a String identifier for the filter (see note below on filter names)
• getId returns an int identifier for the filter (see note below on filter ids)
• encode takes a byte[] of unfiltered data and returns a byte[] of filtered data
• decode takes a byte[] of filtered data and returns a byte[] of unfiltered data

Your Filter class should look something like this:

byte[] dataOut = null;
public class MyFilter extends Filter {

  static final String name = "myFilter";
  static final int id = 32768;

  @Override
  public String getName() {
    return name;
  }

  @Override
  public int getId() {
    return id;
  }

  @Override
  public byte[] encode(byte[] dataIn) throws IOException {
    // your encoding implementation here
    return dataOut;
  }

  @Override
  public byte[] decode(byte[] dataIn) throws IOException {
    // your decoding implementation here
    return dataOut;
  }
}

FilterProvider implementation

For the netCDF-Java library to find your Filter implementation, you will need to provide a FilterProvider as well.

public class MyFilterProvider implements FilterProvider {

  @Override
  public String getName() {
    // returns a string identifier for your filter
    return MyFilter.name; // should match name of your Filter class
  }

  @Override
  public int getId() {
    // returns a numeric identifier for your filter
    return MyFilter.id; // should match id of your Filter class
  }

  @Override
  public Filter create(Map<String, Object> properties) {
    return new MyFilter(properties); // return an instance of your filter
  }
}

There are two more methods in the FilterProvider interface: the canProvide methods. By default, these methods work as follows:

• boolean canProvide(String name) returns true if the string returned by getName() matches the string passed to the method
• boolean canProvide(int id) returns true if the int returned by getId() matches the int passed to the method

It is unlikely that you would want to override these methods, as the netCDF-Java IOSPs look for the correct filter implementations for a dataset by either name or numeric id. However, it is possible to write your own implementation of canProvide; for example, the following FilterProvider returns an instance of a DefaultFilter regardless of the name or id provided.

public class DefaultFilterProvider implements FilterProvider {

  @Override
  public String getName() {
    return DefaultFilter.name;
  }

  @Override
  public int getId() {
    return DefaultFilter.id;
  }

  @Override
  public boolean canProvide(String name) {
    return true;
  }

  @Override
  public boolean canProvide(int id) {
    return true;
  }

  @Override
  public Filter create(Map<String, Object> properties) {
    return new DefaultFilter(properties); // return an instance of your filter
  }
}