NetCDF File Writing (version 4.3+)

You can programmatically create, edit, and add data to netCDF-3 and netCDF-4 files, using NetcdfFormatWriter. To copy an existing CDM dataset, you can use the CDM nccopy application. By combining nccopy and NcML, you can copy just parts of an existing dataset, as well as make modifications to it with NcML.

Requirements to Write netCDF files

CDM version 4.3 and above supports writing netCDF files. Writing netCDF-3 files is included in the core netCDF-Java API.

To write to netCDF-4:

  • include the netcdf4 module in your build (see here for more information)
  • install the netCDF-4 C library on your machine

Creating a new netCDF-3 file

To create a new netCDF-3 file, use NetcdfFormatWriter.createNewNetcdf3:

// 1) Create a new netCDF-3 file builder with the given path and file name
NetcdfFormatWriter.Builder builder = NetcdfFormatWriter.createNewNetcdf3(pathAndFilenameStr);

// 2) Create two Dimensions, named lat and lon, of lengths 64 and 129 respectively, and add them to the root group
Dimension latDim = builder.addDimension("lat", 64);
Dimension lonDim = builder.addDimension("lon", 128);

List<Dimension> dims = new ArrayList<Dimension>();
dims.add(latDim);
dims.add(lonDim);

// 3) Create a builder for a Variable named temperature, or type double, with shape (lat, lon), and add to the root
// group
Variable.Builder t = builder.addVariable("temperature", ArrayType.DOUBLE, dims);

// 4) Add a string Attribute to the temperature Variable, with name units and value K
t.addAttribute(new Attribute("units", "K"));

// 5) Create a 1D integer Array Attribute using Attribute.Builder,with name scale and value (1,2,3)
// and add to the temperature Variables
Array data = Arrays.factory(ArrayType.INT, new int[] {3}, new int[] {1, 2, 3});
t.addAttribute(Attribute.builder("scale").setArrayValues(data).build());

// 6) Create a Variable named svar or type character with length 80
Dimension svar_len = builder.addDimension("svar_len", 80);
builder.addVariable("svar", ArrayType.CHAR, "svar_len");

// 7) Create a 2D Variable names names of type character with length 80
Dimension names = builder.addDimension("names", 3);
builder.addVariable("names", ArrayType.CHAR, "names svar_len");

// 8) Create a scalar Variable names scalar or type double.
// Note that the empty ArrayList means that it is a scalar, i.e. has no dimensions
builder.addVariable("scalar", ArrayType.DOUBLE, new ArrayList<Dimension>());

// 9) Create various global Attributes of different types
builder.addAttribute(new Attribute("versionStr", "v"));
builder.addAttribute(new Attribute("versionD", 1.2));
builder.addAttribute(new Attribute("versionF", (float) 1.2));
builder.addAttribute(new Attribute("versionI", 1));
builder.addAttribute(new Attribute("versionS", (short) 2));
builder.addAttribute(new Attribute("versionB", (byte) 3));

// 10) Now that the metadata (Dimensions, Variables, and Attributes) is added to the builder, build the writer
// At this point, the (empty) file will be written to disk, and the metadata is fixed and cannot be changed or
// added.
try (NetcdfFormatWriter writer = builder.build()) {
  // write data
} catch (IOException e) {
  logger.log(yourCreateNetcdfFileErrorMsgTxt);
}

The above example code produces a file that looks like:

netcdf C:/tmp/testWrite.nc {
 dimensions:
 lat = 64;
 lon = 128;
 svar_len = 80;
 names = 3;
 variables:
   double temperature(lat=64, lon=128);
    :units = "K";
    :scale = 1, 2, 3; // int
   char svar(svar_len=80);
   char names(names=3, svar_len=80);
   double scalar;
    
    // global attributes:
   :yo = "face";
   :versionD = 1.2; // double
   :versionF = 1.2f; // float
   :versionI = 1; // int
   :versionS = 2S; // short
   :versionB = 3B; // byte
   }

By default, The fill property is set to false. When fill = true, all values are written twice: first with the fill value, then with the data values. If you know you will write all the data, you do not need to use fill. If you don’t know if all the data will be written, turning fill on ensures that any values not written will have the fill value. Otherwise, those values will be undefined: possibly zero, or possibly garbage. To enable fill:

builder.setFill(true);

Open an existing file for writing

To open an existing CDM file for writing:

NetcdfFormatWriter updater = NetcdfFormatWriter.openExisting(filePathStr).build();

Writing data to a new or existing file

In both cases (new and existing files) the data writing is the same.

Note: As of netCDF-Java 6.0, all objects used by read and writes are immutable, to ensure you data is not changed. The ucar.ma2.Array has been deprecated and replaced by the immutable ucar.array.Array class, which does not support data manipulation (e.g. permute, transpose, etc.). This means that users must handle data manipulation prior to beginning write operations, as the netCDF-Java library expects to be given a ucar.array.Array object that is ready to be written.

The following examples demonstrate several ways to write data to an opened file.

1) Writing numeric data:

// 1) Create a primitive array of the same shape as temperature(lat, lon) and fill it with some values
Variable v = writer.findVariable(varName);
int[] shape = v.getShape();
double[] a = new double[shape[0] * shape[1]];
for (int i = 0; i < a.length; i++) {
  a[i] = (i * 1000000);
}

// 2) Create an immutable Array<Double> from the primitive array
Array<Double> A = Arrays.factory(ArrayType.DOUBLE, shape, a);

// 3) Or create an evenly spaced Array of doubles
// public Array<T> makeArray(ArrayType type, int npts, double start, double incr, int... shape)
Array<Double> A2 = Arrays.makeArray(ArrayType.DOUBLE, 20, 0, 5, 4, 5);

// 4) Write the data to the temperature Variable, with origin all zeros.
// origin array is converted to an immutable Index with `Index.of`
int[] origin = new int[2]; // initialized to zeros
try {
  writer.write(v, Index.of(origin), A);
} catch (IOException | InvalidRangeException e) {
  logger.log(yourWriteNetcdfFileErrorMsgTxt);
}

2) Writing char data as a String:

// write char variable as String
Variable v = writer.findVariable(varName);

// 1) Create an immutable Array<char>> from primitive strings
Array<Character> ac = Arrays.factory(ArrayType.CHAR, new int[] {someStringValue.length()},
    someStringValue.toCharArray());

// 2) Write the data. The origin parameter is initilized with zeros using the rank of the variable
try {
  writer.write(v, Index.ofRank(v.getRank()), ac);
} catch (IOException | InvalidRangeException e) {
  logger.log(yourWriteNetcdfFileErrorMsgTxt);
}

Note that because Arrays are immutable, writing record variables (unlimited dimensions) will require multiple write operations.

Writing to a netCDF-4 file with compression (version 4.5)

The main use of netCDF-4 is to get the performance benefits from compression, and possibly from chunking (why it matters). By default, the Java library will use the default chunking algorithm to write chunked and compressed netcdf-4 files. To control chunking and compression settings, you must create a Nc4Chunking object and pass it into NetcdfFormatWriter.createNewNetcdf4:

// 1) Create an Nc4Chunking object
Nc4Chunking.Strategy type = strategyType;
int deflateLevel = dl;
boolean shuffle = shfl;
Nc4Chunking chunker = Nc4ChunkingStrategy.factory(type, deflateLevel, shuffle);

// 3) Create a new netCDF-4 file builder with the given path and file name and Nc4Chunking object
NetcdfFileFormat format = nc4format;
NetcdfFormatWriter.Builder builder =
    NetcdfFormatWriter.createNewNetcdf4(NetcdfFileFormat.NETCDF4, outFilePath, chunker);

// 4) Create a NetcdfCopier and pass it the opened file and NetcdfFormatWriter.Builder
NetcdfCopier copier = NetcdfCopier.create(inFile, builder);

try {
  // 5) Write new file
  copier.write(null);
  // do stuff with newly create chunked and compressed file
} catch (IOException e) {
  logger.log(yourWriteNetcdfFileErrorMsgTxt);
}

See here for more details on Nc4Chunking.