NetCDF  4.8.1
NetCDF File and Data I/O

NetCDF opens datasets as files or remote access URLs. More...

Functions

int nc__create (const char *path, int cmode, size_t initialsz, size_t *chunksizehintp, int *ncidp)
 Create a netCDF file with some extra parameters controlling classic file caching. More...
 
int nc__enddef (int ncid, size_t h_minfree, size_t v_align, size_t v_minfree, size_t r_align)
 
int nc__open (const char *path, int omode, size_t *chunksizehintp, int *ncidp)
 
int nc_abort (int ncid)
 
int nc_close (int ncid)
 
int nc_close_memio (int ncid, NC_memio *memio)
 
int nc_create (const char *path, int cmode, int *ncidp)
 
int nc_create_mem (const char *path, int mode, size_t initialsize, int *ncidp)
 
int nc_create_par (const char *path, int cmode, MPI_Comm comm, MPI_Info info, int *ncidp)
 Create a netCDF file for parallel I/O. More...
 
int nc_create_par_fortran (const char *path, int cmode, int comm, int info, int *ncidp)
 Create a netCDF file for parallel access from the Fortran API. More...
 
int nc_def_user_format (int mode_flag, NC_Dispatch *dispatch_table, char *magic_number)
 Add handling of user-defined format. More...
 
int nc_enddef (int ncid)
 
int nc_inq (int ncid, int *ndimsp, int *nvarsp, int *nattsp, int *unlimdimidp)
 
int nc_inq_format (int ncid, int *formatp)
 
int nc_inq_format_extended (int ncid, int *formatp, int *modep)
 
int nc_inq_path (int ncid, size_t *pathlen, char *path)
 
int nc_inq_type (int ncid, nc_type xtype, char *name, size_t *size)
 
int nc_inq_user_format (int mode_flag, NC_Dispatch **dispatch_table, char *magic_number)
 Inquire about user-defined format. More...
 
int nc_open (const char *path, int omode, int *ncidp)
 Open an existing netCDF file. More...
 
int nc_open_mem (const char *path, int omode, size_t size, void *memory, int *ncidp)
 
int nc_open_memio (const char *path, int omode, NC_memio *params, int *ncidp)
 
int nc_open_par (const char *path, int omode, MPI_Comm comm, MPI_Info info, int *ncidp)
 Open an existing netCDF file for parallel I/O. More...
 
int nc_open_par_fortran (const char *path, int omode, int comm, int info, int *ncidp)
 This is the same as nc_open_par(), but accepts the MPI comm/info as integers. More...
 
int nc_redef (int ncid)
 
int nc_set_fill (int ncid, int fillmode, int *old_modep)
 
int nc_sync (int ncid)
 
int nc_var_par_access (int ncid, int varid, int par_access)
 

Detailed Description

NetCDF opens datasets as files or remote access URLs.

A netCDF dataset that has not yet been opened can only be referred to by its dataset name. Once a netCDF dataset is opened, it is referred to by a netCDF ID, which is a small non-negative integer returned when you create or open the dataset. A netCDF ID is much like a file descriptor in C or a logical unit number in FORTRAN. In any single program, the netCDF IDs of distinct open netCDF datasets are distinct. A single netCDF dataset may be opened multiple times and will then have multiple distinct netCDF IDs; however at most one of the open instances of a single netCDF dataset should permit writing. When an open netCDF dataset is closed, the ID is no longer associated with a netCDF dataset.

Functions that deal with the netCDF library include:

The operations supported on a netCDF dataset as a single object are:

Function Documentation

◆ nc__create()

int nc__create ( const char *  path,
int  cmode,
size_t  initialsz,
size_t *  chunksizehintp,
int *  ncidp 
)

Create a netCDF file with some extra parameters controlling classic file caching.

Like nc_create(), this function creates a netCDF file.

Parameters
pathThe file name of the new netCDF dataset.
cmodeThe creation mode flag, the same as in nc_create().
initialszOn some systems, and with custom I/O layers, it may be advantageous to set the size of the output file at creation time. This parameter sets the initial size of the file at creation time. This only applies to classic CDF-1, 2, and 5 files. The special value NC_SIZEHINT_DEFAULT (which is the value 0), lets the netcdf library choose a suitable initial size.
chunksizehintpA pointer to the chunk size hint, which controls a space versus time tradeoff, memory allocated in the netcdf library versus number of system calls. Because of internal requirements, the value may not be set to exactly the value requested. The actual value chosen is returned by reference. Using a NULL pointer or having the pointer point to the value NC_SIZEHINT_DEFAULT causes the library to choose a default. How the system chooses the default depends on the system. On many systems, the "preferred I/O block size" is available from the stat() system call, struct stat member st_blksize. If this is available it is used. Lacking that, twice the system pagesize is used. Lacking a call to discover the system pagesize, we just set default bufrsize to 8192. The bufrsize is a property of a given open netcdf descriptor ncid, it is not a persistent property of the netcdf dataset. This only applies to classic files.
ncidpPointer to location where returned netCDF ID is to be stored.
Note
This function uses the same return codes as the nc_create() function.
Returns
NC_NOERR No error.
NC_ENOMEM System out of memory.
NC_EHDFERR HDF5 error (netCDF-4 files only).
NC_EFILEMETA Error writing netCDF-4 file-level metadata in HDF5 file. (netCDF-4 files only).
NC_EDISKLESS if there was an error in creating the in-memory file.

Examples

In this example we create a netCDF dataset named foo_large.nc; we want the dataset to be created in the current directory only if a dataset with that name does not already exist. We also specify that bufrsize and initial size for the file.

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
int intialsz = 2048;
int *bufrsize;
...
*bufrsize = 1024;
status = nc__create("foo.nc", NC_NOCLOBBER, initialsz, bufrsize, &ncid);
if (status != NC_NOERR) handle_error(status);
Author
Glenn Davis

Definition at line 461 of file dfile.c.

◆ nc__enddef()

int nc__enddef ( int  ncid,
size_t  h_minfree,
size_t  v_align,
size_t  v_minfree,
size_t  r_align 
)

Leave define mode with performance tuning

The function nc__enddef takes an open netCDF dataset out of define mode. The changes made to the netCDF dataset while it was in define mode are checked and committed to disk if no problems occurred. Non-record variables may be initialized to a "fill value" as well with nc_set_fill(). The netCDF dataset is then placed in data mode, so variable data can be read or written.

This call may involve copying data under some circumstances. For a more extensive discussion see File Structure and Performance.

Warning
This function exposes internals of the netcdf version 1 file format. Users should use nc_enddef() in most circumstances. This function may not be available on future netcdf implementations.

The classic netcdf file format has three sections, the "header" section, the data section for fixed size variables, and the data section for variables which have an unlimited dimension (record variables).

The header begins at the beginning of the file. The index (offset) of the beginning of the other two sections is contained in the header. Typically, there is no space between the sections. This causes copying overhead to accrue if one wishes to change the size of the sections, as may happen when changing names of things, text attribute values, adding attributes or adding variables. Also, for buffered i/o, there may be advantages to aligning sections in certain ways.

The minfree parameters allow one to control costs of future calls to nc_redef, nc_enddef() by requesting that minfree bytes be available at the end of the section.

The align parameters allow one to set the alignment of the beginning of the corresponding sections. The beginning of the section is rounded up to an index which is a multiple of the align parameter. The flag value ALIGN_CHUNK tells the library to use the bufrsize (see above) as the align parameter. It has nothing to do with the chunking (multidimensional tiling) features of netCDF-4.

The file format requires mod 4 alignment, so the align parameters are silently rounded up to multiples of 4. The usual call,

nc_enddef(ncid);

is equivalent to

nc__enddef(ncid, 0, 4, 0, 4);

The file format does not contain a "record size" value, this is calculated from the sizes of the record variables. This unfortunate fact prevents us from providing minfree and alignment control of the "records" in a netcdf file. If you add a variable which has an unlimited dimension, the third section will always be copied with the new variable added.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
h_minfreeSets the pad at the end of the "header" section.
v_alignControls the alignment of the beginning of the data section for fixed size variables.
v_minfreeSets the pad at the end of the data section for fixed size variables.
r_alignControls the alignment of the beginning of the data section for variables which have an unlimited dimension (record variables).
Returns
NC_NOERR No error.
NC_EBADID Invalid ncid passed.

Definition at line 1109 of file dfile.c.

◆ nc__open()

int nc__open ( const char *  path,
int  omode,
size_t *  chunksizehintp,
int *  ncidp 
)

Open a netCDF file with extra performance parameters for the classic library.

Parameters
pathFile name for netCDF dataset to be opened. When DAP support is enabled, then the path may be an OPeNDAP URL rather than a file path.
omodeThe open mode flag may include NC_WRITE (for read/write access) and NC_SHARE as in nc_open().
chunksizehintpA size hint for the classic library. Only applies to classic files. See below for more information.
ncidpPointer to location where returned netCDF ID is to be stored.

The chunksizehintp Parameter

The argument referenced by bufrsizehintp controls a space versus time tradeoff, memory allocated in the netcdf library versus number of system calls.

Because of internal requirements, the value may not be set to exactly the value requested. The actual value chosen is returned by reference.

Using a NULL pointer or having the pointer point to the value NC_SIZEHINT_DEFAULT causes the library to choose a default. How the system chooses the default depends on the system. On many systems, the "preferred I/O block size" is available from the stat() system call, struct stat member st_blksize. If this is available it is used. Lacking that, twice the system pagesize is used.

Lacking a call to discover the system pagesize, we just set default bufrsize to 8192.

The bufrsize is a property of a given open netcdf descriptor ncid, it is not a persistent property of the netcdf dataset.

Returns
NC_NOERR No error.
NC_ENOMEM Out of memory.
NC_EHDFERR HDF5 error. (NetCDF-4 files only.)
NC_EDIMMETA Error in netCDF-4 dimension metadata. (NetCDF-4 files only.)

Definition at line 712 of file dfile.c.

◆ nc_abort()

int nc_abort ( int  ncid)

No longer necessary for user to invoke manually.

Warning
Users no longer need to call this function since it is called automatically by nc_close() in case the dataset is in define mode and something goes wrong with committing the changes. The function nc_abort() just closes the netCDF dataset, if not in define mode. If the dataset is being created and is still in define mode, the dataset is deleted. If define mode was entered by a call to nc_redef(), the netCDF dataset is restored to its state before definition mode was entered and the dataset is closed.
Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
Returns
NC_NOERR No error.

Example

Here is an example using nc_abort to back out of redefinitions of a dataset named foo.nc:

#include <netcdf.h>
...
int ncid, status, latid;
...
status = nc_open("foo.nc", NC_WRITE, &ncid);
if (status != NC_NOERR) handle_error(status);
...
status = nc_redef(ncid);
if (status != NC_NOERR) handle_error(status);
...
status = nc_def_dim(ncid, "lat", 18L, &latid);
if (status != NC_NOERR) {
handle_error(status);
status = nc_abort(ncid);
if (status != NC_NOERR) handle_error(status);
}

Definition at line 1238 of file dfile.c.

◆ nc_close()

int nc_close ( int  ncid)

Close an open netCDF dataset

If the dataset in define mode, nc_enddef() will be called before closing. (In this case, if nc_enddef() returns an error, nc_abort() will automatically be called to restore the dataset to the consistent state before define mode was last entered.) After an open netCDF dataset is closed, its netCDF ID may be reassigned to the next netCDF dataset that is opened or created.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
Returns
NC_NOERR No error.
NC_EBADID Invalid id passed.
NC_EBADGRPID ncid did not contain the root group id of this file. (NetCDF-4 only).

Example

Here is an example using nc_close to finish the definitions of a new netCDF dataset named foo.nc and release its netCDF ID:

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_create("foo.nc", NC_NOCLOBBER, &ncid);
if (status != NC_NOERR) handle_error(status);
... create dimensions, variables, attributes
status = nc_close(ncid);
if (status != NC_NOERR) handle_error(status);

Definition at line 1291 of file dfile.c.

◆ nc_close_memio()

int nc_close_memio ( int  ncid,
NC_memio memio 
)

Do a normal close (see nc_close()) on an in-memory dataset, then return a copy of the final memory contents of the dataset.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
memioa pointer to an NC_memio object into which the final valid memory size and memory will be returned.
Returns
NC_NOERR No error.
NC_EBADID Invalid id passed.
NC_ENOMEM Out of memory.
NC_EDISKLESS if the file was not created as an inmemory file.
NC_EBADGRPID ncid did not contain the root group id of this file. (NetCDF-4 only).

Example

Here is an example using nc_close_mem to finish the definitions of a new netCDF dataset named foo.nc, return the final memory, and release its netCDF ID:

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
NC_memio finalmem;
size_t initialsize = 65000;
...
status = nc_create_mem("foo.nc", NC_NOCLOBBER, initialsize, &ncid);
if (status != NC_NOERR) handle_error(status);
... create dimensions, variables, attributes
status = nc_close_memio(ncid,&finalmem);
if (status != NC_NOERR) handle_error(status);

Definition at line 1350 of file dfile.c.

◆ nc_create()

int nc_create ( const char *  path,
int  cmode,
int *  ncidp 
)

Create a new netCDF file.

This function creates a new netCDF dataset, returning a netCDF ID that can subsequently be used to refer to the netCDF dataset in other netCDF function calls. The new netCDF dataset opened for write access and placed in define mode, ready for you to add dimensions, variables, and attributes.

Parameters
pathThe file name of the new netCDF dataset.
cmodeThe creation mode flag. The following flags are available: NC_CLOBBER (overwrite existing file), NC_NOCLOBBER (do not overwrite existing file), NC_SHARE (limit write caching - netcdf classic files only), NC_64BIT_OFFSET (create 64-bit offset file), NC_64BIT_DATA (alias NC_CDF5) (create CDF-5 file), NC_NETCDF4 (create netCDF-4/HDF5 file), NC_CLASSIC_MODEL (enforce netCDF classic mode on netCDF-4/HDF5 files), NC_DISKLESS (store data in memory), and NC_PERSIST (force the NC_DISKLESS data from memory to a file), NC_MMAP (use MMAP for NC_DISKLESS instead of NC_INMEMORY – deprecated). See discussion below.
ncidpPointer to location where returned netCDF ID is to be stored.

The cmode Flag

The cmode flag is used to control the type of file created, and some aspects of how it may be used.

Setting NC_NOCLOBBER means you do not want to clobber (overwrite) an existing dataset; an error (NC_EEXIST) is returned if the specified dataset already exists.

The NC_SHARE flag is appropriate when one process may be writing the dataset and one or more other processes reading the dataset concurrently; it means that dataset accesses are not buffered and caching is limited. Since the buffering scheme is optimized for sequential access, programs that do not access data sequentially may see some performance improvement by setting the NC_SHARE flag. This flag is ignored for netCDF-4 files.

Setting NC_64BIT_OFFSET causes netCDF to create a 64-bit offset format file, instead of a netCDF classic format file. The 64-bit offset format imposes far fewer restrictions on very large (i.e. over 2 GB) data files. See Large File Support.

Setting NC_64BIT_DATA (alias NC_CDF5) causes netCDF to create a CDF-5 file format that supports large files (i.e. over 2GB) and large variables (over 2B array elements.). See Large File Support.

A zero value (defined for convenience as NC_CLOBBER) specifies the default behavior: overwrite any existing dataset with the same file name and buffer and cache accesses for efficiency. The dataset will be in netCDF classic format. See NetCDF Classic Format Limitations.

Setting NC_NETCDF4 causes netCDF to create a HDF5/NetCDF-4 file.

Setting NC_CLASSIC_MODEL causes netCDF to enforce the classic data model in this file. (This only has effect for netCDF-4/HDF5 files, as CDF-1, 2 and 5 files always use the classic model.) When used with NC_NETCDF4, this flag ensures that the resulting netCDF-4/HDF5 file may never contain any new constructs from the enhanced data model. That is, it cannot contain groups, user defined types, multiple unlimited dimensions, or new atomic types. The advantage of this restriction is that such files are guaranteed to work with existing netCDF software.

Setting NC_DISKLESS causes netCDF to create the file only in memory and to optionally write the final contents to the correspondingly named disk file. This allows for the use of files that have no long term purpose. Operating on an existing file in memory may also be faster. The decision on whether or not to "persist" the memory contents to a disk file is described in detail in the file docs/inmemory.md, which is definitive. By default, closing a diskless fill will cause it's contents to be lost.

If NC_DISKLESS is going to be used for creating a large classic file, it behooves one to use nc__create and specify an appropriately large value of the initialsz parameter to avoid to many extensions to the in-memory space for the file. This flag applies to files in classic format and to file in extended format (netcdf-4).

Note that nc_create(path,cmode,ncidp) is equivalent to the invocation of nc__create(path,cmode,NC_SIZEHINT_DEFAULT,NULL,ncidp).

Returns
NC_NOERR No error.
NC_EEXIST Specifying a file name of a file that exists and also specifying NC_NOCLOBBER.
NC_EPERM Attempting to create a netCDF file in a directory where you do not have permission to create files.
NC_ENOMEM System out of memory.
NC_ENFILE Too many files open.
NC_EHDFERR HDF5 error (netCDF-4 files only).
NC_EFILEMETA Error writing netCDF-4 file-level metadata in HDF5 file. (netCDF-4 files only).
NC_EDISKLESS if there was an error in creating the in-memory file.
Note
When creating a netCDF-4 file HDF5 error reporting is turned off, if it is on. This doesn't stop the HDF5 error stack from recording the errors, it simply stops their display to the user through stderr.

Examples

In this example we create a netCDF dataset named foo.nc; we want the dataset to be created in the current directory only if a dataset with that name does not already exist:

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_create("foo.nc", NC_NOCLOBBER, &ncid);
if (status != NC_NOERR) handle_error(status);

In this example we create a netCDF dataset named foo_large.nc. It will be in the 64-bit offset format.

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_create("foo_large.nc", NC_NOCLOBBER|NC_64BIT_OFFSET, &ncid);
if (status != NC_NOERR) handle_error(status);

In this example we create a netCDF dataset named foo_HDF5.nc. It will be in the HDF5 format.

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_create("foo_HDF5.nc", NC_NOCLOBBER|NC_NETCDF4, &ncid);
if (status != NC_NOERR) handle_error(status);

In this example we create a netCDF dataset named foo_HDF5_classic.nc. It will be in the HDF5 format, but will not allow the use of any netCDF-4 advanced features. That is, it will conform to the classic netCDF-3 data model.

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_create("foo_HDF5_classic.nc", NC_NOCLOBBER|NC_NETCDF4|NC_CLASSIC_MODEL, &ncid);
if (status != NC_NOERR) handle_error(status);

In this example we create an in-memory netCDF classic dataset named diskless.nc whose content will be lost when nc_close() is called.

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_create("diskless.nc", NC_DISKLESS, &ncid);
if (status != NC_NOERR) handle_error(status);

In this example we create a in-memory netCDF classic dataset named diskless.nc and specify that it should be made persistent in a file named diskless.nc when nc_close() is called.

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_create("diskless.nc", NC_DISKLESS|NC_PERSIST, &ncid);
if (status != NC_NOERR) handle_error(status);

A variant of nc_create(), nc__create() (note the double underscore) allows users to specify two tuning parameters for the file that it is creating.

Definition at line 389 of file dfile.c.

◆ nc_create_mem()

int nc_create_mem ( const char *  path,
int  mode,
size_t  initialsize,
int *  ncidp 
)

Create a netCDF file with the contents stored in memory.

Parameters
pathMust be non-null, but otherwise only used to set the dataset name.
modethe mode flags; Note that this procedure uses a limited set of flags because it forcibly sets NC_INMEMORY.
initialsize(advisory) size to allocate for the created file
ncidpPointer to location where returned netCDF ID is to be stored.
Returns
NC_NOERR No error.
NC_ENOMEM Out of memory.
NC_EDISKLESS diskless io is not enabled for fails.
NC_EINVAL, etc. other errors also returned by nc_open.

Examples

In this example we use nc_create_mem() to create a classic netCDF dataset named foo.nc. The initial size is set to 4096.

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
int mode = 0;
size_t initialsize = 4096;
...
status = nc_create_mem("foo.nc", mode, initialsize, &ncid);
if (status != NC_NOERR) handle_error(status);

Definition at line 507 of file dfile.c.

◆ nc_create_par()

int nc_create_par ( const char *  path,
int  cmode,
MPI_Comm  comm,
MPI_Info  info,
int *  ncidp 
)

Create a netCDF file for parallel I/O.

This function creates a new netCDF file for parallel I/O access.

Parallel I/O access is only available in library build which support parallel I/O. To support parallel I/O, netCDF must be built with netCDF-4 enabled (configure options –enable-netcdf-4 and –enable-parallel4) and with a HDF5 library that supports parallel I/O, or with support for the PnetCDF library via the –enable-pnetcdf option. This function is collective, i.e. must be called by all MPI processes defined in the MPI communicator, argument comm. In addition, values of arguments of this function must be the same among all MPI processes.

See nc_create() for a fuller discussion of file creation.

Note
When opening a netCDF-4 file HDF5 error reporting is turned off, if it is on. This doesn't stop the HDF5 error stack from recording the errors, it simply stops their display to the user through stderr.
Parameters
pathThe file name of the new netCDF dataset.
cmodeThe creation mode flag. The following flags are available: NC_CLOBBER (overwrite existing file), NC_NOCLOBBER (do not overwrite existing file), NC_NETCDF4 (create netCDF-4/HDF5 file), NC_CLASSIC_MODEL (enforce netCDF classic mode on netCDF-4/HDF5 files), NC_64BIT_OFFSET (create CDF-2 file), NC_64BIT_DATA (create CDF-5 file).
commthe MPI communicator specifying the processes participating the parallel I/O to this file.
infoMPI info object containing I/O hints or MPI_INFO_NULL.
ncidpPointer to location where returned netCDF ID is to be stored.
Returns
NC_NOERR No error.
NC_ENOPAR Library was not built with parallel I/O features.
NC_EPERM: Attempting to create a netCDF file in a directory where you do not have permission to create files.
NC_ENOTBUILT Library was not built with NETCDF4 or PnetCDF.
NC_EEXIST Specifying a file name of a file that exists and also specifying NC_NOCLOBBER.
NC_EINVAL Invalid input parameters.
NC_ENOMEM System out of memory.
NC_EHDFERR HDF5 error (netCDF-4 files only).
NC_EFILEMETA Error writing netCDF-4 file-level metadata in HDF5 file. (netCDF-4 files only).

Example

In this example from nc_test4/tst_parallel.c, a file is created for parallel I/O.

int mpi_size, mpi_rank;
MPI_Comm comm = MPI_COMM_WORLD;
MPI_Info info = MPI_INFO_NULL;
int ncid, v1id, dimids[NDIMS];
char file_name[NC_MAX_NAME + 1];
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD, &mpi_size);
MPI_Comm_rank(MPI_COMM_WORLD, &mpi_rank);
sprintf(file_name, "%s/%s", TEMP_LARGE, FILE);
if ((res = nc_create_par(file_name, NC_NETCDF4, comm, info, &ncid))) ERR;
if (nc_def_dim(ncid, "d1", DIMSIZE, dimids)) ERR;
if (nc_def_dim(ncid, "d2", DIMSIZE, &dimids[1])) ERR;
if (nc_def_dim(ncid, "d3", NUM_SLABS, &dimids[2])) ERR;
if ((res = nc_def_var(ncid, "v1", NC_INT, NDIMS, dimids, &v1id))) ERR;
... use collective I/O
err = nc_var_par_access(ncid, v1id, NC_COLLECTIVE); ERR
if ((res = nc_enddef(ncid))) ERR;
...
if ((res = nc_close(ncid))) ERR;
Author
Ed Hartnett, Dennis Heimbigner

Definition at line 100 of file dparallel.c.

◆ nc_create_par_fortran()

int nc_create_par_fortran ( const char *  path,
int  cmode,
int  comm,
int  info,
int *  ncidp 
)

Create a netCDF file for parallel access from the Fortran API.

This function calls nc_create_par() after converting the MPI comm and info from Fortran to C, if necessary.

Parameters
pathThe file name of the new netCDF dataset.
cmodeThe creation mode flag. The following flags are available: NC_CLOBBER (overwrite existing file), NC_NOCLOBBER (do not overwrite existing file), NC_NETCDF4 (create netCDF-4/HDF5 file), NC_CLASSIC_MODEL (enforce netCDF classic mode on netCDF-4/HDF5 files), NC_64BIT_OFFSET (create CDF-2 file) NC_64BIT_DATA (create CDF-5 file)
commthe MPI communicator specifying the processes participating the parallel I/O to this file.
infoMPI info object containing I/O hints or MPI_INFO_NULL.
ncidpPointer to location where returned netCDF ID is to be stored.
Returns
NC_NOERR No error.
NC_ENOPAR Library was not built with parallel I/O features.
NC_EPERM: Attempting to create a netCDF file in a directory where you do not have permission to create files.
NC_EINVAL Invalid input parameters.
NC_ENOMEM System out of memory.
NC_ENOTNC Binary format could not be determined.
NC_EHDFERR HDF5 error (netCDF-4 files only).
NC_EFILEMETA Error writing netCDF-4 file-level metadata in HDF5 file. (netCDF-4 files only).
Author
Ed Hartnett, Dennis Heimbigner

Definition at line 438 of file dparallel.c.

◆ nc_def_user_format()

int nc_def_user_format ( int  mode_flag,
NC_Dispatch *  dispatch_table,
char *  magic_number 
)

Add handling of user-defined format.

User-defined formats allow users to write a library which can read their own proprietary format as if it were netCDF. This allows existing netCDF codes to work on non-netCDF data formats.

User-defined formats work by specifying a netCDF dispatch table. The dispatch table is a struct of (mostly) C function pointers. It contains pointers to the key functions of the netCDF API. Once these functions are provided, and the dispatch table is specified, the netcdf-c library can read any format.

Note
Unlike the public netCDF API, the dispatch table may not be backward compatible between netCDF releases. Instead, it contains a dispatch version number. If this number is not correct (i.e. does not match the current dispatch table version), then NC_EINVAL will be returned.
Parameters
mode_flagNC_UDF0 or NC_UDF1
dispatch_tablePointer to dispatch table to use for this user format.
magic_numberMagic number used to identify file. Ignored if NULL.
Returns
NC_NOERR No error.
NC_EINVAL Invalid input.
Author
Ed Hartnett

Definition at line 120 of file dfile.c.

◆ nc_enddef()

int nc_enddef ( int  ncid)

Leave define mode

The function nc_enddef() takes an open netCDF dataset out of define mode. The changes made to the netCDF dataset while it was in define mode are checked and committed to disk if no problems occurred. Non-record variables may be initialized to a "fill value" as well with nc_set_fill(). The netCDF dataset is then placed in data mode, so variable data can be read or written.

It's not necessary to call nc_enddef() for netCDF-4 files. With netCDF-4 files, nc_enddef() is called when needed by the netcdf-4 library. User calls to nc_enddef() for netCDF-4 files still flush the metadata to disk.

This call may involve copying data under some circumstances. For a more extensive discussion see File Structure and Performance.

For netCDF-4/HDF5 format files there are some variable settings (the compression, endianness, fletcher32 error correction, and fill value) which must be set (if they are going to be set at all) between the nc_def_var() and the next nc_enddef(). Once the nc_enddef() is called, these settings can no longer be changed for a variable.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().

If you use a group id (in a netCDF-4/HDF5 file), the enddef will apply to the entire file. That means the enddef will not just end define mode in one group, but in the entire file.

Returns
NC_NOERR no error
NC_EBADID Invalid ncid passed.

Example

Here is an example using nc_enddef() to finish the definitions of a new netCDF dataset named foo.nc and put it into data mode:

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_create("foo.nc", NC_NOCLOBBER, &ncid);
if (status != NC_NOERR) handle_error(status);
... create dimensions, variables, attributes
status = nc_enddef(ncid);
if (status != NC_NOERR) handle_error(status);

Definition at line 1018 of file dfile.c.

◆ nc_inq()

int nc_inq ( int  ncid,
int *  ndimsp,
int *  nvarsp,
int *  nattsp,
int *  unlimdimidp 
)

Inquire about a file or group.

Parameters
ncidNetCDF or group ID, from a previous call to nc_open(), nc_create(), nc_def_grp(), or associated inquiry functions such as nc_inq_ncid().
ndimspPointer to location for returned number of dimensions defined for this netCDF dataset. Ignored if NULL.
nvarspPointer to location for returned number of variables defined for this netCDF dataset. Ignored if NULL.
nattspPointer to location for returned number of global attributes defined for this netCDF dataset. Ignored if NULL.
unlimdimidpPointer to location for returned ID of the unlimited dimension, if there is one for this netCDF dataset. If no unlimited length dimension has been defined, -1 is returned. Ignored if NULL. If there are multiple unlimited dimensions (possible only for netCDF-4 files), only a pointer to the first is returned, for backward compatibility. If you want them all, use nc_inq_unlimids().
Returns
NC_NOERR No error.
NC_EBADID Invalid ncid passed.

Example

Here is an example using nc_inq to find out about a netCDF dataset named foo.nc:

#include <netcdf.h>
...
int status, ncid, ndims, nvars, ngatts, unlimdimid;
...
status = nc_open("foo.nc", NC_NOWRITE, &ncid);
if (status != NC_NOERR) handle_error(status);
...
status = nc_inq(ncid, &ndims, &nvars, &ngatts, &unlimdimid);
if (status != NC_NOERR) handle_error(status);

Definition at line 1626 of file dfile.c.

◆ nc_inq_format()

int nc_inq_format ( int  ncid,
int *  formatp 
)

Inquire about the binary format of a netCDF file as presented by the API.

This function returns the (rarely needed) format version.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
formatpPointer to location for returned format version, one of NC_FORMAT_CLASSIC, NC_FORMAT_64BIT_OFFSET, NC_FORMAT_CDF5, NC_FORMAT_NETCDF4, NC_FORMAT_NETCDF4_CLASSIC.
Returns
NC_NOERR No error.
NC_EBADID Invalid ncid passed.
Author
Dennis Heimbigner

Definition at line 1538 of file dfile.c.

◆ nc_inq_format_extended()

int nc_inq_format_extended ( int  ncid,
int *  formatp,
int *  modep 
)

Obtain more detailed (vis-a-vis nc_inq_format) format information about an open dataset.

Note that the netcdf API will present the file as if it had the format specified by nc_inq_format. The true file format, however, may not even be a netcdf file; it might be DAP, HDF4, or PNETCDF, for example. This function returns that true file type. It also returns the effective mode for the file.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
formatpPointer to location for returned true format.
modepPointer to location for returned mode flags.

Refer to the actual list in the file netcdf.h to see the currently defined set.

Returns
NC_NOERR No error.
NC_EBADID Invalid ncid passed.

Definition at line 1573 of file dfile.c.

◆ nc_inq_path()

int nc_inq_path ( int  ncid,
size_t *  pathlen,
char *  path 
)

Get the file pathname (or the opendap URL) which was used to open/create the ncid's file.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
pathlenPointer where length of path will be returned. Ignored if NULL.
pathPointer where path name will be copied. Space must already be allocated. Ignored if NULL.
Returns
NC_NOERR No error.
NC_EBADID Invalid ncid passed.

Definition at line 889 of file dfile.c.

◆ nc_inq_type()

int nc_inq_type ( int  ncid,
nc_type  xtype,
char *  name,
size_t *  size 
)

Inquire about a type.

Given an ncid and a typeid, get the information about a type. This function will work on any type, including atomic and any user defined type, whether compound, opaque, enumeration, or variable length array.

For even more information about a user defined type nc_inq_user_type().

Parameters
ncidThe ncid for the group containing the type (ignored for atomic types).
xtypeThe typeid for this type, as returned by nc_def_compound, nc_def_opaque, nc_def_enum, nc_def_vlen, or nc_inq_var, or as found in netcdf.h in the list of atomic types (NC_CHAR, NC_INT, etc.).
nameIf non-NULL, the name of the user defined type will be copied here. It will be NC_MAX_NAME bytes or less. For atomic types, the type name from CDL will be given.
sizeIf non-NULL, the (in-memory) size of the type in bytes will be copied here. VLEN type size is the size of nc_vlen_t. String size is returned as the size of a character pointer. The size may be used to malloc space for the data, no matter what the type.
Returns
NC_NOERR No error.
NC_EBADTYPE Bad typeid.
NC_ENOTNC4 Seeking a user-defined type in a netCDF-3 file.
NC_ESTRICTNC3 Seeking a user-defined type in a netCDF-4 file for which classic model has been turned on.
NC_EBADGRPID Bad group ID in ncid.
NC_EBADID Type ID not found.
NC_EHDFERR An error was reported by the HDF5 layer.

Example

This example is from the test program tst_enums.c, and it uses all the possible inquiry functions on an enum type.

if (nc_inq_user_type(ncid, typeids[0], name_in, &base_size_in, &base_nc_type_in,
&nfields_in, &class_in)) ERR;
if (strcmp(name_in, TYPE_NAME) || base_size_in != sizeof(int) ||
base_nc_type_in != NC_INT || nfields_in != NUM_MEMBERS || class_in != NC_ENUM) ERR;
if (nc_inq_type(ncid, typeids[0], name_in, &base_size_in)) ERR;
if (strcmp(name_in, TYPE_NAME) || base_size_in != sizeof(int)) ERR;
if (nc_inq_enum(ncid, typeids[0], name_in, &base_nc_type, &base_size_in, &num_members)) ERR;
if (strcmp(name_in, TYPE_NAME) || base_nc_type != NC_INT || num_members != NUM_MEMBERS) ERR;
for (i = 0; i < NUM_MEMBERS; i++)
{
if (nc_inq_enum_member(ncid, typeid, i, name_in, &value_in)) ERR;
if (strcmp(name_in, member_name[i]) || value_in != member_value[i]) ERR;
if (nc_inq_enum_ident(ncid, typeid, member_value[i], name_in)) ERR;
if (strcmp(name_in, member_name[i])) ERR;
}
if (nc_close(ncid)) ERR;

Definition at line 1719 of file dfile.c.

◆ nc_inq_user_format()

int nc_inq_user_format ( int  mode_flag,
NC_Dispatch **  dispatch_table,
char *  magic_number 
)

Inquire about user-defined format.

Parameters
mode_flagNC_UDF0 or NC_UDF1
dispatch_tablePointer that gets pointer to dispatch table to use for this user format, or NULL if this user-defined format is not defined. Ignored if NULL.
magic_numberPointer that gets magic number used to identify file, if one has been set. Magic number will be of max size NC_MAX_MAGIC_NUMBER_LEN. Ignored if NULL.
Returns
NC_NOERR No error.
NC_EINVAL Invalid input.
Author
Ed Hartnett

Definition at line 170 of file dfile.c.

◆ nc_open()

int nc_open ( const char *  path,
int  omode,
int *  ncidp 
)

Open an existing netCDF file.

This function opens an existing netCDF dataset for access. It determines the underlying file format automatically. Use the same call to open a netCDF classic or netCDF-4 file.

Parameters
pathFile name for netCDF dataset to be opened. When the dataset is located on some remote server, then the path may be an OPeNDAP URL rather than a file path.
omodeThe open mode flag may include NC_WRITE (for read/write access) and NC_SHARE (see below) and NC_DISKLESS (see below).
ncidpPointer to location where returned netCDF ID is to be stored.

Open Mode

A zero value (or NC_NOWRITE) specifies the default behavior: open the dataset with read-only access, buffering and caching accesses for efficiency.

Otherwise, the open mode is NC_WRITE, NC_SHARE, or NC_WRITE|NC_SHARE. Setting the NC_WRITE flag opens the dataset with read-write access. ("Writing" means any kind of change to the dataset, including appending or changing data, adding or renaming dimensions, variables, and attributes, or deleting attributes.)

The NC_SHARE flag is only used for netCDF classic files. It is appropriate when one process may be writing the dataset and one or more other processes reading the dataset concurrently; it means that dataset accesses are not buffered and caching is limited. Since the buffering scheme is optimized for sequential access, programs that do not access data sequentially may see some performance improvement by setting the NC_SHARE flag.

This procedure may also be invoked with the NC_DISKLESS flag set in the omode argument if the file to be opened is a classic format file. For nc_open(), this flag applies only to files in classic format. If the file is of type NC_NETCDF4, then the NC_DISKLESS flag will be ignored.

If NC_DISKLESS is specified, then the whole file is read completely into memory. In effect this creates an in-memory cache of the file. If the omode flag also specifies NC_PERSIST, then the in-memory cache will be re-written to the disk file when nc_close() is called. For some kinds of manipulations, having the in-memory cache can speed up file processing. But in simple cases, non-cached processing may actually be faster than using cached processing. You will need to experiment to determine if the in-memory caching is worthwhile for your application.

Normally, NC_DISKLESS allocates space in the heap for storing the in-memory file. If, however, the ./configure flags –enable-mmap is used, and the additional omode flag NC_MMAP is specified, then the file will be opened using the operating system MMAP facility. This flag only applies to files in classic format. Extended format (netcdf-4) files will ignore the NC_MMAP flag.

In most cases, using MMAP provides no advantage for just NC_DISKLESS. The one case where using MMAP is an advantage is when a file is to be opened and only a small portion of its data is to be read and/or written. In this scenario, MMAP will cause only the accessed data to be retrieved from disk. Without MMAP, NC_DISKLESS will read the whole file into memory on nc_open. Thus, MMAP will provide some performance improvement in this case.

It is not necessary to pass any information about the format of the file being opened. The file type will be detected automatically by the netCDF library.

If a the path is a DAP URL, then the open mode is read-only. Setting NC_WRITE will be ignored.

As of version 4.3.1.2, multiple calls to nc_open with the same path will return the same ncid value.

Note
When opening a netCDF-4 file HDF5 error reporting is turned off, if it is on. This doesn't stop the HDF5 error stack from recording the errors, it simply stops their display to the user through stderr.

nc_open()returns the value NC_NOERR if no errors occurred. Otherwise, the returned status indicates an error. Possible causes of errors include:

Note that nc_open(path,omode,ncidp) is equivalent to the invocation of nc__open(path,omode,NC_SIZEHINT_DEFAULT,NULL,ncidp).

Returns
NC_NOERR No error.
NC_EPERM Attempting to create a netCDF file in a directory where you do not have permission to open files.
NC_ENFILE Too many files open
NC_ENOMEM Out of memory.
NC_EHDFERR HDF5 error. (NetCDF-4 files only.)
NC_EDIMMETA Error in netCDF-4 dimension metadata. (NetCDF-4 files only.)

Examples

Here is an example using nc_open()to open an existing netCDF dataset named foo.nc for read-only, non-shared access:

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_open("foo.nc", 0, &ncid);
if (status != NC_NOERR) handle_error(status);
Author
Glenn Davis, Ed Hartnett, Dennis Heimbigner

Definition at line 655 of file dfile.c.

◆ nc_open_mem()

int nc_open_mem ( const char *  path,
int  omode,
size_t  size,
void *  memory,
int *  ncidp 
)

Open a netCDF file with the contents taken from a block of memory.

Parameters
pathMust be non-null, but otherwise only used to set the dataset name.
omodethe open mode flags; Note that this procedure uses a limited set of flags because it forcibly sets NC_INMEMORY.
sizeThe length of the block of memory being passed.
memoryPointer to the block of memory containing the contents of a netcdf file.
ncidpPointer to location where returned netCDF ID is to be stored.
Returns
NC_NOERR No error.
NC_ENOMEM Out of memory.
NC_EDISKLESS diskless io is not enabled for fails.
NC_EINVAL, etc. other errors also returned by nc_open.

Examples

Here is an example using nc_open_mem() to open an existing netCDF dataset named foo.nc for read-only, non-shared access. It differs from the nc_open() example in that it assumes the contents of foo.nc have been read into memory.

#include <netcdf.h>
#include <netcdf_mem.h>
...
int status = NC_NOERR;
int ncid;
size_t size;
void* memory;
...
size = <compute file size of foo.nc in bytes>;
memory = malloc(size);
...
status = nc_open_mem("foo.nc", 0, size, memory, &ncid);
if (status != NC_NOERR) handle_error(status);

Definition at line 767 of file dfile.c.

◆ nc_open_memio()

int nc_open_memio ( const char *  path,
int  omode,
NC_memio params,
int *  ncidp 
)

Open a netCDF file with the contents taken from a block of memory. Similar to nc_open_mem, but with parameters. Warning: if you do specify that the provided memory is locked, then never pass in non-heap allocated memory. Additionally, if not locked, then do not assume that the memory returned by nc_close_mem is the same as passed to nc_open_memio. You must check before attempting to free the original memory.

Parameters
pathMust be non-null, but otherwise only used to set the dataset name.
omodethe open mode flags; Note that this procedure uses a limited set of flags because it forcibly sets NC_INMEMORY.
paramscontrolling parameters
ncidpPointer to location where returned netCDF ID is to be stored.
Returns
NC_NOERR No error.
NC_ENOMEM Out of memory.
NC_EDISKLESS diskless io is not enabled for fails.
NC_EINVAL, etc. other errors also returned by nc_open.

Examples

Here is an example using nc_open_memio() to open an existing netCDF dataset named foo.nc for read-only, non-shared access. It differs from the nc_open_mem() example in that it uses a parameter block.

#include <netcdf.h>
#include <netcdf_mem.h>
...
int status = NC_NOERR;
int ncid;
NC_memio params;
...
params.size = <compute file size of foo.nc in bytes>;
params.memory = malloc(size);
params.flags = <see netcdf_mem.h>
...
status = nc_open_memio("foo.nc", 0, &params, &ncid);
if (status != NC_NOERR) handle_error(status);

Definition at line 832 of file dfile.c.

◆ nc_open_par()

int nc_open_par ( const char *  path,
int  omode,
MPI_Comm  comm,
MPI_Info  info,
int *  ncidp 
)

Open an existing netCDF file for parallel I/O.

This function opens an existing netCDF dataset for parallel I/O access. It determines the underlying file format automatically. Use the same call to open a netCDF classic, 64-bit offset, CDF-5, or netCDF-4 file.

Parallel I/O access is only available in library build which support parallel I/O. To support parallel I/O, netCDF must be built with netCDF-4 enabled (configure options –enable-netcdf-4 and –enable-parallel4) and with a HDF5 library that supports parallel I/O, or with support for the PnetCDF library via the –enable-pnetcdf option. This function is collective, i.e. must be called by all MPI processes defined in the MPI communicator, argument comm. In addition, values of arguments of this function must be the same among all MPI processes.

It is not necessary to pass any information about the format of the file being opened. The file type will be detected automatically by the netCDF library.

As of version 4.3.1.2, multiple calls to nc_open_par() with the same path will return the same ncid value.

Note
When opening a netCDF-4 file HDF5 error reporting is turned off, if it is on. This doesn't stop the HDF5 error stack from recording the errors, it simply stops their display to the user through stderr.
Parameters
pathFile name for netCDF dataset to be opened.
omodeThe open mode flag may be NC_WRITE (for read/write access) or NC_NOWRITE (for read-only access).
commthe MPI communicator specifying the processes participating the parallel I/O to this file.
infoMPI info object containing I/O hints or MPI_INFO_NULL.
ncidpPointer to location where returned netCDF ID is to be stored.

nc_open_par() returns the value NC_NOERR if no errors occurred. Otherwise, the returned status indicates an error. Possible causes of errors include:

Returns
NC_NOERR No error.
NC_ENOPAR Library was not built with parallel I/O features.
NC_EPERM: Attempting to open a netCDF file where you do not have permission to write the file.
NC_ENOTBUILT Library was not built with NETCDF4 or PnetCDF.
NC_EINVAL Invalid parameters.
NC_ENOTNC Not a netCDF file.
NC_ENOMEM Out of memory.
NC_EHDFERR HDF5 error. (NetCDF-4 files only.)
NC_EDIMMETA Error in netCDF-4 dimension metadata (NetCDF-4 files only.)

Examples

Here is an example using nc_open_par() from examples/C/parallel_vara.c.

#include <mpi.h>
#include <netcdf.h>
#include <netcdf_par.h>
...
int ncid, omode;
char filename[128];
...
omode = NC_NOWRITE;
err = nc_open_par(filename, omode, MPI_COMM_WORLD, MPI_INFO_NULL, &ncid); FATAL_ERR
Author
Ed Hartnett, Dennis Heimbigner

Definition at line 208 of file dparallel.c.

◆ nc_open_par_fortran()

int nc_open_par_fortran ( const char *  path,
int  omode,
int  comm,
int  info,
int *  ncidp 
)

This is the same as nc_open_par(), but accepts the MPI comm/info as integers.

Parameters
pathFile name for netCDF dataset to be opened.
omodeThe open mode flag may be NC_WRITE (for read/write access) or NC_NOWRITE (for read-only access).
commthe MPI communicator specifying the processes participating the parallel I/O to this file.
infoMPI info object containing I/O hints or MPI_INFO_NULL.
ncidpPointer to location where returned netCDF ID is to be stored.

nc_open_par() returns the value NC_NOERR if no errors occurred. Otherwise, the returned status indicates an error. Possible causes of errors include:

Returns
NC_NOERR No error.
NC_EINVAL Invalid parameters.
NC_ENOTNC Not a netCDF file.
NC_ENOMEM Out of memory.
NC_EHDFERR HDF5 error. (NetCDF-4 files only.)
NC_EDIMMETA Error in netCDF-4 dimension metadata. (NetCDF-4 files only.)
Author
Ed Hartnett

Definition at line 261 of file dparallel.c.

◆ nc_redef()

int nc_redef ( int  ncid)

Put open netcdf dataset into define mode

The function nc_redef puts an open netCDF dataset into define mode, so dimensions, variables, and attributes can be added or renamed and attributes can be deleted.

For netCDF-4 files (i.e. files created with NC_NETCDF4 in the cmode in their call to nc_create()), it is not necessary to call nc_redef() unless the file was also created with NC_STRICT_NC3. For straight-up netCDF-4 files, nc_redef() is called automatically, as needed.

For all netCDF-4 files, the root ncid must be used. This is the ncid returned by nc_open() and nc_create(), and points to the root of the hierarchy tree for netCDF-4 files.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
Returns
NC_NOERR No error.
NC_EBADID Bad ncid.
NC_EBADGRPID The ncid must refer to the root group of the file, that is, the group returned by nc_open() or nc_create().
NC_EINDEFINE Already in define mode.
NC_EPERM File is read-only.

Example

Here is an example using nc_redef to open an existing netCDF dataset named foo.nc and put it into define mode:

#include <netcdf.h>
...
int status = NC_NOERR;
int ncid;
...
status = nc_open("foo.nc", NC_WRITE, &ncid);
if (status != NC_NOERR) handle_error(status);
...
status = nc_redef(ncid);
if (status != NC_NOERR) handle_error(status);

Definition at line 954 of file dfile.c.

◆ nc_set_fill()

int nc_set_fill ( int  ncid,
int  fillmode,
int *  old_modep 
)

Change the fill-value mode to improve write performance.

This function is intended for advanced usage, to optimize writes under some circumstances described below. The function nc_set_fill() sets the fill mode for a netCDF dataset open for writing and returns the current fill mode in a return parameter. The fill mode can be specified as either NC_FILL or NC_NOFILL. The default behavior corresponding to NC_FILL is that data is pre-filled with fill values, that is fill values are written when you create non-record variables or when you write a value beyond data that has not yet been written. This makes it possible to detect attempts to read data before it was written. For more information on the use of fill values see Fill Values. For information about how to define your own fill values see Attribute Conventions.

The behavior corresponding to NC_NOFILL overrides the default behavior of prefilling data with fill values. This can be used to enhance performance, because it avoids the duplicate writes that occur when the netCDF library writes fill values that are later overwritten with data.

A value indicating which mode the netCDF dataset was already in is returned. You can use this value to temporarily change the fill mode of an open netCDF dataset and then restore it to the previous mode.

After you turn on NC_NOFILL mode for an open netCDF dataset, you must be certain to write valid data in all the positions that will later be read. Note that nofill mode is only a transient property of a netCDF dataset open for writing: if you close and reopen the dataset, it will revert to the default behavior. You can also revert to the default behavior by calling nc_set_fill() again to explicitly set the fill mode to NC_FILL.

There are three situations where it is advantageous to set nofill mode:

  • Creating and initializing a netCDF dataset. In this case, you should set nofill mode before calling nc_enddef() and then write completely all non-record variables and the initial records of all the record variables you want to initialize.
  • Extending an existing record-oriented netCDF dataset. Set nofill mode after opening the dataset for writing, then append the additional records to the dataset completely, leaving no intervening unwritten records.
  • Adding new variables that you are going to initialize to an existing netCDF dataset. Set nofill mode before calling nc_enddef() then write all the new variables completely.

If the netCDF dataset has an unlimited dimension and the last record was written while in nofill mode, then the dataset may be shorter than if nofill mode was not set, but this will be completely transparent if you access the data only through the netCDF interfaces.

The use of this feature may not be available (or even needed) in future releases. Programmers are cautioned against heavy reliance upon this feature.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
fillmodeDesired fill mode for the dataset, either NC_NOFILL or NC_FILL.
old_modepPointer to location for returned current fill mode of the dataset before this call, either NC_NOFILL or NC_FILL.
Returns
NC_NOERR No error.
NC_EBADID The specified netCDF ID does not refer to an open netCDF dataset.
NC_EPERM The specified netCDF ID refers to a dataset open for read-only access.
NC_EINVAL The fill mode argument is neither NC_NOFILL nor NC_FILL.

Example

Here is an example using nc_set_fill() to set nofill mode for subsequent writes of a netCDF dataset named foo.nc:

#include <netcdf.h>
...
int ncid, status, old_fill_mode;
...
status = nc_open("foo.nc", NC_WRITE, &ncid);
if (status != NC_NOERR) handle_error(status);
... write data with default prefilling behavior
status = nc_set_fill(ncid, ::NC_NOFILL, &old_fill_mode);
if (status != NC_NOERR) handle_error(status);
... write data with no prefilling

Definition at line 1465 of file dfile.c.

◆ nc_sync()

int nc_sync ( int  ncid)

Synchronize an open netcdf dataset to disk

The function nc_sync() offers a way to synchronize the disk copy of a netCDF dataset with in-memory buffers. There are two reasons you might want to synchronize after writes:

  • To minimize data loss in case of abnormal termination, or
  • To make data available to other processes for reading immediately after it is written. But note that a process that already had the dataset open for reading would not see the number of records increase when the writing process calls nc_sync(); to accomplish this, the reading process must call nc_sync.

This function is backward-compatible with previous versions of the netCDF library. The intent was to allow sharing of a netCDF dataset among multiple readers and one writer, by having the writer call nc_sync() after writing and the readers call nc_sync() before each read. For a writer, this flushes buffers to disk. For a reader, it makes sure that the next read will be from disk rather than from previously cached buffers, so that the reader will see changes made by the writing process (e.g., the number of records written) without having to close and reopen the dataset. If you are only accessing a small amount of data, it can be expensive in computer resources to always synchronize to disk after every write, since you are giving up the benefits of buffering.

An easier way to accomplish sharing (and what is now recommended) is to have the writer and readers open the dataset with the NC_SHARE flag, and then it will not be necessary to call nc_sync() at all. However, the nc_sync() function still provides finer granularity than the NC_SHARE flag, if only a few netCDF accesses need to be synchronized among processes.

It is important to note that changes to the ancillary data, such as attribute values, are not propagated automatically by use of the NC_SHARE flag. Use of the nc_sync() function is still required for this purpose.

Sharing datasets when the writer enters define mode to change the data schema requires extra care. In previous releases, after the writer left define mode, the readers were left looking at an old copy of the dataset, since the changes were made to a new copy. The only way readers could see the changes was by closing and reopening the dataset. Now the changes are made in place, but readers have no knowledge that their internal tables are now inconsistent with the new dataset schema. If netCDF datasets are shared across redefinition, some mechanism external to the netCDF library must be provided that prevents access by readers during redefinition and causes the readers to call nc_sync before any subsequent access.

When calling nc_sync(), the netCDF dataset must be in data mode. A netCDF dataset in define mode is synchronized to disk only when nc_enddef() is called. A process that is reading a netCDF dataset that another process is writing may call nc_sync to get updated with the changes made to the data by the writing process (e.g., the number of records written), without having to close and reopen the dataset.

Data is automatically synchronized to disk when a netCDF dataset is closed, or whenever you leave define mode.

Parameters
ncidNetCDF ID, from a previous call to nc_open() or nc_create().
Returns
NC_NOERR No error.
NC_EBADID Invalid ncid passed.

Definition at line 1186 of file dfile.c.

◆ nc_var_par_access()

int nc_var_par_access ( int  ncid,
int  varid,
int  par_access 
)

This function will change the parallel access of a variable from independent to collective and vice versa.

This function is collective, i.e. must be called by all MPI processes defined in the MPI communicator used in nc_create_par() or nc_open_par(). In addition, values of arguments of this function must be the same among all MPI processes.

To obtain a good I/O performance, users are recommended to use collective mode. In addition, switching between collective and independent I/O mode can be expensive.

In netcdf-c-4.7.4 or later, using hdf5-1.10.2 or later, the zlib, szip, fletcher32, and other filters may be used when writing data with parallel I/O. The use of these filters require collective access. Turning on the zlib (deflate) or fletcher32 filter for a variable will automatically set its access to collective if the file has been opened for parallel I/O. Attempts to set access to independent will return NC_EINVAL.

Note
When the library is build with –enable-pnetcdf, and when file is opened/created to use PnetCDF library to perform parallel I/O underneath, argument varid is ignored and the mode changed by this function applies to all variables. This is because PnetCDF does not support access mode change for individual variables. In this case, users may use NC_GLOBAL in varid argument for better program readability.
Parameters
ncidNetCDF or group ID, from a previous call to nc_open_par(), nc_create_par(), nc_def_grp(), or associated inquiry functions such as nc_inq_ncid().
varidVariable ID
par_accessNC_COLLECTIVE or NC_INDEPENDENT.
Returns
NC_NOERR No error.
NC_EBADID Invalid ncid passed.
NC_ENOTVAR Invalid varid passed.
NC_ENOPAR File was not opened with nc_open_par/nc_create_var.
NC_EINVAL Invalid par_access specified, or attempt to set filtered variable to independent access.

Example

Here is an example from examples/C/parallel_vara.c which changes the parallel access of a variable and then writes to it.

#define NY 10
#define NX 4
#include <mpi.h>
#include <netcdf.h>
#include <netcdf_par.h>
...
char filename[128];
int err, ncid, cmode, varid, dimid, dimid[2], buf[NY][NX];
size_t global_ny, global_nx, start[2], count[2];
...
global_ny = NY;
global_nx = NX * nprocs;
...
cmode = NC_CLOBBER;
err = nc_create_par(filename, cmode, MPI_COMM_WORLD, MPI_INFO_NULL, &ncid); FATAL_ERR
...
err = nc_def_dim(ncid, "Y", global_ny, &dimid[0]); ERR
err = nc_def_dim(ncid, "X", global_nx, &dimid[1]); ERR
err = nc_def_var(ncid, "var", NC_INT, 2, dimid, &varid); ERR
...
... set collective I/O globally (for all variables)
err = nc_enddef(ncid); ERR
...
start[0] = 0;
start[1] = NX * rank;
count[0] = NY;
count[1] = NX;
err = nc_put_vara_int(ncid, varid, start, count, &buf[0][0]); ERR
Author
Ed Hartnett, Dennis Heimbigner

Definition at line 381 of file dparallel.c.

NC_NOERR
#define NC_NOERR
No Error.
Definition: netcdf.h:333
nc_open_par
EXTERNL int nc_open_par(const char *path, int mode, MPI_Comm comm, MPI_Info info, int *ncidp)
Open an existing netCDF file for parallel I/O.
Definition: dparallel.c:208
NC_PERSIST
#define NC_PERSIST
Save diskless contents to disk.
Definition: netcdf.h:161
NC_NOWRITE
#define NC_NOWRITE
Set read-only access for nc_open().
Definition: netcdf.h:125
NC_DISKLESS
#define NC_DISKLESS
Use diskless file.
Definition: netcdf.h:130
nc_inq_user_type
EXTERNL int nc_inq_user_type(int ncid, nc_type xtype, char *name, size_t *size, nc_type *base_nc_typep, size_t *nfieldsp, int *classp)
Definition: dtype.c:146
nc_close_memio
EXTERNL int nc_close_memio(int ncid, NC_memio *info)
Definition: dfile.c:1350
nc_inq_enum_ident
EXTERNL int nc_inq_enum_ident(int ncid, nc_type xtype, long long value, char *identifier)
Definition: denum.c:168
NC_NOFILL
#define NC_NOFILL
Argument to nc_set_fill() to turn off filling of data.
Definition: netcdf.h:115
netcdf_mem.h
nc_redef
EXTERNL int nc_redef(int ncid)
Definition: dfile.c:954
nc_def_var
EXTERNL int nc_def_var(int ncid, const char *name, nc_type xtype, int ndims, const int *dimidsp, int *varidp)
Define a new variable.
Definition: dvar.c:212
nc__enddef
EXTERNL int nc__enddef(int ncid, size_t h_minfree, size_t v_align, size_t v_minfree, size_t r_align)
Definition: dfile.c:1109
nc_inq_enum
EXTERNL int nc_inq_enum(int ncid, nc_type xtype, char *name, nc_type *base_nc_typep, size_t *base_sizep, size_t *num_membersp)
Definition: denum.c:107
nc_def_dim
EXTERNL int nc_def_dim(int ncid, const char *name, size_t len, int *idp)
Define a new dimension.
Definition: ddim.c:121
NC_COLLECTIVE
#define NC_COLLECTIVE
Use with nc_var_par_access() to set parallel access to collective.
Definition: netcdf_par.h:28
NC_INT
#define NC_INT
signed 4 byte integer
Definition: netcdf.h:38
NC_WRITE
#define NC_WRITE
Set read-write access for nc_open().
Definition: netcdf.h:126
NC_MAX_NAME
#define NC_MAX_NAME
Maximum for classic library.
Definition: netcdf.h:276
nc_open_memio
EXTERNL int nc_open_memio(const char *path, int mode, NC_memio *info, int *ncidp)
Definition: dfile.c:832
netcdf_par.h
nc_inq
EXTERNL int nc_inq(int ncid, int *ndimsp, int *nvarsp, int *nattsp, int *unlimdimidp)
Definition: dfile.c:1626
NC_GLOBAL
#define NC_GLOBAL
Attribute id to put/get a global attribute.
Definition: netcdf.h:249
NC_memio
Definition: netcdf_mem.h:22
NC_NOCLOBBER
#define NC_NOCLOBBER
Don't destroy existing file.
Definition: netcdf.h:129
nc_close
EXTERNL int nc_close(int ncid)
Definition: dfile.c:1291
NC_64BIT_OFFSET
#define NC_64BIT_OFFSET
Use large (64-bit) file offsets.
Definition: netcdf.h:140
netcdf.h
nc_abort
EXTERNL int nc_abort(int ncid)
Definition: dfile.c:1238
nc_inq_type
EXTERNL int nc_inq_type(int ncid, nc_type xtype, char *name, size_t *size)
Definition: dfile.c:1719
nc_var_par_access
EXTERNL int nc_var_par_access(int ncid, int varid, int par_access)
Definition: dparallel.c:381
nc_open_mem
EXTERNL int nc_open_mem(const char *path, int mode, size_t size, void *memory, int *ncidp)
Definition: dfile.c:767
nc_set_fill
EXTERNL int nc_set_fill(int ncid, int fillmode, int *old_modep)
Definition: dfile.c:1465
nc_create_par
EXTERNL int nc_create_par(const char *path, int cmode, MPI_Comm comm, MPI_Info info, int *ncidp)
Create a netCDF file for parallel I/O.
Definition: dparallel.c:100
nc_open
EXTERNL int nc_open(const char *path, int mode, int *ncidp)
Open an existing netCDF file.
Definition: dfile.c:655
NC_ENUM
#define NC_ENUM
enum types
Definition: netcdf.h:55
nc__create
EXTERNL int nc__create(const char *path, int cmode, size_t initialsz, size_t *chunksizehintp, int *ncidp)
Create a netCDF file with some extra parameters controlling classic file caching.
Definition: dfile.c:461
nc_put_vara_int
int nc_put_vara_int(int ncid, int varid, const size_t *startp, const size_t *countp, const int *op)
Definition: dvarput.c:676
NC_CLASSIC_MODEL
#define NC_CLASSIC_MODEL
Enforce classic model on netCDF-4.
Definition: netcdf.h:139
nc_create_mem
EXTERNL int nc_create_mem(const char *path, int mode, size_t initialsize, int *ncidp)
Definition: dfile.c:507
NC_NETCDF4
#define NC_NETCDF4
Use netCDF-4/HDF5 format.
Definition: netcdf.h:152
nc_enddef
EXTERNL int nc_enddef(int ncid)
Definition: dfile.c:1018
nc_create
EXTERNL int nc_create(const char *path, int cmode, int *ncidp)
Definition: dfile.c:389
NC_CLOBBER
#define NC_CLOBBER
Destroy existing file.
Definition: netcdf.h:128
nc_inq_enum_member
EXTERNL int nc_inq_enum_member(int ncid, nc_type xtype, int idx, char *name, void *value)
Definition: denum.c:140