 |
NetCDF
4.8.1
|
|
NetCDF
4.8.1
|
NetCDF opens datasets as files or remote access URLs. More...
Functions | |
| int | nc__create (const char *path, int cmode, size_t initialsz, size_t *chunksizehintp, int *ncidp) |
| Create a netCDF file with some extra parameters controlling classic file caching. More... | |
| int | nc__enddef (int ncid, size_t h_minfree, size_t v_align, size_t v_minfree, size_t r_align) |
| int | nc__open (const char *path, int omode, size_t *chunksizehintp, int *ncidp) |
| int | nc_abort (int ncid) |
| int | nc_close (int ncid) |
| int | nc_close_memio (int ncid, NC_memio *memio) |
| int | nc_create (const char *path, int cmode, int *ncidp) |
| int | nc_create_mem (const char *path, int mode, size_t initialsize, int *ncidp) |
| int | nc_create_par (const char *path, int cmode, MPI_Comm comm, MPI_Info info, int *ncidp) |
| Create a netCDF file for parallel I/O. More... | |
| int | nc_create_par_fortran (const char *path, int cmode, int comm, int info, int *ncidp) |
| Create a netCDF file for parallel access from the Fortran API. More... | |
| int | nc_def_user_format (int mode_flag, NC_Dispatch *dispatch_table, char *magic_number) |
| Add handling of user-defined format. More... | |
| int | nc_enddef (int ncid) |
| int | nc_inq (int ncid, int *ndimsp, int *nvarsp, int *nattsp, int *unlimdimidp) |
| int | nc_inq_format (int ncid, int *formatp) |
| int | nc_inq_format_extended (int ncid, int *formatp, int *modep) |
| int | nc_inq_path (int ncid, size_t *pathlen, char *path) |
| int | nc_inq_type (int ncid, nc_type xtype, char *name, size_t *size) |
| int | nc_inq_user_format (int mode_flag, NC_Dispatch **dispatch_table, char *magic_number) |
| Inquire about user-defined format. More... | |
| int | nc_open (const char *path, int omode, int *ncidp) |
| Open an existing netCDF file. More... | |
| int | nc_open_mem (const char *path, int omode, size_t size, void *memory, int *ncidp) |
| int | nc_open_memio (const char *path, int omode, NC_memio *params, int *ncidp) |
| int | nc_open_par (const char *path, int omode, MPI_Comm comm, MPI_Info info, int *ncidp) |
| Open an existing netCDF file for parallel I/O. More... | |
| int | nc_open_par_fortran (const char *path, int omode, int comm, int info, int *ncidp) |
| This is the same as nc_open_par(), but accepts the MPI comm/info as integers. More... | |
| int | nc_redef (int ncid) |
| int | nc_set_fill (int ncid, int fillmode, int *old_modep) |
| int | nc_sync (int ncid) |
| int | nc_var_par_access (int ncid, int varid, int par_access) |
NetCDF opens datasets as files or remote access URLs.
A netCDF dataset that has not yet been opened can only be referred to by its dataset name. Once a netCDF dataset is opened, it is referred to by a netCDF ID, which is a small non-negative integer returned when you create or open the dataset. A netCDF ID is much like a file descriptor in C or a logical unit number in FORTRAN. In any single program, the netCDF IDs of distinct open netCDF datasets are distinct. A single netCDF dataset may be opened multiple times and will then have multiple distinct netCDF IDs; however at most one of the open instances of a single netCDF dataset should permit writing. When an open netCDF dataset is closed, the ID is no longer associated with a netCDF dataset.
Functions that deal with the netCDF library include:
The operations supported on a netCDF dataset as a single object are:
| int nc__create | ( | const char * | path, |
| int | cmode, | ||
| size_t | initialsz, | ||
| size_t * | chunksizehintp, | ||
| int * | ncidp | ||
| ) |
Create a netCDF file with some extra parameters controlling classic file caching.
Like nc_create(), this function creates a netCDF file.
| path | The file name of the new netCDF dataset. |
| cmode | The creation mode flag, the same as in nc_create(). |
| initialsz | On some systems, and with custom I/O layers, it may be advantageous to set the size of the output file at creation time. This parameter sets the initial size of the file at creation time. This only applies to classic CDF-1, 2, and 5 files. The special value NC_SIZEHINT_DEFAULT (which is the value 0), lets the netcdf library choose a suitable initial size. |
| chunksizehintp | A pointer to the chunk size hint, which controls a space versus time tradeoff, memory allocated in the netcdf library versus number of system calls. Because of internal requirements, the value may not be set to exactly the value requested. The actual value chosen is returned by reference. Using a NULL pointer or having the pointer point to the value NC_SIZEHINT_DEFAULT causes the library to choose a default. How the system chooses the default depends on the system. On many systems, the "preferred I/O block size" is available from the stat() system call, struct stat member st_blksize. If this is available it is used. Lacking that, twice the system pagesize is used. Lacking a call to discover the system pagesize, we just set default bufrsize to 8192. The bufrsize is a property of a given open netcdf descriptor ncid, it is not a persistent property of the netcdf dataset. This only applies to classic files. |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
In this example we create a netCDF dataset named foo_large.nc; we want the dataset to be created in the current directory only if a dataset with that name does not already exist. We also specify that bufrsize and initial size for the file.
| int nc__enddef | ( | int | ncid, |
| size_t | h_minfree, | ||
| size_t | v_align, | ||
| size_t | v_minfree, | ||
| size_t | r_align | ||
| ) |
Leave define mode with performance tuning
The function nc__enddef takes an open netCDF dataset out of define mode. The changes made to the netCDF dataset while it was in define mode are checked and committed to disk if no problems occurred. Non-record variables may be initialized to a "fill value" as well with nc_set_fill(). The netCDF dataset is then placed in data mode, so variable data can be read or written.
This call may involve copying data under some circumstances. For a more extensive discussion see File Structure and Performance.
The classic netcdf file format has three sections, the "header" section, the data section for fixed size variables, and the data section for variables which have an unlimited dimension (record variables).
The header begins at the beginning of the file. The index (offset) of the beginning of the other two sections is contained in the header. Typically, there is no space between the sections. This causes copying overhead to accrue if one wishes to change the size of the sections, as may happen when changing names of things, text attribute values, adding attributes or adding variables. Also, for buffered i/o, there may be advantages to aligning sections in certain ways.
The minfree parameters allow one to control costs of future calls to nc_redef, nc_enddef() by requesting that minfree bytes be available at the end of the section.
The align parameters allow one to set the alignment of the beginning of the corresponding sections. The beginning of the section is rounded up to an index which is a multiple of the align parameter. The flag value ALIGN_CHUNK tells the library to use the bufrsize (see above) as the align parameter. It has nothing to do with the chunking (multidimensional tiling) features of netCDF-4.
The file format requires mod 4 alignment, so the align parameters are silently rounded up to multiples of 4. The usual call,
is equivalent to
The file format does not contain a "record size" value, this is calculated from the sizes of the record variables. This unfortunate fact prevents us from providing minfree and alignment control of the "records" in a netcdf file. If you add a variable which has an unlimited dimension, the third section will always be copied with the new variable added.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
| h_minfree | Sets the pad at the end of the "header" section. |
| v_align | Controls the alignment of the beginning of the data section for fixed size variables. |
| v_minfree | Sets the pad at the end of the data section for fixed size variables. |
| r_align | Controls the alignment of the beginning of the data section for variables which have an unlimited dimension (record variables). |
| int nc__open | ( | const char * | path, |
| int | omode, | ||
| size_t * | chunksizehintp, | ||
| int * | ncidp | ||
| ) |
Open a netCDF file with extra performance parameters for the classic library.
| path | File name for netCDF dataset to be opened. When DAP support is enabled, then the path may be an OPeNDAP URL rather than a file path. |
| omode | The open mode flag may include NC_WRITE (for read/write access) and NC_SHARE as in nc_open(). |
| chunksizehintp | A size hint for the classic library. Only applies to classic files. See below for more information. |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
The argument referenced by bufrsizehintp controls a space versus time tradeoff, memory allocated in the netcdf library versus number of system calls.
Because of internal requirements, the value may not be set to exactly the value requested. The actual value chosen is returned by reference.
Using a NULL pointer or having the pointer point to the value NC_SIZEHINT_DEFAULT causes the library to choose a default. How the system chooses the default depends on the system. On many systems, the "preferred I/O block size" is available from the stat() system call, struct stat member st_blksize. If this is available it is used. Lacking that, twice the system pagesize is used.
Lacking a call to discover the system pagesize, we just set default bufrsize to 8192.
The bufrsize is a property of a given open netcdf descriptor ncid, it is not a persistent property of the netcdf dataset.
| int nc_abort | ( | int | ncid | ) |
No longer necessary for user to invoke manually.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
Here is an example using nc_abort to back out of redefinitions of a dataset named foo.nc:
| int nc_close | ( | int | ncid | ) |
Close an open netCDF dataset
If the dataset in define mode, nc_enddef() will be called before closing. (In this case, if nc_enddef() returns an error, nc_abort() will automatically be called to restore the dataset to the consistent state before define mode was last entered.) After an open netCDF dataset is closed, its netCDF ID may be reassigned to the next netCDF dataset that is opened or created.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
Here is an example using nc_close to finish the definitions of a new netCDF dataset named foo.nc and release its netCDF ID:
| int nc_close_memio | ( | int | ncid, |
| NC_memio * | memio | ||
| ) |
Do a normal close (see nc_close()) on an in-memory dataset, then return a copy of the final memory contents of the dataset.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
| memio | a pointer to an NC_memio object into which the final valid memory size and memory will be returned. |
Here is an example using nc_close_mem to finish the definitions of a new netCDF dataset named foo.nc, return the final memory, and release its netCDF ID:
| int nc_create | ( | const char * | path, |
| int | cmode, | ||
| int * | ncidp | ||
| ) |
Create a new netCDF file.
This function creates a new netCDF dataset, returning a netCDF ID that can subsequently be used to refer to the netCDF dataset in other netCDF function calls. The new netCDF dataset opened for write access and placed in define mode, ready for you to add dimensions, variables, and attributes.
| path | The file name of the new netCDF dataset. |
| cmode | The creation mode flag. The following flags are available: NC_CLOBBER (overwrite existing file), NC_NOCLOBBER (do not overwrite existing file), NC_SHARE (limit write caching - netcdf classic files only), NC_64BIT_OFFSET (create 64-bit offset file), NC_64BIT_DATA (alias NC_CDF5) (create CDF-5 file), NC_NETCDF4 (create netCDF-4/HDF5 file), NC_CLASSIC_MODEL (enforce netCDF classic mode on netCDF-4/HDF5 files), NC_DISKLESS (store data in memory), and NC_PERSIST (force the NC_DISKLESS data from memory to a file), NC_MMAP (use MMAP for NC_DISKLESS instead of NC_INMEMORY – deprecated). See discussion below. |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
The cmode flag is used to control the type of file created, and some aspects of how it may be used.
Setting NC_NOCLOBBER means you do not want to clobber (overwrite) an existing dataset; an error (NC_EEXIST) is returned if the specified dataset already exists.
The NC_SHARE flag is appropriate when one process may be writing the dataset and one or more other processes reading the dataset concurrently; it means that dataset accesses are not buffered and caching is limited. Since the buffering scheme is optimized for sequential access, programs that do not access data sequentially may see some performance improvement by setting the NC_SHARE flag. This flag is ignored for netCDF-4 files.
Setting NC_64BIT_OFFSET causes netCDF to create a 64-bit offset format file, instead of a netCDF classic format file. The 64-bit offset format imposes far fewer restrictions on very large (i.e. over 2 GB) data files. See Large File Support.
Setting NC_64BIT_DATA (alias NC_CDF5) causes netCDF to create a CDF-5 file format that supports large files (i.e. over 2GB) and large variables (over 2B array elements.). See Large File Support.
A zero value (defined for convenience as NC_CLOBBER) specifies the default behavior: overwrite any existing dataset with the same file name and buffer and cache accesses for efficiency. The dataset will be in netCDF classic format. See NetCDF Classic Format Limitations.
Setting NC_NETCDF4 causes netCDF to create a HDF5/NetCDF-4 file.
Setting NC_CLASSIC_MODEL causes netCDF to enforce the classic data model in this file. (This only has effect for netCDF-4/HDF5 files, as CDF-1, 2 and 5 files always use the classic model.) When used with NC_NETCDF4, this flag ensures that the resulting netCDF-4/HDF5 file may never contain any new constructs from the enhanced data model. That is, it cannot contain groups, user defined types, multiple unlimited dimensions, or new atomic types. The advantage of this restriction is that such files are guaranteed to work with existing netCDF software.
Setting NC_DISKLESS causes netCDF to create the file only in memory and to optionally write the final contents to the correspondingly named disk file. This allows for the use of files that have no long term purpose. Operating on an existing file in memory may also be faster. The decision on whether or not to "persist" the memory contents to a disk file is described in detail in the file docs/inmemory.md, which is definitive. By default, closing a diskless fill will cause it's contents to be lost.
If NC_DISKLESS is going to be used for creating a large classic file, it behooves one to use nc__create and specify an appropriately large value of the initialsz parameter to avoid to many extensions to the in-memory space for the file. This flag applies to files in classic format and to file in extended format (netcdf-4).
Note that nc_create(path,cmode,ncidp) is equivalent to the invocation of nc__create(path,cmode,NC_SIZEHINT_DEFAULT,NULL,ncidp).
In this example we create a netCDF dataset named foo.nc; we want the dataset to be created in the current directory only if a dataset with that name does not already exist:
In this example we create a netCDF dataset named foo_large.nc. It will be in the 64-bit offset format.
In this example we create a netCDF dataset named foo_HDF5.nc. It will be in the HDF5 format.
In this example we create a netCDF dataset named foo_HDF5_classic.nc. It will be in the HDF5 format, but will not allow the use of any netCDF-4 advanced features. That is, it will conform to the classic netCDF-3 data model.
In this example we create an in-memory netCDF classic dataset named diskless.nc whose content will be lost when nc_close() is called.
In this example we create a in-memory netCDF classic dataset named diskless.nc and specify that it should be made persistent in a file named diskless.nc when nc_close() is called.
A variant of nc_create(), nc__create() (note the double underscore) allows users to specify two tuning parameters for the file that it is creating.
| int nc_create_mem | ( | const char * | path, |
| int | mode, | ||
| size_t | initialsize, | ||
| int * | ncidp | ||
| ) |
Create a netCDF file with the contents stored in memory.
| path | Must be non-null, but otherwise only used to set the dataset name. |
| mode | the mode flags; Note that this procedure uses a limited set of flags because it forcibly sets NC_INMEMORY. |
| initialsize | (advisory) size to allocate for the created file |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
In this example we use nc_create_mem() to create a classic netCDF dataset named foo.nc. The initial size is set to 4096.
| int nc_create_par | ( | const char * | path, |
| int | cmode, | ||
| MPI_Comm | comm, | ||
| MPI_Info | info, | ||
| int * | ncidp | ||
| ) |
Create a netCDF file for parallel I/O.
This function creates a new netCDF file for parallel I/O access.
Parallel I/O access is only available in library build which support parallel I/O. To support parallel I/O, netCDF must be built with netCDF-4 enabled (configure options –enable-netcdf-4 and –enable-parallel4) and with a HDF5 library that supports parallel I/O, or with support for the PnetCDF library via the –enable-pnetcdf option. This function is collective, i.e. must be called by all MPI processes defined in the MPI communicator, argument comm. In addition, values of arguments of this function must be the same among all MPI processes.
See nc_create() for a fuller discussion of file creation.
| path | The file name of the new netCDF dataset. |
| cmode | The creation mode flag. The following flags are available: NC_CLOBBER (overwrite existing file), NC_NOCLOBBER (do not overwrite existing file), NC_NETCDF4 (create netCDF-4/HDF5 file), NC_CLASSIC_MODEL (enforce netCDF classic mode on netCDF-4/HDF5 files), NC_64BIT_OFFSET (create CDF-2 file), NC_64BIT_DATA (create CDF-5 file). |
| comm | the MPI communicator specifying the processes participating the parallel I/O to this file. |
| info | MPI info object containing I/O hints or MPI_INFO_NULL. |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
In this example from nc_test4/tst_parallel.c, a file is created for parallel I/O.
Definition at line 100 of file dparallel.c.
| int nc_create_par_fortran | ( | const char * | path, |
| int | cmode, | ||
| int | comm, | ||
| int | info, | ||
| int * | ncidp | ||
| ) |
Create a netCDF file for parallel access from the Fortran API.
This function calls nc_create_par() after converting the MPI comm and info from Fortran to C, if necessary.
| path | The file name of the new netCDF dataset. |
| cmode | The creation mode flag. The following flags are available: NC_CLOBBER (overwrite existing file), NC_NOCLOBBER (do not overwrite existing file), NC_NETCDF4 (create netCDF-4/HDF5 file), NC_CLASSIC_MODEL (enforce netCDF classic mode on netCDF-4/HDF5 files), NC_64BIT_OFFSET (create CDF-2 file) NC_64BIT_DATA (create CDF-5 file) |
| comm | the MPI communicator specifying the processes participating the parallel I/O to this file. |
| info | MPI info object containing I/O hints or MPI_INFO_NULL. |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
Definition at line 438 of file dparallel.c.
| int nc_def_user_format | ( | int | mode_flag, |
| NC_Dispatch * | dispatch_table, | ||
| char * | magic_number | ||
| ) |
Add handling of user-defined format.
User-defined formats allow users to write a library which can read their own proprietary format as if it were netCDF. This allows existing netCDF codes to work on non-netCDF data formats.
User-defined formats work by specifying a netCDF dispatch table. The dispatch table is a struct of (mostly) C function pointers. It contains pointers to the key functions of the netCDF API. Once these functions are provided, and the dispatch table is specified, the netcdf-c library can read any format.
| mode_flag | NC_UDF0 or NC_UDF1 |
| dispatch_table | Pointer to dispatch table to use for this user format. |
| magic_number | Magic number used to identify file. Ignored if NULL. |
| int nc_enddef | ( | int | ncid | ) |
Leave define mode
The function nc_enddef() takes an open netCDF dataset out of define mode. The changes made to the netCDF dataset while it was in define mode are checked and committed to disk if no problems occurred. Non-record variables may be initialized to a "fill value" as well with nc_set_fill(). The netCDF dataset is then placed in data mode, so variable data can be read or written.
It's not necessary to call nc_enddef() for netCDF-4 files. With netCDF-4 files, nc_enddef() is called when needed by the netcdf-4 library. User calls to nc_enddef() for netCDF-4 files still flush the metadata to disk.
This call may involve copying data under some circumstances. For a more extensive discussion see File Structure and Performance.
For netCDF-4/HDF5 format files there are some variable settings (the compression, endianness, fletcher32 error correction, and fill value) which must be set (if they are going to be set at all) between the nc_def_var() and the next nc_enddef(). Once the nc_enddef() is called, these settings can no longer be changed for a variable.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
If you use a group id (in a netCDF-4/HDF5 file), the enddef will apply to the entire file. That means the enddef will not just end define mode in one group, but in the entire file.
Here is an example using nc_enddef() to finish the definitions of a new netCDF dataset named foo.nc and put it into data mode:
| int nc_inq | ( | int | ncid, |
| int * | ndimsp, | ||
| int * | nvarsp, | ||
| int * | nattsp, | ||
| int * | unlimdimidp | ||
| ) |
Inquire about a file or group.
| ncid | NetCDF or group ID, from a previous call to nc_open(), nc_create(), nc_def_grp(), or associated inquiry functions such as nc_inq_ncid(). |
| ndimsp | Pointer to location for returned number of dimensions defined for this netCDF dataset. Ignored if NULL. |
| nvarsp | Pointer to location for returned number of variables defined for this netCDF dataset. Ignored if NULL. |
| nattsp | Pointer to location for returned number of global attributes defined for this netCDF dataset. Ignored if NULL. |
| unlimdimidp | Pointer to location for returned ID of the unlimited dimension, if there is one for this netCDF dataset. If no unlimited length dimension has been defined, -1 is returned. Ignored if NULL. If there are multiple unlimited dimensions (possible only for netCDF-4 files), only a pointer to the first is returned, for backward compatibility. If you want them all, use nc_inq_unlimids(). |
Here is an example using nc_inq to find out about a netCDF dataset named foo.nc:
| int nc_inq_format | ( | int | ncid, |
| int * | formatp | ||
| ) |
Inquire about the binary format of a netCDF file as presented by the API.
This function returns the (rarely needed) format version.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
| formatp | Pointer to location for returned format version, one of NC_FORMAT_CLASSIC, NC_FORMAT_64BIT_OFFSET, NC_FORMAT_CDF5, NC_FORMAT_NETCDF4, NC_FORMAT_NETCDF4_CLASSIC. |
| int nc_inq_format_extended | ( | int | ncid, |
| int * | formatp, | ||
| int * | modep | ||
| ) |
Obtain more detailed (vis-a-vis nc_inq_format) format information about an open dataset.
Note that the netcdf API will present the file as if it had the format specified by nc_inq_format. The true file format, however, may not even be a netcdf file; it might be DAP, HDF4, or PNETCDF, for example. This function returns that true file type. It also returns the effective mode for the file.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
| formatp | Pointer to location for returned true format. |
| modep | Pointer to location for returned mode flags. |
Refer to the actual list in the file netcdf.h to see the currently defined set.
| int nc_inq_path | ( | int | ncid, |
| size_t * | pathlen, | ||
| char * | path | ||
| ) |
Get the file pathname (or the opendap URL) which was used to open/create the ncid's file.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
| pathlen | Pointer where length of path will be returned. Ignored if NULL. |
| path | Pointer where path name will be copied. Space must already be allocated. Ignored if NULL. |
| int nc_inq_type | ( | int | ncid, |
| nc_type | xtype, | ||
| char * | name, | ||
| size_t * | size | ||
| ) |
Inquire about a type.
Given an ncid and a typeid, get the information about a type. This function will work on any type, including atomic and any user defined type, whether compound, opaque, enumeration, or variable length array.
For even more information about a user defined type nc_inq_user_type().
| ncid | The ncid for the group containing the type (ignored for atomic types). |
| xtype | The typeid for this type, as returned by nc_def_compound, nc_def_opaque, nc_def_enum, nc_def_vlen, or nc_inq_var, or as found in netcdf.h in the list of atomic types (NC_CHAR, NC_INT, etc.). |
| name | If non-NULL, the name of the user defined type will be copied here. It will be NC_MAX_NAME bytes or less. For atomic types, the type name from CDL will be given. |
| size | If non-NULL, the (in-memory) size of the type in bytes will be copied here. VLEN type size is the size of nc_vlen_t. String size is returned as the size of a character pointer. The size may be used to malloc space for the data, no matter what the type. |
This example is from the test program tst_enums.c, and it uses all the possible inquiry functions on an enum type.
| int nc_inq_user_format | ( | int | mode_flag, |
| NC_Dispatch ** | dispatch_table, | ||
| char * | magic_number | ||
| ) |
Inquire about user-defined format.
| mode_flag | NC_UDF0 or NC_UDF1 |
| dispatch_table | Pointer that gets pointer to dispatch table to use for this user format, or NULL if this user-defined format is not defined. Ignored if NULL. |
| magic_number | Pointer that gets magic number used to identify file, if one has been set. Magic number will be of max size NC_MAX_MAGIC_NUMBER_LEN. Ignored if NULL. |
| int nc_open | ( | const char * | path, |
| int | omode, | ||
| int * | ncidp | ||
| ) |
Open an existing netCDF file.
This function opens an existing netCDF dataset for access. It determines the underlying file format automatically. Use the same call to open a netCDF classic or netCDF-4 file.
| path | File name for netCDF dataset to be opened. When the dataset is located on some remote server, then the path may be an OPeNDAP URL rather than a file path. |
| omode | The open mode flag may include NC_WRITE (for read/write access) and NC_SHARE (see below) and NC_DISKLESS (see below). |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
A zero value (or NC_NOWRITE) specifies the default behavior: open the dataset with read-only access, buffering and caching accesses for efficiency.
Otherwise, the open mode is NC_WRITE, NC_SHARE, or NC_WRITE|NC_SHARE. Setting the NC_WRITE flag opens the dataset with read-write access. ("Writing" means any kind of change to the dataset, including appending or changing data, adding or renaming dimensions, variables, and attributes, or deleting attributes.)
The NC_SHARE flag is only used for netCDF classic files. It is appropriate when one process may be writing the dataset and one or more other processes reading the dataset concurrently; it means that dataset accesses are not buffered and caching is limited. Since the buffering scheme is optimized for sequential access, programs that do not access data sequentially may see some performance improvement by setting the NC_SHARE flag.
This procedure may also be invoked with the NC_DISKLESS flag set in the omode argument if the file to be opened is a classic format file. For nc_open(), this flag applies only to files in classic format. If the file is of type NC_NETCDF4, then the NC_DISKLESS flag will be ignored.
If NC_DISKLESS is specified, then the whole file is read completely into memory. In effect this creates an in-memory cache of the file. If the omode flag also specifies NC_PERSIST, then the in-memory cache will be re-written to the disk file when nc_close() is called. For some kinds of manipulations, having the in-memory cache can speed up file processing. But in simple cases, non-cached processing may actually be faster than using cached processing. You will need to experiment to determine if the in-memory caching is worthwhile for your application.
Normally, NC_DISKLESS allocates space in the heap for storing the in-memory file. If, however, the ./configure flags –enable-mmap is used, and the additional omode flag NC_MMAP is specified, then the file will be opened using the operating system MMAP facility. This flag only applies to files in classic format. Extended format (netcdf-4) files will ignore the NC_MMAP flag.
In most cases, using MMAP provides no advantage for just NC_DISKLESS. The one case where using MMAP is an advantage is when a file is to be opened and only a small portion of its data is to be read and/or written. In this scenario, MMAP will cause only the accessed data to be retrieved from disk. Without MMAP, NC_DISKLESS will read the whole file into memory on nc_open. Thus, MMAP will provide some performance improvement in this case.
It is not necessary to pass any information about the format of the file being opened. The file type will be detected automatically by the netCDF library.
If a the path is a DAP URL, then the open mode is read-only. Setting NC_WRITE will be ignored.
As of version 4.3.1.2, multiple calls to nc_open with the same path will return the same ncid value.
nc_open()returns the value NC_NOERR if no errors occurred. Otherwise, the returned status indicates an error. Possible causes of errors include:
Note that nc_open(path,omode,ncidp) is equivalent to the invocation of nc__open(path,omode,NC_SIZEHINT_DEFAULT,NULL,ncidp).
Here is an example using nc_open()to open an existing netCDF dataset named foo.nc for read-only, non-shared access:
| int nc_open_mem | ( | const char * | path, |
| int | omode, | ||
| size_t | size, | ||
| void * | memory, | ||
| int * | ncidp | ||
| ) |
Open a netCDF file with the contents taken from a block of memory.
| path | Must be non-null, but otherwise only used to set the dataset name. |
| omode | the open mode flags; Note that this procedure uses a limited set of flags because it forcibly sets NC_INMEMORY. |
| size | The length of the block of memory being passed. |
| memory | Pointer to the block of memory containing the contents of a netcdf file. |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
Here is an example using nc_open_mem() to open an existing netCDF dataset named foo.nc for read-only, non-shared access. It differs from the nc_open() example in that it assumes the contents of foo.nc have been read into memory.
| int nc_open_memio | ( | const char * | path, |
| int | omode, | ||
| NC_memio * | params, | ||
| int * | ncidp | ||
| ) |
Open a netCDF file with the contents taken from a block of memory. Similar to nc_open_mem, but with parameters. Warning: if you do specify that the provided memory is locked, then never pass in non-heap allocated memory. Additionally, if not locked, then do not assume that the memory returned by nc_close_mem is the same as passed to nc_open_memio. You must check before attempting to free the original memory.
| path | Must be non-null, but otherwise only used to set the dataset name. |
| omode | the open mode flags; Note that this procedure uses a limited set of flags because it forcibly sets NC_INMEMORY. |
| params | controlling parameters |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
Here is an example using nc_open_memio() to open an existing netCDF dataset named foo.nc for read-only, non-shared access. It differs from the nc_open_mem() example in that it uses a parameter block.
| int nc_open_par | ( | const char * | path, |
| int | omode, | ||
| MPI_Comm | comm, | ||
| MPI_Info | info, | ||
| int * | ncidp | ||
| ) |
Open an existing netCDF file for parallel I/O.
This function opens an existing netCDF dataset for parallel I/O access. It determines the underlying file format automatically. Use the same call to open a netCDF classic, 64-bit offset, CDF-5, or netCDF-4 file.
Parallel I/O access is only available in library build which support parallel I/O. To support parallel I/O, netCDF must be built with netCDF-4 enabled (configure options –enable-netcdf-4 and –enable-parallel4) and with a HDF5 library that supports parallel I/O, or with support for the PnetCDF library via the –enable-pnetcdf option. This function is collective, i.e. must be called by all MPI processes defined in the MPI communicator, argument comm. In addition, values of arguments of this function must be the same among all MPI processes.
It is not necessary to pass any information about the format of the file being opened. The file type will be detected automatically by the netCDF library.
As of version 4.3.1.2, multiple calls to nc_open_par() with the same path will return the same ncid value.
| path | File name for netCDF dataset to be opened. |
| omode | The open mode flag may be NC_WRITE (for read/write access) or NC_NOWRITE (for read-only access). |
| comm | the MPI communicator specifying the processes participating the parallel I/O to this file. |
| info | MPI info object containing I/O hints or MPI_INFO_NULL. |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
nc_open_par() returns the value NC_NOERR if no errors occurred. Otherwise, the returned status indicates an error. Possible causes of errors include:
Here is an example using nc_open_par() from examples/C/parallel_vara.c.
Definition at line 208 of file dparallel.c.
| int nc_open_par_fortran | ( | const char * | path, |
| int | omode, | ||
| int | comm, | ||
| int | info, | ||
| int * | ncidp | ||
| ) |
This is the same as nc_open_par(), but accepts the MPI comm/info as integers.
| path | File name for netCDF dataset to be opened. |
| omode | The open mode flag may be NC_WRITE (for read/write access) or NC_NOWRITE (for read-only access). |
| comm | the MPI communicator specifying the processes participating the parallel I/O to this file. |
| info | MPI info object containing I/O hints or MPI_INFO_NULL. |
| ncidp | Pointer to location where returned netCDF ID is to be stored. |
nc_open_par() returns the value NC_NOERR if no errors occurred. Otherwise, the returned status indicates an error. Possible causes of errors include:
Definition at line 261 of file dparallel.c.
| int nc_redef | ( | int | ncid | ) |
Put open netcdf dataset into define mode
The function nc_redef puts an open netCDF dataset into define mode, so dimensions, variables, and attributes can be added or renamed and attributes can be deleted.
For netCDF-4 files (i.e. files created with NC_NETCDF4 in the cmode in their call to nc_create()), it is not necessary to call nc_redef() unless the file was also created with NC_STRICT_NC3. For straight-up netCDF-4 files, nc_redef() is called automatically, as needed.
For all netCDF-4 files, the root ncid must be used. This is the ncid returned by nc_open() and nc_create(), and points to the root of the hierarchy tree for netCDF-4 files.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
Here is an example using nc_redef to open an existing netCDF dataset named foo.nc and put it into define mode:
| int nc_set_fill | ( | int | ncid, |
| int | fillmode, | ||
| int * | old_modep | ||
| ) |
Change the fill-value mode to improve write performance.
This function is intended for advanced usage, to optimize writes under some circumstances described below. The function nc_set_fill() sets the fill mode for a netCDF dataset open for writing and returns the current fill mode in a return parameter. The fill mode can be specified as either NC_FILL or NC_NOFILL. The default behavior corresponding to NC_FILL is that data is pre-filled with fill values, that is fill values are written when you create non-record variables or when you write a value beyond data that has not yet been written. This makes it possible to detect attempts to read data before it was written. For more information on the use of fill values see Fill Values. For information about how to define your own fill values see Attribute Conventions.
The behavior corresponding to NC_NOFILL overrides the default behavior of prefilling data with fill values. This can be used to enhance performance, because it avoids the duplicate writes that occur when the netCDF library writes fill values that are later overwritten with data.
A value indicating which mode the netCDF dataset was already in is returned. You can use this value to temporarily change the fill mode of an open netCDF dataset and then restore it to the previous mode.
After you turn on NC_NOFILL mode for an open netCDF dataset, you must be certain to write valid data in all the positions that will later be read. Note that nofill mode is only a transient property of a netCDF dataset open for writing: if you close and reopen the dataset, it will revert to the default behavior. You can also revert to the default behavior by calling nc_set_fill() again to explicitly set the fill mode to NC_FILL.
There are three situations where it is advantageous to set nofill mode:
If the netCDF dataset has an unlimited dimension and the last record was written while in nofill mode, then the dataset may be shorter than if nofill mode was not set, but this will be completely transparent if you access the data only through the netCDF interfaces.
The use of this feature may not be available (or even needed) in future releases. Programmers are cautioned against heavy reliance upon this feature.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
| fillmode | Desired fill mode for the dataset, either NC_NOFILL or NC_FILL. |
| old_modep | Pointer to location for returned current fill mode of the dataset before this call, either NC_NOFILL or NC_FILL. |
Here is an example using nc_set_fill() to set nofill mode for subsequent writes of a netCDF dataset named foo.nc:
| int nc_sync | ( | int | ncid | ) |
Synchronize an open netcdf dataset to disk
The function nc_sync() offers a way to synchronize the disk copy of a netCDF dataset with in-memory buffers. There are two reasons you might want to synchronize after writes:
This function is backward-compatible with previous versions of the netCDF library. The intent was to allow sharing of a netCDF dataset among multiple readers and one writer, by having the writer call nc_sync() after writing and the readers call nc_sync() before each read. For a writer, this flushes buffers to disk. For a reader, it makes sure that the next read will be from disk rather than from previously cached buffers, so that the reader will see changes made by the writing process (e.g., the number of records written) without having to close and reopen the dataset. If you are only accessing a small amount of data, it can be expensive in computer resources to always synchronize to disk after every write, since you are giving up the benefits of buffering.
An easier way to accomplish sharing (and what is now recommended) is to have the writer and readers open the dataset with the NC_SHARE flag, and then it will not be necessary to call nc_sync() at all. However, the nc_sync() function still provides finer granularity than the NC_SHARE flag, if only a few netCDF accesses need to be synchronized among processes.
It is important to note that changes to the ancillary data, such as attribute values, are not propagated automatically by use of the NC_SHARE flag. Use of the nc_sync() function is still required for this purpose.
Sharing datasets when the writer enters define mode to change the data schema requires extra care. In previous releases, after the writer left define mode, the readers were left looking at an old copy of the dataset, since the changes were made to a new copy. The only way readers could see the changes was by closing and reopening the dataset. Now the changes are made in place, but readers have no knowledge that their internal tables are now inconsistent with the new dataset schema. If netCDF datasets are shared across redefinition, some mechanism external to the netCDF library must be provided that prevents access by readers during redefinition and causes the readers to call nc_sync before any subsequent access.
When calling nc_sync(), the netCDF dataset must be in data mode. A netCDF dataset in define mode is synchronized to disk only when nc_enddef() is called. A process that is reading a netCDF dataset that another process is writing may call nc_sync to get updated with the changes made to the data by the writing process (e.g., the number of records written), without having to close and reopen the dataset.
Data is automatically synchronized to disk when a netCDF dataset is closed, or whenever you leave define mode.
| ncid | NetCDF ID, from a previous call to nc_open() or nc_create(). |
| int nc_var_par_access | ( | int | ncid, |
| int | varid, | ||
| int | par_access | ||
| ) |
This function will change the parallel access of a variable from independent to collective and vice versa.
This function is collective, i.e. must be called by all MPI processes defined in the MPI communicator used in nc_create_par() or nc_open_par(). In addition, values of arguments of this function must be the same among all MPI processes.
To obtain a good I/O performance, users are recommended to use collective mode. In addition, switching between collective and independent I/O mode can be expensive.
In netcdf-c-4.7.4 or later, using hdf5-1.10.2 or later, the zlib, szip, fletcher32, and other filters may be used when writing data with parallel I/O. The use of these filters require collective access. Turning on the zlib (deflate) or fletcher32 filter for a variable will automatically set its access to collective if the file has been opened for parallel I/O. Attempts to set access to independent will return NC_EINVAL.
| ncid | NetCDF or group ID, from a previous call to nc_open_par(), nc_create_par(), nc_def_grp(), or associated inquiry functions such as nc_inq_ncid(). |
| varid | Variable ID |
| par_access | NC_COLLECTIVE or NC_INDEPENDENT. |
Here is an example from examples/C/parallel_vara.c which changes the parallel access of a variable and then writes to it.
Definition at line 381 of file dparallel.c.
Generated on Mon Aug 23 2021 13:28:33 for NetCDF. NetCDF is
a Unidata library.