Package ucar.unidata.io.s3

Class CdmS3Uri

  • @Immutable
public final class CdmS3Uri
extends Object

    • Constructor Detail

      • CdmS3Uri

        public CdmS3Uri​(String cdmUriString)
         throws URISyntaxException
        A URI for use by the CDM for identifying a resource on an Object Store, such as AWS S3, Google Cloud Storage, Azure Blob Storage, Ceph, etc.

        Using the generic URI syntax from RFC3986, the CDM will identify resources located in an object store as follows:

        • scheme (required): defined to be cdms3
        • authority (optional for AWS S3, otherwise required): If present, the authority component is preceded by a double slash ("//") and is terminated by the next slash ("/").
          As with the generic URI syntax, the authority is composed of three parts:
          authority = [ userinfo "@" ] host [ ":" port ]
          • userinfo (optional): name of the profile to be used by the AWS SDK
            • Note: netCDF-Java uses the AWS SDK to manage credentials, even for non-AWS object stores. One method for supplying credentials is through the use of a special credentials file, in which named profiles can be used to manage multiple sets of credentials. The profile name referenced here corresponds to a named profile in an AWS credentials file. For more information, please see the AWS Documentation.
          • host (required): host name of the object store
            • Note: If you need to supply a profile name when accessing an AWS S3 object, you must use the generic host name AWS in order to have a valid URI.
          • port(optional): default: 443
        • path (required): path associated with the bucket
          • may not be empty.
          • the final path segment is interpreted to be the name of the object stores bucket.
            • if there are no path segments (no authority component and the path is relative), the path defines the bucket name.
        • query (optional): full or partial object key
          • Only full keys can be used to read an object through the netCDF-Java API.
          • Partial keys are treated as prefixes, and are used by netCDF-Java when, for example, performing bucket listing operations.
        • fragment (optional): configuration options
          • Configuration options may be passed through the fragment on the CDM S3 URI
          • Currently, only one configuration option is available and is used to describe a delimiter for keys that have been designed to be hierarchical. A commonly encountered case is that the object keys are the same as the file path on the system from which they were uploaded. In this case, the delimiter might be the "/" character. If the fragment is not used, netCDF-Java will assume there is no hierarchical structure to the object keys.

        Example 1: cdms3://my-profile@bucket.unidata.ucar.edu:443/data/gfs0p25?26/03/2020/data.nc#delimiter=/

        • scheme: cdms3
        • authority: my-profile@bucket.unidata.ucar.edu:443
          • AWS Profile Name: my-profile
        • path:/data/gfs0p25
          • bucket: gfs0p25
        • query (object key): 26/03/2020/data.nc
        • fragment (configuration): delimiter=/

        Example 2: cdms3:noaa-goes16?ABI-L1b-RadC/2017/242/00/data.nc

        • scheme: cdms3
        • authority: none
        • path: noaa-goes16
          • bucket: noaa-goes16
        • query (object key): ABI-L1b-RadC/2017/242/00/data.nc

        Example cdms3 URIs:

        • cdms3://profile_name@my.endpoint.edu/endpoint/path/bucket-name?super/long/key
        • cdms3://profile_name@my.endpoint.edu/bucket-name?super/long/key
        • cdms3://my.endpoint.edu/endpoint/path/bucket-name?super/long/key
        • cdms3://my.endpoint.edu/bucket-name?super/long/key
        Secure HTTP access is assumed by default. Insecure HTTP access is attempted when of the following ports is explicitly referenced in the authority portion of the cdms3 URI:
        • 80
        • 8008
        • 8080
        • 7001 (WebLogic)
        • 9080 (WebSphere)
        • 16080 (Mac OS X Server)
        Example cdms3 URIs (specific to AWS S3):
        • cdms3:bucket-name?super/long/key
        • cdms3://profile_name@aws/bucket-name?super/long/key
          • *
        • query (optional): full or partial object key
          • *
            *
          • Only full keys can be used to read an object through the netCDF-Java API.
            • *
          • Partial keys are treated as prefixes, and are used by netCDF-Java when, for example, performing bucket listing * operations.
            • *
        Note: In order to supply a profile name (one way to set the region and/or credentials) while maintaining conformance to the URI specification, you may use "aws" as the host. When the generic "aws" host is used, netCDF-Java will ignore the host and allow the AWS SDK to set the appropriate host based on region, as described in the AWS Documentation.

        Parameters:
        cdmUriString - String representation of the cdms3 URI
        Throws:
        URISyntaxException - If the given string violates RFC2396, as augmented by URI, or does not define the required URI components for a cdms3 URI.