With some exceptions (e.g. see the section on redirection) The libcurl authorization mechanisms can be accessed in two ways
http://username:password@host/...This username and password will be used if the server asks for authentication. Note that only simple password authentication is supported in this format. Specifically note that redirection based authorization will not work with this because the username and password will only be used on the initial request, not the redirection
The file must be called one of the following names: ".daprc" or ".dodsrc" If both .daprc and .dodsrc exist, then the .daprc file will take precedence.
The rc file is searched for first in the current directory and then in the home directory (as defined by the HOME environment variable).
The rc file format is a series of lines of the general form:
[<host:port>]<key>=<value>where the bracket-enclosed host:port is optional and will be discussed subsequently.
The currently defined set of authorization-related keys are as follows. The second column is the affected curl_easy_setopt option(s), if any.
Key | Affected curl_easy_setopt Options | Notes |
---|---|---|
HTTP.COOKIEJAR | CURLOPT_COOKIEJAR | |
HTTP.COOKIEFILE | CURLOPT_COOKIEJAR | Alias for CURLOPT_COOKIEJAR |
HTTP.PROXY_SERVER | CURLOPT_PROXY, CURLOPT_PROXYPORT, CURLOPT_PROXYUSERPWD | |
HTTP.SSL.CERTIFICATE | CURLOPT_SSLCERT | |
HTTP.SSL.KEY | CURLOPT_SSLKEY | |
HTTP.SSL.KEYPASSWORD | CURLOPT_KEYPASSWORD | |
HTTP.SSL.CAINFO | CURLOPT_SSLCAINFO | |
HTTP.SSL.CAPATH | CURLOPT_SSLCAPATH | |
HTTP.SSL.VERIFYPEER | CURLOPT_SSL_VERIFYPEER | |
HTTP.SSL.VALIDATE | CURLOPT_SSL_VERIFYPEER, CURLOPT_SSL_VERIFYHOST | |
HTTP.CREDENTIALS.USERPASSWORD | CURLOPT_USERPASSWORD | |
HTTP.NETRC | N.A. | Specify path of the .netrc file |
HTTP.SSL.KEY is essentially the same as HTTP.SSL.CERTIFICATE and should always have the same value.
HTTP.SSL.KEYPASSWORD specifies the password for accessing the HTTP.SSL.CERTIFICAT/HTTP.SSL.key file.
HTTP.SSL.CAPATH specifies the path to a directory containing trusted certificates for validating server certificates.
HTTP.SSL.VALIDATE is a boolean (1/0) value that if true (1) specifies that the client should verify the server's presented certificate.
HTTP.PROXY_SERVER specifies the url for accessing the proxy: e.g. http://[username:password@]host[:port]
HTTP.NETRC specifies the absolute path of the .netrc file. See redirection authorization for information about using .netrc.
The process is usually as follows.
It turns out that libcurl uses the password in the .daprc file — or from the url — only for the initial connection. This causes problems because the redirected connection is the one that actually requires the password. This is where .netrc comes in. Libcurl will use .netrc for the redirected connection. It is possible to cause libcurl to use the .daprc password always, but this introduces a security hole because it may send the initial user+pwd to the redirection site. In summary, if you are using redirection, then you must create a .netrc file to hold the password for the site to which the redirection is sent.
The format of this .netrc file will contain content that typically look like this.
machine mmmmmm login xxxxxx password yyyyyywhere the machine, mmmmmm, is the hostname of the machine to which the client is redirected for authorization, and the login and password are those needed to authenticate on that machine.
The .netrc file can be specified by putting the following line in your .daprc/.dodsrc file.
HTTP.NETRC=<path to netrc file>
One final note. In using this, it is almost certain that you will need to specify a real cookie jar file (HTTP.COOKIEJAR) so that the redirect site can pass back authorization information.
Examples.
[remotetest.unidata.ucar.edu]HTTP.VERBOSE=1 or [fake.ucar.edu:9090]HTTP.VERBOSE=0If the url request from, say, the netcdf_open method has a host+port matching one of the prefixes in the rc file, then the corresponding entry will be used, otherwise ignored.
For example, the URL
http://remotetest.unidata.ucar.edu/thredds/dodsC/testdata/testData.ncwill have HTTP.VERBOSE set to 1.
Similarly,
http://fake.ucar.edu:9090/dts/test.01will have HTTP.VERBOSE set to 0.
Key | curl_easy_setopt Option |
---|---|
HTTP.DEFLATE | CUROPT_DEFLATE with value "deflate,gzip" |
HTTP.VERBOSE | CUROPT_VERBOSE |
HTTP.TIMEOUT | CUROPT_TIMEOUT |
HTTP.USERAGENT | CUROPT_USERAGENT |
HTTP.COOKIEJAR | CUROPT_COOKIEJAR |
HTTP.COOKIE_JAR | CUROPT_COOKIEJAR |
HTTP.PROXY_SERVER | CURLOPT_PROXY, CURLOPT_PROXYPORT, CURLOPT_PROXYUSERPWD |
HTTP.SSL.CERTIFICATE | CUROPT_SSLCERT |
HTTP.SSL.KEY | CUROPT_SSLKEY |
HTTP.SSL.KEYPASSWORD | CUROPT_KEYPASSWORD |
HTTP.SSL.CAINFO | CUROPT_SSLCAINFO |
HTTP.SSL.CAPATH | CUROPT_SSLCAPATH |
HTTP.SSL.VERIFYPEER | CUROPT_SSL_VERIFYPEER |
HTTP.CREDENTIALS.USERPASSWORD | CUROPT_USERPASSWORD |
HTTP.NETRC | CURLOPT_NETRC,CURLOPT_NETRC_FILE |
In order to access ESG datasets, however, it is necessary to register as a user with ESG and to setup your environment so that proper authentication is established between an netcdf client program and the ESG data server. Specifically, it is necessary to use what is called "client-side keys" to enable this authentication. Normally, when a client accesses a server in a secure fashion (using "https"), the server provides an authentication certificate to the client. With client-side keys, the client must also provide a certificate to the server so that the server can know with whom it is communicating.
The netcdf library uses the curl library and it is that underlying library that must be properly configured.
https://ceda.ac.uk/openid/Firstname.Lastname
openssl pkcs8 -topk8 -nocrypt -in x509up_u13615 -inform PEM -out key.der -outform DER openssl x509 -in x509up_u13615 -inform PEM -out cert.der -outform DER java -classpathNote, the file names "key.der" and "cert.der" can be whatever you choose. It is probably best to leave the .der extension, though.-Dkeypassword=" " -Dkeystore=./ key.der cert.der
keytool -trustcacerts -storepass "password" -v -keystore "truststore" -importcert -file "${c}"
As noted, also uses re-direction based authentication. So, when it receives an initial connection from a client, it redirects to a separate authentication server. When that server has authenticated the client, it redirects back to the original url to complete the request.
#!/bin/sh -x KEYSTORE="esgkeystore" TRUSTSTORE="esgtruststore" GLOBUS="globus" TRUSTROOT="certificates" CERT="x509up_u13615" TRUSTROOTPATH="$GLOBUS/$TRUSTROOT" CERTFILE="$GLOBUS/$CERT" PWD="password" D="-Dglobus=$GLOBUS" CCP="bcprov-jdk16-145.jar" CP="./build:${CCP}" JAR="myproxy.jar" # Initialize needed directories rm -fr build mkdir build rm -fr $GLOBUS mkdir $GLOBUS rm -f $KEYSTORE rm -f $TRUSTSTORE # Compile MyProxyCmd and ImportKey javac -d ./build -classpath "$CCP" *.java javac -d ./build ImportKey.java # Execute MyProxyCmd java -cp "$CP myproxy.MyProxyCmd # Build the keystore openssl pkcs8 -topk8 -nocrypt -in $CERTFILE -inform PEM -out key.der -outform DER openssl x509 -in $CERTFILE -inform PEM -out cert.der -outform DER java -Dkeypassword=$PWD -Dkeystore=./${KEYSTORE} -cp ./build ImportKey key.der cert.der # Clean up the certificates in the globus directory for c in ${TRUSTROOTPATH}/*.0 ; do alias=`basename $c .0` sed -e '0,/---/d' <$c >/tmp/${alias} echo "-----BEGIN CERTIFICATE-----" >$c cat /tmp/${alias} >>$c done # Build the truststore for c in ${TRUSTROOTPATH}/*.0 ; do alias=`basename $c .0` echo "adding: $TRUSTROOTPATH/${c}" echo "alias: $alias" yes | keytool -trustcacerts -storepass "$PWD" -v -keystore ./$TRUSTSTORE -alias $alias -importcert -file "${c}" done exit