Configuring GINI, NEXRAD, NEXRCOMP, and NOWrad® ADDE Servers

McIDAS-X contains ADDE servers for a variety of data that is stored in non-McIDAS formats:

GINI GOES sectors from NOAAPORT
Nexrad Level III products from NOAAPORT and WSI Corporation
Nexrad Level III composite products from Unidata
NOWrad(tm) Nexrad Level III composite products from WSI Corporation

These servers, giniadir, giniaget, nexradir, nexraget, nowradir, and nowraget, provide users with direct access to the GINI, NEXRAD, and NOWrad (both Lambert Conformal and 5-minute Rectilinear), in their native formats. Decoding into McIDAS AREA format is not necessary.

The design of these servers allows us to break away from the restrictive naming convention inherent in AREA files. Data may be filed using whatever name a site likes and in whatever directory structure that is convenient as long as two simple rules are followed:

Data files are grouped by type and are named similarly
Data file groups must exist in a single directory or in directories that are at the same depth from a single toplevel directory

What this means is that files of a similar type (e.g., the NOWrad 8 km national composites are one type; the NOWrad 2 km national composites are another type; the Nexrad Level III Base Reflectivity at Tilt 1 (N0R) products are another type; GINI GOES-East 1 km VIS images are another; etc.) must reside in a single subdirectory, or in subdirectories on the same level from a single toplevel directory.

As an example, the following table illustrates how NOAAPORT NEXRAD, UPC-generated NEXRCOMP, WSI NIDS and NOWrad, and GINI imagery are currently organized at the UPC.

Data type Directory Location

NOAAPORT zlib-Compressed NEXRAD Level III Products

N0R - Base Reflectivity Tilt 1 /data/ldm/mcidas/images/NIDS/id/N0R

NCR - Composite Reflectivity /data/ldm/mcidas/images/NIDS/id/NCR

N0V - Radial Velocity Tilt 1 /data/ldm/mcidas/images/NIDS/id/N0V

N0S - Storm Relative Radial Velocity Tilt 1 /data/ldm/mcidas/images/NIDS/id/N0S

NPR - Storm Total Precipitation /data/ldm/mcidas/images/NIDS/id/NTP

NVL - Vertically Integrated Liquid Water /data/ldm/mcidas/images/NIDS/id/NVL

etc.

(more NEXRAD Level III image products may be added to NOAAPORT by the NWS in the future)

UPC-generated NEXRAD Composite Products

1KN0R-NAT - 1 km Base Reflectivity Tilt 1 National Composite /data/ldm/mcidas/images/NEXRCOMP/1KN0R-NAT

6KN0R-NAT - 6 km Base Reflectivity Tilt 1 National Composite /data/ldm/mcidas/images/NEXRCOMP/6KN0R-NAT

2KN1P-NAT - 2 km 1-hour Precipitation National Composite /data/ldm/mcidas/images/NEXRCOMP/2KN1P-NAT

4KNTP-NAT - 4 km Storm Total Precipitation National Composite /data/ldm/mcidas/images/NEXRCOMP/4KNTP-NAT

10KRCM-NAT - 10 km Radar Coded Message National Composite /data/ldm/mcidas/images/NEXRCOMP/10KRCM-NAT

1KN0R-FLT - 1 km Base Reflectivity Tilt 1 Regional Composite /data/ldm/mcidas/images/NEXRCOMP/1KN0R-FLT

WSI NIDS Products

BREF1 - Base Reflectivity Tilt 1 /data/ldm/mcidas/images/WSINIDS/id/BREF1

BREF2 - Base Reflectivity Tilt 2 /data/ldm/mcidas/images/WSINIDS/id/BREF2

BREF3 - Base Reflectivity Tilt 3 /data/ldm/mcidas/images/WSINIDS/id/BREF3

BREF4 - Base Reflectivity Tilt 4 /data/ldm/mcidas/images/WSINIDS/id/BREF4

etc.

WSI NOWrad Images

8KM - NOWrad 8 km National Composite /data/ldm/mcidas/images/USRAD/8km/NOW

2KM - NOWrad 2 km Master Sector /data/ldm/mcidas/images/USRAD/2km/NOW

NOAAPORT GINI Images

GE1KVIS - GINI GOES-East 1 km 0.65 um (visible) /data/ldm/mcidas/images/sat/EAST-CONUS/1km/VIS

GE4K39 - GINI GOES-East 4 km 3.9 um (infrared) /data/ldm/mcidas/images/sat/EAST-CONUS/4km/3.9

GE4KWV - GINI GOES-East 4 km 6.5 um (water vapor) /data/ldm/mcidas/images/sat/EAST-CONUS/4km/6.8

GE4KIR - GINI GOES-East 4 km 10.7 um (thermal infrared) /data/ldm/mcidas/images/sat/EAST-CONUS/4km/10.7

GE4KCO2 - GINI GOES-East 4 km 13.3 um (CO2) /data/ldm/mcidas/images/sat/EAST-CONUS/4km/13.3

etc.

Data type	Directory Location
NOAAPORT zlib-Compressed NEXRAD Level III Products
N0R - Base Reflectivity Tilt 1	/data/ldm/mcidas/images/NIDS/id/N0R
NCR - Composite Reflectivity	/data/ldm/mcidas/images/NIDS/id/NCR
N0V - Radial Velocity Tilt 1	/data/ldm/mcidas/images/NIDS/id/N0V
N0S - Storm Relative Radial Velocity Tilt 1	/data/ldm/mcidas/images/NIDS/id/N0S
NPR - Storm Total Precipitation	/data/ldm/mcidas/images/NIDS/id/NTP
NVL - Vertically Integrated Liquid Water	/data/ldm/mcidas/images/NIDS/id/NVL
etc.
(more NEXRAD Level III image products may be added to NOAAPORT by the NWS in the future)

UPC-generated NEXRAD Composite Products
1KN0R-NAT - 1 km Base Reflectivity Tilt 1 National Composite	/data/ldm/mcidas/images/NEXRCOMP/1KN0R-NAT
6KN0R-NAT - 6 km Base Reflectivity Tilt 1 National Composite	/data/ldm/mcidas/images/NEXRCOMP/6KN0R-NAT
2KN1P-NAT - 2 km 1-hour Precipitation National Composite	/data/ldm/mcidas/images/NEXRCOMP/2KN1P-NAT
4KNTP-NAT - 4 km Storm Total Precipitation National Composite	/data/ldm/mcidas/images/NEXRCOMP/4KNTP-NAT
10KRCM-NAT - 10 km Radar Coded Message National Composite	/data/ldm/mcidas/images/NEXRCOMP/10KRCM-NAT
1KN0R-FLT - 1 km Base Reflectivity Tilt 1 Regional Composite	/data/ldm/mcidas/images/NEXRCOMP/1KN0R-FLT

WSI NIDS Products
BREF1 - Base Reflectivity Tilt 1	/data/ldm/mcidas/images/WSINIDS/id/BREF1
BREF2 - Base Reflectivity Tilt 2	/data/ldm/mcidas/images/WSINIDS/id/BREF2
BREF3 - Base Reflectivity Tilt 3	/data/ldm/mcidas/images/WSINIDS/id/BREF3
BREF4 - Base Reflectivity Tilt 4	/data/ldm/mcidas/images/WSINIDS/id/BREF4
etc.
WSI NOWrad Images
8KM - NOWrad 8 km National Composite	/data/ldm/mcidas/images/USRAD/8km/NOW
2KM - NOWrad 2 km Master Sector	/data/ldm/mcidas/images/USRAD/2km/NOW

NOAAPORT GINI Images
GE1KVIS - GINI GOES-East 1 km 0.65 um (visible)	/data/ldm/mcidas/images/sat/EAST-CONUS/1km/VIS
GE4K39 - GINI GOES-East 4 km 3.9 um (infrared)	/data/ldm/mcidas/images/sat/EAST-CONUS/4km/3.9
GE4KWV - GINI GOES-East 4 km 6.5 um (water vapor)	/data/ldm/mcidas/images/sat/EAST-CONUS/4km/6.8
GE4KIR - GINI GOES-East 4 km 10.7 um (thermal infrared)	/data/ldm/mcidas/images/sat/EAST-CONUS/4km/10.7
GE4KCO2 - GINI GOES-East 4 km 13.3 um (CO2)	/data/ldm/mcidas/images/sat/EAST-CONUS/4km/13.3
etc.

In the table above, id represents the NEXRAD station id (ex: FTG, CYS, FFC, PUX, GYX, etc.).

The hierarchical nature of the Nexrad Level III data storage is further illustrated by:


% ls /data/ldm/mcidas/images/NIDS

ARX/ FTG/  CYS/  FFC/  PUX/  GYX/  LIX/  LTX/  HGX/  MQT/  ABX/  MOB/


% ls /data/ldm/mcidas/images/USRAD

8km/  2km/

A listing of the contents of any of the NOAAPORT NEXRAD toplevel directories shows that there are several different kinds of data for each NEXRAD station, and that these are best organized in a directories by data type:


% ls /data/ldm/mcidas/images/NIDS/ARX

NCR/     NTP/     N0R/     N0V/     N0S/     NVL/

Within each type subdirectory, data files are named with a descriptive name that is similar from image to image.


% ls /data/ldm/nexrad/NIDS/FTG/N0R

N0R_19991206_0545   N0R_19991206_0245   N0R_19991206_0035
N0R_19991206_0535   N0R_19991206_0240   N0R_19991206_0030
N0R_19991206_0525   N0R_19991206_0235   N0R_19991206_0025
N0R_19991206_0515   N0R_19991206_0230   N0R_19991206_0020
N0R_19991206_0505   N0R_19991206_0225   N0R_19991206_0010
N0R_19991206_0455   N0R_19991206_0215   N0R_19991206_0005
N0R_19991206_0445   N0R_19991206_0210   N0R_19991206_0000

In order to be able to serve data, the GINI, NEXRCOMP, NEXRAD, and NOWrad servers need to be " told " where the files reside and how they are named. This is done using regular expressions in one of two ways:

if one does not need to restrict access to the data, DSSERVE may be used directly to configure the dataset
if one needs to restrict access to the data, then configuration information is placed in an ASCII text configuration file and DSSERVE is used to tell the server the name of the configuration file

Case 1: Data access is open to the world (most typical)

Redistribution of NOAAPORT Nexrad Level III and UPC-generated Nexrad Level III composite images is not restricted, so DSSERVE can be used directly to specify access to dataset elements.

For the purposes of illustration, assume the following:

we have GOES-East 1 km visible GINI imagery and it is located in the directory:
```
/data/ldm/mcidas/images/sat/EAST-CONUS/1km/VIS
```

the GOES-East 1 km visible image files have names that look like:

VIS_ccyymmdd_hhmm ccyymmdd - 4-digit year, month, and day hhmm - hour and minute

some example names might be:

VIS_20001009_2145  VIS_20001009_2231  VIS_20001009_2315
VIS_20001009_2201  VIS_20001009_2245  VIS_20001009_2331
VIS_20001009_2215  VIS_20001009_2301  VIS_20001009_2345
etc.

we want to access these images from a dataset whose group name is GINIEAST and whose type (descriptor) is GE1KVIS

The DSSERVE command that would be used to define the GINIEAST/GE1KVIS dataset element is:


DSSERVE ADD GINIEAST/GE1KVIS GINI DIRFILE=/data/ldm/mcidas/images/sat/EAST-CONUS/1km/VIS/VIS_* "GINI 1 km VIS East CONUS

Since there are a number of types of GINI images, one would need to run a number of DSSERVE commands to define access to all of them. In order to simplify things, Unidata McIDAS-X is distributed with a BATCH file, GINIADDE.BAT, that contains DSSERVE entries that can be edited and used to define dataset entries for GINI data.

The user should make a copy of GINIADDE.BAT:

Type: cd ~mcidas/data

Type: cp GINIADDE.BAT LOCGINI.BAT

and edit LOCGINI.BAT to set the DIRFILE= value for each GROUP/TYPE (e.g., GINIEAST/GE1KVIS, etc.). Please note that you should NOT edit GINIADDE.BAT directly as this file will be overwritten with each new McIDAS-X installation!

Example:

The skeletal GINIADDE.BAT entry for GINIEAST/GE1KVIS images is:

DSSERVE ADD GINIEAST/GE1KVIS GINI TYPE=IMAGE DIRFILE=/data/ldm/mcidas/images/sat/EAST-CONUS/1km/VIS/VIS_* "e;GINI 1 km VIS East CONUS

If the GOES-East 1 km visible GINI imagery is located in the directory in a different directory, say /local/ldm/data/images/sat/GOES-E/1km/VIS, one would need to change the LOCGINI.BAT entry for GINIEAST/GE1KVIS elements to:

DSSERVE ADD GINIEAST/GE1KVIS GINI TYPE=IMAGE DIRFILE=/local/ldm/data/images/sat/GOES-E/1km/VIS/VIS_* "e;GINI 1 km VIS East CONUS

and so on.

Case 2: Data access is controlled by IP address

If you want to limit access to any of the GINI, Nexrad Level III, Nexrad Level III composite , or NOWrad imagery, you can use the configuration file approach to defining the datasets.

The Unidata McIDAS-X distribution contains example configuration files, for Nexrad Level III products, NNEXRAD.CFG, for Nexrad Level III composite products, NEXRCOMP.CFG, for NOAAPORT GINI images, GINI.CFG, for WSI Nexrad Level III products, WNEXRAD.CFG, and for WSI Nexrad Level III composite products, NOWRAD.CFG, The use of these configuration files is illustrated in the following table:

Data type Configuration File

NOAAPORT NEXRAD Level III Products

N0R - Base Reflectivity Tilt 1 NNEXRAD.CFG

N1R - Base Reflectivity Tilt 2 NNEXRAD.CFG

N2R - Base Reflectivity Tilt 3 NNEXRAD.CFG

N3R - Base Reflectivity Tilt 4 NNEXRAD.CFG

etc.

UPC-generated NEXRAD Composite Images

1KN0R-NAT - 1 km Base Reflectivity Tilt 1 National Composite NEXRCOMP.CFG

6KN0R-NAT - 6 km Base Reflectivity Tilt 1 National Composite NEXRCOMP.CFG

2KN1P-NAT - 2 km 1-hour Precipitation National Composite NEXRCOMP.CFG

4KNTP-NAT - 4 km Storm Total Precipitation National Composite NEXRCOMP.CFG

10KRCM-NAT - 10 km Radar Coded Message National Composite NEXRCOMP.CFG

1KN0R-FLT - 1 km Base Reflectivity Tilt 1 Regional Composite NEXRCOMP.CFG

NOAAPORT GINI Images

GOES-East 1 km 0.65 um (visible) GINI.CFG

GOES-East 4 km 3.9 um (infrared) GINI.CFG

GOES-East 8 km 6.8 um (water vapor) GINI.CFG

GOES-East 4 km 10.7 um (thermal infrared) GINI.CFG

GOES-East 4 km 12.0 um (infrared) GINI.CFG

etc.

WSI NIDS Products

BREF1 - Base Reflectivity Tilt 1 WNEXRAD.CFG

BREF2 - Base Reflectivity Tilt 2 WNEXRAD.CFG

BREF3 - Base Reflectivity Tilt 3 WNEXRAD.CFG

BREF4 - Base Reflectivity Tilt 4 WNEXRAD.CFG

etc.

WSI NOWRAD Images

NOWrad 8 km National Composite NOWRAD.CFG

NOWrad 2 km Master Sector NOWRAD.CFG

Data type	Configuration File
NOAAPORT NEXRAD Level III Products
N0R - Base Reflectivity Tilt 1	NNEXRAD.CFG
N1R - Base Reflectivity Tilt 2	NNEXRAD.CFG
N2R - Base Reflectivity Tilt 3	NNEXRAD.CFG
N3R - Base Reflectivity Tilt 4	NNEXRAD.CFG
etc.
UPC-generated NEXRAD Composite Images
1KN0R-NAT - 1 km Base Reflectivity Tilt 1 National Composite	NEXRCOMP.CFG
6KN0R-NAT - 6 km Base Reflectivity Tilt 1 National Composite	NEXRCOMP.CFG
2KN1P-NAT - 2 km 1-hour Precipitation National Composite	NEXRCOMP.CFG
4KNTP-NAT - 4 km Storm Total Precipitation National Composite	NEXRCOMP.CFG
10KRCM-NAT - 10 km Radar Coded Message National Composite	NEXRCOMP.CFG
1KN0R-FLT - 1 km Base Reflectivity Tilt 1 Regional Composite	NEXRCOMP.CFG

NOAAPORT GINI Images
GOES-East 1 km 0.65 um (visible)	GINI.CFG
GOES-East 4 km 3.9 um (infrared)	GINI.CFG
GOES-East 8 km 6.8 um (water vapor)	GINI.CFG
GOES-East 4 km 10.7 um (thermal infrared)	GINI.CFG
GOES-East 4 km 12.0 um (infrared)	GINI.CFG
etc.
WSI NIDS Products
BREF1 - Base Reflectivity Tilt 1	WNEXRAD.CFG
BREF2 - Base Reflectivity Tilt 2	WNEXRAD.CFG
BREF3 - Base Reflectivity Tilt 3	WNEXRAD.CFG
BREF4 - Base Reflectivity Tilt 4	WNEXRAD.CFG
etc.
WSI NOWRAD Images
NOWrad 8 km National Composite	NOWRAD.CFG
NOWrad 2 km Master Sector	NOWRAD.CFG

The parameters used in these configuration files are very simple. Keywords are used to fully describe the location and name of data files in a dataset and IP address(es) of the machines allowed to access the data. If the location of the files can all be defined by a simple regular expression, then the following keywords need to be defined:

DIRMASK=
FILEMASK=
IPMASK=

Key name Purpose

DIRMASK regular expression for directory containing the data files

FILEMASK regular expression for the data file names
IPMASK regular expression for IP address(es) of allowed machines

Key name	Purpose
DIRMASK	regular expression for directory containing the data files
FILEMASK	regular expression for the data file names
IPMASK	regular expression for IP address(es) of allowed machines

If the location for GINI image files is significantly different from image type to image type, then the following keywords should be used to specified each product:

type_DIR=
type_FILE=
type_IP=

Key name Purpose

type_DIR regular expression for directory containing data files

type_FILE regular expression for data file names
type_IP regular expression for IP address(es) of allowed users

Key name	Purpose
type_DIR	regular expression for directory containing data files
type_FILE	regular expression for data file names
type_IP	regular expression for IP address(es) of allowed users

Here type is to be replaced by the name of the dataset descriptor. For instance a set of group/descriptor definitions might look like:

GINIEAST/GW1KVIS
GINIEAST/GW4K39
GINIEAST/GW8KWV
GINIEAST/GW4KIR
GINIEAST/GW4K12
     ^      ^____ dataset descriptor name
     |___________ dataset group name

The descriptors (e.g., GW1KVIS, GW4K39, etc.) for the dataset GINIEAST might be used in the second configuration file form as follows:

GW1KVIS_DIR=/data/ldm/gini/goeseast/1km/VIS
GW1KVIS_FILE=VIS_*
GW1KVIS_IP=*

GW4K39_DIR=/data/ldm/gini/goeseast/4km/3.9
GW4K39_FILE=3.9_*
GW4K39_IP=*

GW4KWV_DIR=/data/ldm/gini/goeseast/8km/WV
GW4KWV_FILE=WV_*
GW4KWV_IP=*

GW4KIR_DIR=/data/ldm/gini/goeseast/4km/IR
GW4KIR_FILE=IR_*
GW4KIR_IP=*

GW4K12_DIR=/data/ldm/gini/goeseast/4km/12.0
GW4K12_FILE=12.0_*
GW4K12_IP=*

Things to note:

this second configuration file form takes precedence over the first. Also, the last specification that matches a product in the configuration file takes precedence (e.g. last listed, first used).
blank lines are not important
all characters after a # on a line are treated as comments
the syntax for setting the keywords is the same as in McIDAS itself
the descriptor portion of the ADDE dataset name may be referenced symbolically by a token named \TYPE or, equivalently, \PROD. This token can be very useful in compactly describing sets of information.

All of the configuration files get installed in mcidas working directory, ~mcidas/workdata. The user mcidas must make local copies of these files and then edit the copies to change values to match their installation.