LDM Reference Glossary and Index
This is a hyperlinked glossary and index of the terms, and phrases used to explain the fundamentals of the
Unidata LDM.
A
B
C
D
E
F
G
H
I
K
L
M
N
O
P
R
S
T
U
W
V
Y
A
- ACCEPT entry
- An entry in the LDM configuration-file
ldmd.conf that allows an
upstream LDM to send unrequested
data-products.
Description of entry in ldmd.conf.
- ALLOW entry
- An entry in the LDM configuration-file
ldmd.conf that allows a
downstream LDM to connect and indicates what
data-products it is allowed to receive.
- anti-DOS feature
- A feature of the LDM that mitigates deliberate or accidental denial-of-service attacks by ensuring
non-overlapping feeds to every downstream host. This feature is enabled by default but can be disabled through
the enable-anti-DOS parameter of the
LDM registry. See the description of the behavior of
an upstream LDM process for details.
- asynchronous
- Describes something that does not cause the initiator to wait until the thing is finished. For example, an
asynchronous RPC message does not cause the sender to wait for a reply.
- autoshift
- Describes the capability of a downstream LDM 6 to change the
transfer-mode of its connection depending on its success-rate of inserting
data-products into the
product-queue. LDM 6.4 was the first version to have this
capability.
B
- BLKDATA
- A message from an upstream LDM to a
downstream LDM containing
data-product data.
LDM 6 protocol message.
C
- child process
- A UNIX process created by a parent process.
Official description.
- CONDUIT
- A feedtype comprising a huge volume of model output.
- creation-time, data-product
- See data-product creation-time.
- crontab utility
- A UNIX utility for scheduling the periodic execution of commands.
Official description. See
suggested entries for the LDM user's crontab(1) file.
- COMINGSOON
- An RPC message from an upstream LDM to a
downstream LDM containing
data-product metadata asking if the
downstream LDM wants the corresponding
data-product.
LDM 6 protocol message.
D
- data-product
- The fundamental quantum of data acted upon by an LDM.
Structure of a data-product.
- data-product data
- The data portion of a data-product.
Further information.
- data-product creation-time
- The time when the data-product was created by the relevant
LDM ingester.
- data-product identifier
- A character-string that uniquely identifies a data-product. Product
identifiers are created by LDM ingesters
using the policy of the administrative entity responsible for the relevant
feedtype.
- data-product latency
- The difference between when a data-product is received by a
downstream LDM and the
data-product's creation-time.
- data-product metadata
- The metadata portion of a data-product.
Further information.
- data-product origin
- A character-string that identifies the computer on which the LDM
ingester was running when it created the
data-product. Typically, this is the fully-qualified hostname of the
computer.
- data-product selection-criteria
- The criteria used for selecting data-products. The criteria is
composed of a start-time and a stop-time that must bracket the
data-product creation-time of matching products, and a list of pairs of
feedtype and data-product identifier
ERE-s.
- data-product signature
- An array of 16 unsigned bytes that is unique to a data-product.
Signatures are computed by LDM ingesters
using the MD5 checksum algorithm.
An MD5 checksum webpage.
- data-product size
- The number of bytes in the data section of a data-product. The value
must be less than or equal to 0xFFFFFFFF (4294967295).
- decoder
- A feedtype-specific program that disposes of
data-products that it receives from the
pqact process. Further information.
- deprecated
- This item or feature might go away in a future release. Users are cautioned against using or depending
on it.
- downstream host
- The computer on which a downstream LDM is running.
- downstream LDM
- Typically, a forked, child process of
ldmd that sends a FEEDME request to an
upstream LDM and, consequently, receives
data-products, which it then writes into the
product-queue. In atypical situations, this phrase can also refer to a
notifyme process.
- downstream LDM 5
- A downstream LDM that uses the
LDM 5 protocol. For backward compatibility, LDM 6
is capable of acting as a downstream LDM 5.
- downstream LDM 6
- A downstream LDM that uses the
LDM 6 protocol. Behavior.
E
- EXEC entry
- An entry in the LDM configuration-file
ldmd.conf that tells the top-level ldmd to
asynchronously execute a program as a child process on startup.
- environment variable
- A generally mutable object with a name and a character-string value that can be set by a user via the shell or
by a program. Copies of environment variables are inherited by child processes.
Official description.
- execvp() function
- A UNIX function that executes a command-string.
Official description.
- ERE
- Extended Regular Expression. ERE-s provide a mechanism to select specific strings from a set of character
strings. ERE webpage.
F
- FEEDME
- A message from a downstream LDM to an
upstream LDM requesting
data-products.
LDM 6 protocol message.
- feedtype
- A class of data-products coming from a common source (e.g., WMO,
NPORT). Feedtypes may be hierarchically arranged. A feedtype may be represented as an integral value (e.g., 1)
or as a name (e.g., PPS). Feedtype webpage.
- fork
- A UNIX system-call that duplicates the invoking parent process to create a new
child process.
Official description.
G
- gateway LDM
- A site's LDM that receives data from offsite or that sends data to an offsite
LDM
- gdbm database
- A database based on the GNU dbm database
routines.
- GEMPAK
- A UPC-supported package for analyzing and displaying primarily meteorological
data (with emphasis on gridded model output).
GEMPAK homepage.
- gnuplot utility
- A freely-available, 3rd-party UNIX utility for graphing data.
GNU distribution.
- grep utility
- The UNIX utility for searching text files.
Official description.
H
- HEREIS
- A message from an upstream LDM to a
downstream LDM that contains a
data-product.
- HIYA
- An unsolicited message from an upstream LDM to a
downstream LDM specifying what
data-products are about to be sent.
LDM 6 protocol message.
I
- IDD
- The Unidata Internet Data Distribution system. The IDD system is a network of
LDMs and their associated institutions that cooperate to distribute
meteorological and related data-products for research and education
under the auspices of the Unidata program.
IDD homepage.
- IDD rtstats webpages
- A set of webpages for displaying the statistics of non-gateway LDMs on the Unidata
IDD that are reported via the rtstats utility.
Top-level webpage.
Use in monitoring the LDM.
- identifier, data-product
- See data-product identifier.
- ingester
- A feedtype-specific program that creates
data-products and inserts them into the
product-queue.
Further information.
- insertion-time
- The time at which a data-product was inserted into the local
product-queue.
- iostat utility
- A commonly-available UNIX utility for displaying system I/O statistics.
- IS_ALIVE
- A message from a downstream LDM to the top-level
ldmd process on an upstream host
inquiring whether or not the relevant upstream LDM process is still
running. LDM 6 protocol message.
K
- kill utility
- The UNIX utility for sending signals to processes.
Official description.
L
- LDM
- Local Data Manager. A system of software for the distribution and processing of
data-products in near real-time via the Internet.
LDM homepage.
- LDM, downstream
- See downstream LDM.
- LDM, receiving
- See downstream LDM.
- LDM, sending
- See upstream LDM.
- LDM, upstream
- See upstream LDM.
- LDM 4
- An LDM that uses the LDM 4 protocol.
- LDM 4 protocol
- The LDM protocol used by version 4 of the LDM.
- LDM 5
- An LDM that uses the LDM 5 protocol.
- LDM 5 protocol
- The LDM protocol used by version 5 of the LDM.
- LDM 6
- An LDM that uses the LDM 6 protocol.
- LDM 6 protocol
- The LDM protocol used by version 6 of the LDM.
LDM 6 protocol webpage.
- LDM log file
- The file used to log diagnostic messages from LDM programs.
Format and interpretation. Rotation and purging.
- LDM protocol
- The set of RPC messages used by a version of the LDM to
communicate with another LDM of the same version.
- LDM registry
- An LDM-specific database of configuration parameters.
LDM registry webpage.
- LDM user
- The user on a computer system that owns the local LDM installation. Typically,
the user's name is "ldm". LDM user webpage.
Creating user-account for.
Becoming the LDM user.
- ldmadmin
- Perl script for managing an operational LDM system.
Further information.
- ldmd
- The top-level LDM server.
- ldmping
- A utility that temporarily connects to an upstream LDM to test its
availability.
- ldmd.conf
- The configuration-file for the top-level ldmd process.
ldmd.conf webpage.
- lex utility
- A UNIX utility for generating a lexical scanner.
Official description.
- localn
- The facility used by the LDM for logging messages via the syslog daemon. By default,
the local0 facility is used. Configuring the syslog.conf file.
Setting the facility via the
configurescript.
- logfile, LDM
- See LDM logfile.
M
- make utility
- A UNIX utility for maintaining, updating, and regenerating groups of programs.
Official description.
- man utility
- A UNIX utility for displaying reference-documentation on UNIX utilities.
Official description.
User-support example.
- MANPATH
- The environment variable that determines what directories are searched for
manual pages and in what order. Setting in preinstallation step.
User-support reference. The man(1) utility.
- maximum HEREIS size
- The threshold, in bytes, above which an upstream LDM 6 uses a
COMINGSOON/BLKDATA message-pair to
send a data-product rather than a single
HEREIS message.
- McIDAS
- A UPC-supported package for analyzing and displaying primarily meteorological
data (with emphasis on satellite imagery).
McIDAS homepage.
- metadata, data-product
- See data-product metadata.
- minimum virtual residence time
- The minimum, observed virtual residence time since this parameter was
initialized (either by creation of the product-queue or via the
pqutil utility. If this interval is not greater than the maximum
data-product latency configuration parameter, then a
downstream LDM might not reject a duplicate incoming
data-product.
N
- netstat utility
- A utility for displaying network connections.
- NOAAPORT
- A satellite broadcast network run by NOAA to disseminate weather data to field offices. The
LDM can be built to ingest data-products
from the NOAAPORT broadcast.
- NOTIFICATION
- A message from an upstream LDM to an
downstream LDM
containing data-product metadata.
LDM 6 protocol message.
- NOTIFYME
- An RPC message from a downstream LDM to an
upstream LDM requesting
data-product metadata.
LDM 6 protocol message.
- notifying upstream LDM 5
- An upstream LDM 5 that sends
NOTIFICATION messages to a
downstream LDM 5 rather than
data-products.
- notifyme utility
- A utility that makes a NOTIFYME request to an
upstream LDM and, optionally, writes the data-product
metadata that it receives to an
LDM logfile.
- NTP
- The Network Time Protocol. This protocol is used to obtain time information from a time-server. See also
ntpd.
- ntpd
- The Network Time Protocol Daemon. This freely-available program ensures that a computer's system clock is
accurate, which is one of the prerequisites for correct operation of the
LDM. ntp.org home page.
- ntpdate
- A utility that uses the Network Time Protocol (NTP) to either check the accuracy of the system clock or to
set the system clock. See also ntpd.
- NULLPROC
- A message used only to flush or verify a connection.
O
- origin, data-product
- See data-product origin.
P
- parent process
- The UNIX process that created a child process.
Official description.
- pathological regular-expression
- An extended regular-expression (ERE) that is longer than two characters and has a
".*" prefix.
- pathname
- A character-string that identifies a file. Pathnames may be absolute or relative to the current working
directory.
Official description.
- perl utility
- A freely-available scripting utility. Official webpage.
- plotMetrics utility
- An LDM utility for plotting time-series of performance metrics. Uses the
gnuplot utility.
- PIPE action
- See pqact PIPE action.
- portmapper
- A program that maps RPC program numbers and versions to Internet port numbers. The utility
rpcbind is a portmapper.
- pqact
- A program that reads data-products from the
product-queue and disposes of them according to user-configurable
instructions in a pqact configuration-file.
Further information.
- pqact EXEC action
- An entry in a pqact configuration-file that tells the associated
pqact utility to execute the associated program as a
child process when a matching
data-product is encountered.
Configuration-file entry.
- pqact PIPE action
- An entry in a pqact configuration-file that tells the associated
pqact utility to pipe the
data-product data from the matching
data-product to a program that the
pqact utility starts as a child process.
Configuration-file entry.
- pqact configuration-file
- A file that tells the associated pqact utility how to dispose of
data-products. Further information.
- pqcat
- A program that reads data-products from the
product-queue and writes them to a file or to
standard output.
Further information.
- pqcheck
- A program that checks the writer-counter in the
product-queue.
- pqing
- An LDM ingester that reads meteorological data in WMO or AFOS format from
a file or serial port.
- pqinsert
- An LDM ingester that inserts files.
- pqmon
- A program that prints a summary of the state of the product-queue.
Use in monitoring the product-queue.
- pqsend
- A program that sends selected data-products from the
product-queue to an LDM using
HIYA messages and then terminates.
- pqsurf
- A program that reads data-products that are composite WMO surface
bulletins from the product-queue, splits them into individual
bulletins, inserts them as data-products into a separate
product-queue, and processes the individual bulletins similar to
pqact. Deprecated.
- pqutil
- A utility for interacting with the product-queue.
- process
- The address space and system resources in which a program executes on a UNIX system.
Official description. Child process.
- process group
- A collection of processes that can be signaled as a group.
Official description.
- process group leader
- The top-level process of a process group.
Official description.
- product creation-time
- See data-product creation-time.
- product identifier
- See data-product identifier.
- product metadata
- See data-product metadata.
- product origin
- See data-product origin.
- product-queue
- The memory-mapped disk-file used by the LDM system to cache
data-products. Typically,
data-products are written to the product-queue by
LDM ingesters and read from the product-queue
by upstream LDM processes.
- product signature
- See data-product signature.
- product size
- See data-product size.
R
- receiving LDM
- See downstream LDM.
- RECLASS
- An RPC reply that specifies a new data-product selection-criteria.
It may be sent from an upstream LDM to a
downstream LDM in response to a
FEEDME request or from a
downstream LDM 5 to an
upstream LDM 5 in response to a synchronous
LDM 5 HEREIS or COMINGSOON message.
- regex utility
- A program for validating ERE-s and for testing their efficiency.
- registry
- See LDM registry.
- regutil
- A program for managing the contents of the LDM registry. Execute the command man
regutil for more information.
- REQUEST entry
- An entry in the LDM configuration-file, ldmd.conf, that causes a request
to be made to an upstream LDM for
data-products.
ldmd.conf webpage.
- residence time
- The amount of time a data-product spends in the
product-queue before being deleted to make room for more products.
- RPC
- Remote Procedure Call. A system by which a function-call on one computer is turned into a function-invocation
on another computer. The LDM uses a particular implementation called ONC RPC.
ONC RPC specification.
ONC RPC tutorial.
- rpcgen utility
- A program for generating RPC-using C code from an input file.
- rtstats
- A utility that monitors the product-queue and periodically sends
summary statistics to a downstream LDM, which is, typically, at the
Unidata Program Center. The utility is usually started by the top-level
ldmd process due to an entry in the LDM
configuration-file, ldmd.conf.
IDD rtstats webpages.
- runtime symbolic link
- A symbolic link in the LDM user's home directory that references
(i.e., points to) the version-specific LDM subdirectory that is the current operational one.
Removal of.
Creation of.
S
- scour
- A crontab-driven utility for deleting old data-files based on
pathname patterns and retention-times.
- sending LDM
- See upstream LDM.
- sequence number
- An unsigned, 32-bit number sequentially assigned by the ingester that created the
data-product. Ignored by almost everything but existing due to tradition
and history.
- set-uid-root
- If a program is set-uid-root, then it is owned by user "root" and the set-user-ID
bit of the file is enabled.
- set-user-ID
- If a program's set-user-ID bit is on, then, when executed, the user-ID of the resulting process will be that
of the owner of the program rather than that of the user who executed the program.
- SIGCONT
- The UNIX "continue" signal. This signal is used by LDM data-product ingesters and
downstream LDMs to notify the LDM process group that
a data-product has just been added to the product-queue.
- SIGINT
- The UNIX interrupt signal. The default action is to terminate a process.
Official description.
- signature, data-product
- See data-product signature.
- SIGTERM
- The UNIX termination signal. The default action is to terminate a process.
Official description.
- SIGUSR2
- The UNIX, user-defined signal #2. The canonical action taken by an LDM process upon receipt of this signal is
to "rotate" the logging-level: notice -> information -> debug -> notice -> ...
- size, data-product
- See data-product size.
- standard error
- The diagnostic print-stream of a UNIX process.
- standard input
- The standard input-stream of a UNIX process.
- standard output
- The standard print-stream of a UNIX process.
- su utility
- A non-standard but common utility for becoming another user.
- synchronous
- Describes something that causes the initiator to wait until the thing is finished. For example, a synchronous
RPC message causes the sender to wait for a reply.
- syslog daemon
- A configurable UNIX daemon that disposes of log messages sent to it by other processes.
- system group
- A group of UNIX system users. Users within a group will often have write access to the same files and
directories.
Official description.
T
- TCP
- Transport Control Protocol. An low-level Internet protocol for reliably transferring a stream of bytes from
one computer to another.
TCP tutorial (including TCP State Transition Diagram). Original
RFC 793.
- time utility
- A UNIX utility for timing a simple command.
Official description.
- top utility
- A freely-available, 3rd-party UNIX utility for displaying system resources and utilization.
GNU distribution.
- transfer-mode
- One of two modes in which a downstream LDM can receive
data-products. In the primary transfer-mode,
data-products are received via asynchronous
HEREIS messages. In the alternate transfer-mode,
data-products are received via synchronous
COMINGSOON and BLKDATA messages.
U
- uldbutil
- A utility for listing and managing the contents of the
upstream LDM database. Execute man uldbutil for the manual-page.
- Unidata Program Center (UPC)
- The UCAR office that coordinates the Unidata program.
UPC homepage.
- upstream filter
- The data-product filter that an
upstream LDM 6 uses to decide whether to send a
data-product to a
downstream LDM 6. The filter has two parts: 1) an "OK" filter that
data-products must match in order to be sent; and 2) an optional "not"
filter that data-products must not match in order to be sent. Such
filters are specified in the ALLOW entries of the LDM configuration-file,
ldmd.conf.
- upstream host
- The computer on which a upstream LDM is running.
- upstream LDM
- Typically, a forked, child process of
ldmd that reads data-products from the
product-queue and sends the
data-products or the
data-product metadata to a
downstream LDM as a result of a
FEEDME or NOTIFYME request. In atypical
situations, this phrase can also refer to a pqsend process.
- upstream LDM 5
- An upstream LDM that uses the
LDM 5 protocol. For backward compatibility, LDM 6
is capable of acting as an upstream LDM 5.
- upstream LDM 6
- An upstream LDM that uses the
LDM 6 protocol. Behavior.
- upstream LDM database
- A database that contains information on all upstream LDMs running on the local
host. The database is transitory: it exists for each LDM session only. It can be accessed via the
uldbutil utility.
- uptime utility
- A freely-available, 3rd-party UNIX utility for displaying system CPU utilization.
GNU distribution.
- user-account
- The account on a UNIX system for a user of the system.
Official description.
V
- virtual residence time
- The time difference between when a data-product is created and when it
is removed from the product-queue. For each product, it is the sum of
that product's latency plus that product's
- vmstat utility
- A utility for displaying system metrics.
W
- writer-counter
- A field in the product-queue whose value is the number of times the
product-queue has been opened for writing without an intervening close.
Further information.
Y
- yacc utility
- A UNIX utility for generating a grammatical parser.
Official description.