PQSURF

pqsurf - program to split surface bulletins and process the observations

pqsurf [-v] [-x] [-l logdest] [-d datadir] [-q q_path] [-Q private_q_path] [-f feedtype] [-p pattern] [-i interval] [-a age] [-t time] [-o time] [conf_path]

This program replaces the LDM 4 program surf_split. It processes data products from a local product queue (see pq(3)). If the product is a WMO format surface bulletins containing SAO, SYNOP, SHIP, METAR, or SPECI observations, the product is broken apart into individual observations. The observations are repackaged into ldm products, one per product, and queued again, in a private queue. The reason for this re-queuing is to eliminate duplicate observations. The new product identifer is derived from the observation type and has the following form:

SAO: sao tt ccc ddhhmm

where tt is the type of SAO: SA, SP or RS; ccc is the station ID like SFO, LXV, etc; and ddhhmm is the time stamp.

SYNOP: aaxx nnnnn ddhhmm

where nnnnn is the WMO station id (5 digit number).

SHIP: bbxx c* ddhhmm

where c* is the call sign.

METAR: metar cccc ddhhmm

where cccc is the call sign

SPECI: speci cccc ddhhmm

Since the routine hourly reports usually are shipped with the appropriate hour in the WMO identifier, the ddhhmm time string for an observation is initialized from the bulletin header. For SAO reports in particular, bulletin header time string is used. The new product sequence number is original sequence number times 1000 plus the sequence of the individual observation within the product. Other aspects of the product info structure are inherited from the original product.

This program takes responsibility for expiration of old products in the private queue, performing the function of pqexpire(1).

A private instance of pqact(1) is run as subprocess of pqsurf on the private queue when pqsurf is started up. This programs performs subsequent processing of the observations according to specifications in the configuration file.

pqsurf is typically started in the background at the same time as other Unidata LDM processes to insure that all the LDM processes are using the same shared product queue and are in the same process group.

Most of the options and arguments to pqsurf are arguments to the private instance of pqact(1).

-v

Verbose logging. Informative messages (level LOG_INFO) are logged, including a line for each product read out of the product queue and a line for each action invoked on a product. By default, only messages of severity LOG_NOTICE and greater are logged.

-x

Debug logging. Debugging messages (level LOG_DEBUG) are logged.

−l logdest

Log to dest. One of ’’ (system logging daemon), ’-’ (standard error stream), or file logdest. Default is the standard error stream if the process has a controlling terminal (i.e., the process isn’t a daemon); otherwise, either the LDM log file or the system logging daemon (execute this program with just the option ’-?’ to determine which).

−d datadir

The directory relative to which file names are specified in the configuration file. The default is $LDMHOME.

-q q_path

The pathname of the data product queue. The default is $(regutil /queue/path) This configuration default can be overridden by setting the environment variable LDMPQFNAME.

-Q private_q_path

The pathname of the private data product queue used by this program and its child pqact. The default is $(regutil /surf-queue/path) It is important that the private queue be distinct from the input queue. This queue is "denser" than the default product queue; the average product size is only 152 bytes. A queue that has 1 Megabyte of data would need 6881 slots as opposed to the default 512. Use the -S option of pqcreate to specify more slots as follows: pqcreate -c -s 1M -S 6881 ‘regutil /surf-queue/path‘

−f feedtype

Reads from the product queue only products that have a feedtype that is a member of the feedtype set. The default is ‘IDS|DDS’, since these feeds contain WMO format surface bulletins.

−p pattern

Reads from the product queue only products whose identifier matches pattern. The default is ‘^S[AIMNP]’, since this is the set of bulletins that the program knows how to split up.

−i time

Polling interval, in seconds. Check for new products in the product queue every time seconds. When pqsurf is run in the same process group as the programs that insert products into the product queue, a signal informs pqsurf and all other interested processes in the process group whenever a new product is available, so polling is not necessary in this case. The default interval is 15 seconds.

−a age

Maximum product age, in hours, as a floating-point number. Products that were created more than age hours ago will be removed from the private queue. The default is 1.004167 hours (one hour + 15 seconds).

−o time

Offset time, in seconds. Begin scan with products created time seconds earlier than the current time. The default is to start with the time stamp of the oldest product in the private queue.

conf_file

Configuration file. This is the pattern-action file that specifies what to do with each product whose feed type and product identifier match a specified pattern. The format of this file is described in the ‘‘Usage’’ section of the man page for pqact(1). The default is $(regutil /pqsurf/config-path).

SIGTERM

Graceful termination.

SIGINT

Exit immediately.

SIGUSR1

Refresh logging (if configure(1) executed without the "--with-ulog" option) and write status and product statistics to log file.

SIGUSR2

Cyclically increment the verbosity of the program. Assumming the program was started without the -v or -x switches, the first SIGUSR2 will make it verbose and LOG_INFO priority messages will appear. The second will turn on the LOG_DEBUG priority messages as well. A third will take it back to the normal state.

SIGCONT

Wake up if sleeping, because there is a new product in the queue.

pqsurf.conf

pqact(1), ldmd(1), ulog(3), pq(3), syslogd(8), WWW URL http://www.unidata.ucar.edu/software/ldm/.

If you have problems with this program, then you should first examine the LDM email archive for similar problems and how they were solved. The email archive is available via the following World Wide Web URL:

http://www.unidata.ucar.edu/software/ldm/

If this does not suffice and your site is a member of the Unidata program, then send an inquiry via email -- together will all relevant information -- to

support@unidata.ucar.edu