|
pqexpire - program to delete old products from a Unidata LDM product queue |
|
pqexpire [-v] [-x] [-w] [-l logdest] [-q queue] [-i interval] [-a age] [-f feedtype] [-p pattern] |
|
This program deletes old data products from a local product queue (see pq(3)). It scans from the oldest product in the queue towards the most recent, deleting those which are older than age which also match the feedtype and pattern specifications. If other programs insert products into a product queue, pqexpire may be run on the queue to keep it from eventually filling up, but it is no longer required. The pq library now automatically deletes oldest products from the queue efficiently to make space for inserting a new product, when needed. Although not required, this program may be used with an ldmd(1) server that inserts products from one or more upstream nodes into a queue, or with pqing(1) ingesters that insert LDM products into a queue from a data source. It can be run manually or periodically by cron(1), or by ldmd(1) at startup, from an exec line in the configuration file. Statistics are output about the number and size of products deleted and queue usage. In typical use (see the -i interval option below), the program makes multiple scans through the queue. In this mode, we wish to keep the scans at the "old" end of the queue (away from the action). So, we terminate a scan by by conditions other than reaching the insertion end of the queue. A scan will be terminated by hitting a lock held by another process. It will also be terminated when it reaches a certain point in the queue, which is determined by a heuristic based upon the product latency encountered during the scan and the interval. |
|
-v |
Verbose logging. When this option is specified, for every product that is deleted from the queue a line is emitted that contains information about the product. Queue usage statistics are emitted on each pass, as well. |
||
|
-x |
Debugging information is also emitted. |
||
|
-w |
Wait on region locks. The default is to treat hitting a locked region the same as end of queue. |
|
−l logdest |
|
Log to logdest. One of ’’ (system logging daemon), ’-’ (standard error stream), or file logdest. Default is the standard error stream if the process has a controlling terminal (i.e., the process isn’t a daemon); otherwise, either the LDM log file or the system logging daemon (execute this program with just the option ’-?’ to determine which). |
|
-q queue |
|
The pathname of the data product queue. The default is $(regutil /queue/path) The configuration default can be overridden by setting the environment variable LDMPQFNAME. Use of -q overrides the default and the environment variable. |
|
−i interval |
|
Looping interval, in seconds. The default is 300 seconds. If the interval is zero, the program makes one pass through the queue and exits. When interval is non-zero, the program performs multiple scans of the queue, sleeping interval seconds between each scan. In this case, the program will continue to run until terminated by an external signal. The time reference for product removal is reset on each pass. When you specify an interval of zero, you probably want to specify the -w flag as well. |
|
−a age |
Minimum product age, in hours, as a floating-point number. Only products that were created more than age hours ago will be removed from the queue. The default is 1.0833 hours (one hour + 300 seconds). |
|
−f feedtype |
|
Deletes from the queue only products that have a feedtype that is a member of the feedtype set. The default is ‘ANY’, specifying all feed types. See pqact(1) for a list of basic feed types and the use of feedtype expressions to specify combinations of basic feed types. |
|
−p pattern |
|
Deletes from the product queue only products whose identifier matches the regular expression pattern. The default is ‘.*’, specifying all products. |