TDS Logging

All the TDS specific log files are located in the ${tomcat_home}/content/thredds/logs directory.

• TDS log files:
• threddsServlet.log
• Other log files
• Log configuration

threddsServlet.log

The threddsServlet.log log file contains the log messages for all requests handled by the TDS. The log file rolls over at the top of every hour with old log files named with the "threddsServlet.log.<yyyy-mm-dd-HH>" pattern. 

All log messages in the threddsServlet.log log file contain: 1) a time stamp; 2) the number of milliseconds since the server was started; 3) a request ID; 4) a logging level indicator ("INFO", "WARN", "ERROR", etc); 5) the location in the code where the message was generated; and 6) the content of the log message. For example, in the following:

2009-06-22T16:12:30.659 -0600 [1217082303][  313654] INFO
  - thredds.server.catalogservice.LocalCatalogServiceController - handleRequestInternal():
  Remote host: 128.117.140.75 - Request: "GET /thredds/catalog.html HTTP/1.1"

the first line contains items 1-4, the second line contains item 5, and the last line contains item 6. [NOTE: This message would all be on a single line in the log file.]

Transaction Bracketing

Each HTTP request handled by the TDS results in at least two log messages:

1. a message indicating the start of the transaction; and
2. a message indicating the end of the transaction.

Between the start and end messages, there may be other entries for this same request (e.g., debugging, warning, and error messages).

All log messages for the same request have the same request ID. Log messages from multiple requests can be interleaved in the log file. When tracking down an error, it is useful to find a specific example of a request that causes that error and extract all the log messages for that request.

Start of Transaction

The main message in a start of transaction log message includes 1) the IP address of the requester and 2) the request being made. For example:

... Remote host: 128.117.140.75 - Request: "GET /thredds/catalog.html HTTP/1.1"

End of Transaction

The main message in an end of transaction log message includes: 1) the status code for the response; 2) the size of the response body in bytes; and 3) the time in milliseconds to handle the request. For example:

... Request Completed - 200 - 3403 - 1

the catalog request above returned with a 200 response code, a catalog HTML page 3403 bytes long, and only took one (1) millisecond to handle.

End of Transaction - Status Codes

Most end of transaction messages include a status code that matches a standard HTTP status code. However, when the client breaks the connection or a request is forwarded to another TDS internal service, one of three TDS specific status codes are used:

• "1000 (Client Abort)" - the connection to the client was broken;
• "1001 (Forwarded)" - the request was forwarded to another TDS internal service; or
• "1002 (Forward Failed)" - an attempt was made but failed to forward the request to another TDS internal service.

If the request is forwarded successfully, another set of transaction messages should be initiated with the ending message containing a standard HTTP status code. If forwarding the request fails, another set of transaction message may or may not be initiated and the entry in the access log will probably either log a 400 (Not Found) or a 500 (Internal Server Error) HTTP status code.

Transaction Bracketing Example

Here is an example of the log messages from a request that resulted in error messages:

2009-06-03T21:49:07.147 -0600 [ 104481318][   27031] INFO  - thredds.server.opendap.OpendapServlet - Remote host: 128.117.140.75 - Request: "GET /thredds/dodsC/nexrad/level2/KFTG/20090602/Level2_KFTG_20090602_0346.ar2v.dds HTTP/1.1"
2009-06-03T21:49:07.415 -0600 [ 104481586][   27031] ERROR - thredds.server.opendap.OpendapServlet - path= /nexrad/level2/KFTG/20090602/Level2_KFTG_20090602_0346.ar2v.dds
java.io.IOException: Cant read /data/ldm/pub/native/radar/level2/KFTG/20090602/Level2_KFTG_20090602_0346.ar2v: not a valid NetCDF file.
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:716)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:381)
 ...
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
        at java.lang.Thread.run(Thread.java:619)
2009-06-03T21:49:07.416 -0600 [ 104481587][   27031] INFO  - thredds.server.opendap.OpendapServlet - Request Completed - 500 - -1 - 269

Notice that 1) the request ID (27031) matches in all messages; 2) the ERROR message contains a stack trace; and 3) the status code in the end of transaction message is 500 (Internal Server Error).

Other Log files

• The serverStartup.log file contains messages from the initialization of the TDS. Helpful for debugging when TDS is failing to start.
• The catalogInit.log file contains messages from reading the configuration catalogs. Always look here for problems after you have modified any configuration catalogs.
• The featureCollectionScan.log file shows messages from feature collections. You should examine this for errors periodically.
• There is one fc.<CollectionName>.log file for each feature collection.
• The cache.log file shows cache initialization and scouring messages. Usually you dont need to look at these.

Log4J

TDS version 4.5+ uses the Log4J2 library, and the SLF4J logging facade to route all log messages to Log4J2. (Users of the TDS can ignore the SLF4J facade).

The default log4j2 configuration file is shipped inside the TDS war file. Once the thredds.war file is expanded, you can find it at ${tomcat_home}/webapps/thredds/WEB-INF/classes/log4j2.xml. You can modify this if you want and restart the TDS to have it take effect. However, it will be overwritten when you install an updated thredds.war file.

You can change the location of the log4j2 configuration file by setting the system property log4j.configurationFile to the apsolute path name of your configuration file. Otherwise, the default one will be use.

For debugging log4j, add "status=DEBUG" in the configuration file. See Log4J2 documentation for details.

Log4J threddsServlet configuration

The threddsServlet logs roll over every hour, or every 100 MB. The log4j entry looks like:

    <RollingFile name="threddsServlet" fileName="${tds.log.dir}/threddsServlet.log" filePattern="${tds.log.dir}/threddsServlet.%d{yyyy-MM-dd-HH}.log">
      <PatternLayout pattern="%d{yyyy-MM-dd'T'HH:mm:ss.SSS Z} [%10r][%8X{ID}] %-5p - %c - %m%n"/>
      <Policies>
        <TimeBasedTriggeringPolicy/>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
    </RollingFile>

As you can see, the PatternLayout looks like:

 "%d{yyyy-MM-dd'T'HH:mm:ss.SSS Z} [%10r][%8X{ID}] %-5p - %c - %m%n"

This produces entries in the threddsServlet.log like:

2006-01-24T17:31:41.452 -0700 [2025609][44] INFO - thredds.servlet.ServletUtil - Remote host: 128.117.140.172 - Request: "GET /thredds/idd/radars.xml HTTP/1.1"

MDC fields

The special MDC fields are available to be added to the log, using the syntax X{name}, where name is one of the following:

This document is maintained by Unidata and was last updated Oct 2013. Send comments to THREDDS support.