Modifying Tomcat Access Logs

Purpose Of Tomcat Access Logs

As part of the installation process, we recommend you make the following changes to the Tomcat Access Logging to aid in future troubleshooting and maintenance.

The Tomcat access log records all requests processed by the server. As of Tomcat 7, access logging is enabled by default in ${tomcat_home}/conf/server.xml.

The Tomcat access logs are valuable for monitoring/identifying the users of your THREDDS Data Server and what content/data they are accessing with what frequency.

This information is helpful in gauging the performance of your server.

TDSMonitor Tool

The TDS comes with a monitoring tool, called TdsMonitor, to help analyze your Tomcat access log files and glean the maximum amount the information from the logs.

In order to gain the most use of the TdsMonitor tool, Unidata requests you change the default configuration of the Tomcat AccessLogValve format as outlined below.

Modifying Tomcat Access Logging For The TdsMonitor

Modify the prefix, suffix, and pattern attributes of the AccessLogValve element.

1. Using your favorite editor open ${tomcat_home}/conf/server.xml:

   $ vi server.xml
   

   Locate the AccessLogValve contained in the Host element (should be near the bottom of the file):

   <!-- Define the default virtual host
           Note: XML Schema validation will not work with Xerces 2.2.
     -->
   <Host name="localhost"  appBase="webapps"
         unpackWARs="true" autoDeploy="true">
   
     <!-- SingleSignOn valve, share authentication between web applications
          Documentation at: /docs/config/valve.html -->
     <!--
     <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
     -->
   
     <!-- Access log processes all example.
       Documentation at: /docs/config/valve.html
       Note: The pattern used is equivalent to using pattern="common" -->
       <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
       prefix="localhost_access_log." suffix=".txt"
       pattern="%h %l %u %t "%r" %s %b" />
   
   </Host>
   

   Change the prefix and suffix attributes to customize the access log name. (The TdsMonitor tool looks for log files to begin with “access”.)

   prefix="access"
   suffix=".log"
   

   To provide more useful information about who is accessing the TDS, change the pattern element to customize the format of each log entry:

   pattern="%h %l %u %t "%r" %s %b "%{Referer}i" "%{User-Agent}i" %D"
   

   When you are finished with your edits, the AccessLogValve should look something like the following:

   <Valve className="org.apache.catalina.valves.AccessLogValve"
      directory="logs"  
      prefix="access"
      suffix=".log"
      pattern="%h %l %u %t &quot;%r&quot; %s %b &quot;%{Referer}i&quot; &quot;%{User-Agent}i&quot; %D" />
   

2. Verify the changes to the access log have taken affect. Restart Tomcat and verify an access log has been generated in the ${tomcat_home}/logs/ directory:

   $ ls -l /usr/local/tds/tomcat/logs
   total 164
   -rw-r----- 1 tomcat tomcat     0 Oct 25 07:06 access.2018-10-25.log
   -rw-r----- 1 tomcat tomcat 58233 Oct 24 19:58 catalina.2018-10-24.log
   -rw-r----- 1 root   root    2453 Oct 25 07:06 catalina.2018-10-25.log
   -rw-r----- 1 tomcat tomcat 72826 Oct 25 07:06 catalina.out
   -rw-r----- 1 tomcat tomcat     0 Oct 24 17:43 host-manager.2018-10-24.log
   -rw-r----- 1 tomcat tomcat     0 Oct 25 07:06 host-manager.2018-10-25.log
   -rw-r----- 1 tomcat tomcat  6874 Oct 24 19:58 localhost.2018-10-24.log
   -rw-r----- 1 root   root     566 Oct 25 07:06 localhost.2018-10-25.log
   -rw-r----- 1 tomcat tomcat  3660 Oct 24 19:58 localhost_access_log.2018-10-24.txt
   -rw-r----- 1 tomcat tomcat  1355 Oct 24 19:58 manager.2018-10-24.log
   -rw-r----- 1 tomcat tomcat     0 Oct 25 07:06 manager.2018-10-25.log
   

Access Log Format

The access log entry format we are using is almost identical to the standard combined logging format with an addition: the %D which is used for documenting the Time taken to process the request, in milliseconds will appear at the end of each log entry:

pattern="%h %l %u %t "%r" %s %b "%{Referer}i" "%{User-Agent}i" %D"

The above pattern makes use of the following codes:

• %h - Remote host name (or IP address if resolveHosts is false)
• %l - Remote logical username from identd (always returns ‘-‘)
• %u - Remote user that was authenticated (if any), else ‘-‘
• %t - Date and time, in Common Log Format
• %r - First line of the request (method and request URI)
• %s - HTTP status code of the response
• %b - Bytes sent, excluding HTTP headers, or ‘-‘ if zero
• %D - Time taken to process the request, in millis

The above pattern translates into:

127.0.0.1 - admin [25/Oct/2018:07:12:49 -0600] "GET /manager/html HTTP/1.1" 200 19930 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36" 16

Another way of looking at it: