Automatic Cleanup of Log Files by Rotatelogs in Secure Gateway 3.0
Secure Gateway 3.0 uses the Apache rotatelogs program to manage the logging of error and access records. See the Apache documentation of the rotatelogs program at:
http://httpd.apache.org/docs-2.0/programs/rotatelogs.html
The Apache rotatelogs program supports automatically starting a new log file whenever the current log file reaches a certain size or has had records written to it for a certain period of time. However, the rotatelogs program, rotatelogs.exe, as implemented in Apache, does not support automatic cleanup of old log files – the administrator has to delete old log files to prevent the disk from filling up. Secure Gateway 3.0 adds the automatic cleanup of old log files by extending the rotatelogs invocation syntax. Secure Gateway 3.0 adds an optional parameter, which specifies the number of days for retention of the log files, to the end of the parameter list passed to rotatelogs.
This optional parameter has the form NumberOfDaysOfFileRetentionD. The Apache rotatelogs syntax:
becomes:
rotatelogs logfile [ rotationtime [ offset ] [ retentiondaysD ]] | [ filesizeM [ retentiondaysD ]]
When you configure Secure Gateway 3.0, the configuration wizard creates the hidden httpd.conf file in the conf folder of the Secure Gateway 3.0 install directory. The Secure Gateway 3.0 configuration wizard does not offer you the option of changing the retention period for log files – if you want to change the default retention period (30 days) you have to edit the httpd.conf file.
To change the retention period for the log files change the number in front of the letter D and save the httpd.conf file. You can use any positive number less than 36500. Note that Secure Gateway 3.0 writes out multiple log files – each rotatelogs directive in the httpd.conf file can specify a different retention period. For example, you can retain error logs for 100 days, while retaining access logs for 30 days.
For the error log, the configuration wizard writes the following rotatelogs parameters to the httpd.conf file:
These parameters specify that rotatelogs, when writing the log file for error records will:
• Place the file in the logs folder, and give it a file name starting with Error_ – the rest of the file name contains a date stamp (more on this soon)
• Check whether to create a new log file every hour (3600 seconds)
• Use a 4-hour offset from the GMT (-240 minutes)
• Retain log files for 30 days (30D)
Whenever the rotatelogs program checks whether to start a new log file (once an hour in the case above) it also checks every log file in the log file directory, created by that rotatelogs process, and deletes those whose latest access, create and modify timestamps are older than the retention period. If the retention days parameter is not specified in the parameters to rotatelogs, or contains an invalid value (for example, a negative number or not a number), rotatelogs does not delete any files.
Rotatelogs deletes log files that match the pattern for the name of the log file. That is, the rotatelogs process for error logging will not delete any access log files, or vice versa. The % symbols in the log file name direct the rotatelogs program to treat them as strftime substitution characters. In the case of error log files, the file name Error_%Y_%m_%d.log causes rotatelogs to create a file with the 4-digit year, 2-digit month and 2-digit day of the month in the file name, separated by ‘_’ character, for example, Error_2005_08_01.log for the file created on August 1, 2005.
This file name pattern, in combination with the 3600 parameter as the rotationtime causes rotatelogs to reopen the same error log file once an hour (every 3600 seconds), because the new file name (Error_yyyy_mm_dd.log) matches the current file name, until rotatelogs executes the hourly processing the first time after midnight. At this point rotatelogs will open a new file, instead of reopening an existing one. The UTC offset ensures that rotatelogs treats the timestamp as the local time, instead of GMT time.
For the list of all ‘%’ symbols that you can use in the file name, and their meanings, see:
http://httpd.apache.org/docs-2.0/programs/rotatelogs.html
Important: For administrators that run Secure Gateway 3.0 with English-US locale: do not use the %c (date and time), %X (time), or %x (date) symbols in the log file name – these result in an invalid log file name for English-US Windows systems. The reasons for invalid file names:
• “time” uses ‘:’ to separate parts of the time stamp, and ‘:’ is an invalid character for a file name.
• “date” uses ‘/’ separators, which make the file look as if it is nested into a sub-directory that does not exist. Rotatelogs will not create the file in a directory which does not exist.
More Information
CTX107581 – Secure Gateway 3.0 Configuration File - httpd.conf
