System Logs

• There are three main reasons for logging system activities:

1. to monitor for security intrusions or attempts,
2. to tweak performance based on historical system use,
   
3. and to diagnose problems by checking the system state just before the problem occurred.

• Most Linux log files will be stored in /var/log. Certain programs may store logs elsewhere, but can typically be configured to use /var/log as well.
• The log files are all written with the same fields: date and time, host name, program name, and the message itself.
• Logs can also be stored on remote machines. This is useful for the following reasons: consolidates all of the logs into one place for easy monitoring, safeguard log entries from fatal disk failures on the local machine, and add an additional impediment to crackers who will typically want to change the log files to cover their tracks.
• For most programs, logging is managed by two daemons: klogd (kernel log daemon) and syslogd (system log daemon). These two daemons accept log messages from the kernel, and all other programs, respectively, and write them to the appropriate log file/device/remote machine, as directed by the /etc/syslog.conf configuration file.
• The kernel also logs boot-time messages. Because these messages are generated before the rc scripts have a chance to start either the klogd or syslogd daemons, the kernel writes these messages to the "kernel ring buffer". The kernel ring buffer is usally about 8K and size, and overwrites old messages as new ones are added. The dmesg command displays the contents of the kernel ring buffer. The Red Hat distribution also stores the contents of the kernel ring buffer as it exists immediately after system startup in the /var/log/dmesg file. This is so that an administrator can diagnose any boot-time problems even if the kernel ring buffer has long since overwritten all of those messages.
• The /etc/syslog.conf file has its own particular syntax. Each logging decision is phrased as "selector action" where the selector is made up of "facility.priority". The facility and priority codes are described in the syslog Facilities and Priorities handout. Multiple selectors can be placed on a single line, separated by semicolons. The action is either a device file name, a log file name, a remote machine designation, or the * character which means display the message on all user terminals.
• Either the facility code or the priority code can be replaced with a * which means match any facility or priority, respectively.
• The priority may also be "none" which means that no messages for the specified facility will be logged. This syntax is typically used to filter out certain facilities from an encompassing selector on the same line.
• When a priority is specified, it means that all priorities of that level and higher will be logged.
• A comma may be used to list multiple facilities for a single priority. This is just a notational shorthand.
• There are other ways to narrow down the exact priority level, but this won't be covered in the course.

syslog Facilities

A facility is a process category. The syslogd and klogd daemons may take different logging actions based on the facility of the process attempting to write a log entry.

Facility	Description
auth, security	User authentication messages.
auth-priv	Private user authentication messages.
cron	crond daemon messages.
daemon	Generic daemon messages.
kern	Kernel messages.
lpr	Printer daemon messages.
mail	Mail daemon messages.
news	News daemon messages.
syslog	System logging daemon messages.
user	User-process messages.
uucp	Unix-to-Unix-Copy networking messages.
localN	Special-purpose messages defined by certain applications or Linux distributions. N can be between 0 and 7.

syslog Priorities

Each log message is assigned a priority. The following table describes all of the defined priority, in order from lowest to highest (most important).

Priority	Description
debug	Debugging messages used by programmers during software development.
info	Informational messages written during normal program execution.
notice	Noteworthy information messages.
warning, warn	Warnings about potential problems.
err, error	Notices of errors.
crit	Notices of errors that are serious enough to abnormally terminate the program.
alert	Notices of errors that may cause other processes to terminate, in addition to the process logging the message.
emerg, panic	Notices of errors that could potentially crash the kernel.