Difference between revisions of "UM:Log Monitoring"

From NetXMS Wiki
Jump to navigation Jump to search
(Replaced content with "Information moved to documentation: https://www.netxms.org/documentation/adminguide/log-monitoring.html#log-monitoring")
Line 1: Line 1:
{{DISPLAYTITLE:Log Monitoring}}
Information moved to documentation:
= Introduction =
With NetXMS you can monitor changes in text log files, Windows Event Log, and built-in syslog server. All log monitoring done by agents, except for built-in syslog server. In general, log processing goes as following:


# When new line added to log file, it is passed to appropriate log parser;
https://www.netxms.org/documentation/adminguide/log-monitoring.html#log-monitoring
# If line matched one of the patterns, event associated with this pattern sent to server;
# Server receives event and passess to event processing policy as usual, with event source set to node from which event was received.
 
 
= Agent Configuration for Log Monitoring =
To be able to monitor logs with NetXMS agent, you should load LOGWATCH subagent and define parser configuration for each log file you wish to monitor. Example of agent configuration file:
 
<syntaxhighlight lang="ini">
SubAgent = logwatch.nsm
 
# Below is log parsers definitions
*LOGWATCH
Parser = C:\NetXMS\parser1.xml
Parser = C:\NetXMS\parser2.xml
</syntaxhighlight>
 
 
= Syslog Monitoring =
NetXMS has built-in syslog server, which can be used to receive logs from network devices and servers. It is also possible to parse incoming syslog messages in a way similar to Windows Event Log monitoring. To parse syslog messages, LOGWATCH subagent is not needed – parsing is done by the server itself. You should only create parser configuration file for syslog in console via Control Panel – Syslog Parser.
 
 
= Parser Definition File =
== Overview ==
Parser definition file is an XML document with the following structure:
 
<syntaxhighlight lang="xml">
<parser>
<file>file name</file>
<macros>
<macro name="name">macro body</macro>
<!-- more <macro> tags can follow -->
</macros>
<rules>
<rule>
<match>regexp</match>
<id>event id</id>
<level>severity level</level>
<source>event source</source>
<event>event</event>
<context>context</context>
</rule>
<!-- more <rule> tags can follow -->
</rules>
</parser>
</syntaxhighlight>
 
 
Entire <nowiki><macros></nowiki> section can be omited, and inside <nowiki><rule></nowiki> tag only <nowiki><match></nowiki> is mandatory.
 
 
== Global Parser Options ==
In the <nowiki><parser></nowiki> tag you can specify the following options:
 
{| class="wikitable" style="width: 70%"
|-
! Option !! Description !! Default value
|-
| processAll || If this option set to 1, parser will always pass log record through all rules. If this option set to 0, processing will stop after first match. || 0
|-
| trace || Trace level. || 0
|}
 
== <file> Tag ==
In the <nowiki><file></nowiki> tag you should specify log file to apply this parser to. To specify Windows Event Log, prepend it's name with asterisk (*), for example '''<nowiki>*System</nowiki>'''.
 
 
== Macros ==
In the <nowiki><macros></nowiki> section you can define macros for use in matching rules. For example, it can be useful to define macro for a timestamp preceding each log record and use it in matching rules instead of actual regexp. You can define as many macros as you wish, each within it's own <nowiki><macro></nowiki> tag. Each macro should have unique name, defined in '''name''' attribute, and can be used in matching rules in form @{''name''}.
 
Example: you need to parse log file where each line starts with timestamp in format dd/mm/yy HH:MM:SS. You can define the following macro:
 
<syntaxhighlight lang="xml">
<macro name="timestamp">[0-9]{2}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}</macro>
</syntaxhighlight>
 
and then use it in matching rules:
 
<syntaxhighlight lang="xml">
<rules>
<rule>
<match>@{timestamp}.*([A-Za-z]+) failed.*</match>
<event>12345</event>
</rule>
<rule>
<match>@{timestamp}.*error.*</match>
<event>45678</event>
</rule>
</rules>
</syntaxhighlight>
 
Please note that <nowiki><macros></nowiki> section always should be located before <nowiki><rules></nowiki> section in parser definition file.
 
 
== Matching rules ==
In the <nowiki><rules></nowiki> section you define matching rules for log records. Each rule placed inside it's own <nowiki><rule></nowiki> tag. Each rule can have additional options:
 
{| class="wikitable" style="width: 70%"
|-
! Option !! Description !! Default value
|-
| break || If this option set to 1 and curent line match to regular expression in the rule, parser will stop processing of current line, even if global parser option '''processAll''' was set to 1. If this option set to 0 (which is by default), processing will stop according to '''processAll''' option settings. || 0
|-
| context || Name of the context this rule belongs to. If this option is set, rule will be processed only if given context was already activated with '''<nowiki><context></nowiki>''' tag in one of the rules processed earlier (it can be either same line or one of the previous lines). || ''empty''
|}
 
Inside the <nowiki><rule></nowiki> section there are the following additional tags: <nowiki><match>, <description>, <event>, and <context></nowiki>. Only <nowiki><match></nowiki> section is mandatory – it specifies regular expression against which log record should be matched. All other tags are optional and define parser behavior if a record matches the regular expression.
 
 
=== <match> Tag ===
<nowiki>Tag <match> contains a POSIX regular expression that is used to match log records. Parts enclosed in parenthesis can be extracted from log record and passed as arguments of generated event. You can use macros defined in <macros> section. Also, it is possible to define inverted match rules (rules when log </nowiki>record considered matching if it does not match regular expression). Inverted match can be set by setting attribute invert to 1.
 
Some examples:
 
<syntaxhighlight lang="xml">
<match>^Error: (.*)</match>
</syntaxhighlight>
 
This regular expression will match any line starting with word '''Error:''', and everything after this word will be extracted from the log record for use with an event.
 
 
<syntaxhighlight lang="xml">
<match>[0-9]{3}</match>
</syntaxhighlight>
 
This regular expression will match any line containing at least 3 consecutive digits.
 
<syntaxhighlight lang="xml">
<match invert="1">abc</match>
</syntaxhighlight>
 
This regular expression will match any line not containing character sequence '''abc'''.
 
=== <id> Tag ===
Tag <nowiki><id></nowiki> can be used to filter records from Windows Event Log by event ID. You can specify either single event ID or ID range (by using two nubers separated with minus sign). For example:
 
<syntaxhighlight lang="xml">
<id>7</id>
</syntaxhighlight>
 
will match records with event ID equal 7, and
 
<syntaxhighlight lang="xml">
<id>10-20</id>
</syntaxhighlight>
 
will match records with ID in range from 10 to 20 (inclusive).
 
This tag has no effect for text log files, and can be used as a synonym for <nowiki><facility></nowiki> tag for syslog monitoring.
 
 
=== <nowiki><source></nowiki> Tag ===
Tag <nowiki><source></nowiki> can be used to filter records from Windows Event Log by event source. You can specify exact event source name or pattern with * and ? metacharacters.
 
Some examples:
 
<syntaxhighlight lang="xml">
<source>Tcpip</source>
</syntaxhighlight>
 
will match records with event source "Tcpip" (case-insensetive), and
 
<syntaxhighlight lang="xml">
<source>X*</source>
</syntaxhighlight>
 
will match records with event source started from letter "X".
 
This tag has no effect for text log files, and can be used as a synonym for <nowiki><tag></nowiki> tag for syslog monitoring.
 
 
=== <level> Tag ===
Tag <nowiki><level></nowiki> can be used to filter records from Windows Event log by event severity level (also called event type in older Windows versions). Each severity level has it's own code, and to filter by multiple severity levels you should specify sum of appropriate codes. Severity level codes are following:
 
{| class="wikitable"
|-
! Code !! Severity
|-
| 1 || Error
|-
| 2 || Warning
|-
| 4 || Information
|-
| 8 || Audit Success
|-
| 16 || Audit Failure
|-
| 256 || Critical (since 2.0-M4 only on Windows 7/Windows Server 2008 and higher)
|}
 
Some examples:
 
<syntaxhighlight lang="xml">
<level>1</level>
</syntaxhighlight>
 
will match all records with severity level "Error", and
 
<syntaxhighlight lang="xml">
<level>6</level>
</syntaxhighlight>
 
will match all records with severity level "Warning" or "Information".
 
This tag has no effect for text log files, and can be used as a synonym for <nowiki><severity></nowiki> tag for syslog monitoring.
 
=== <facility> Tag ===
Tag <nowiki><facility></nowiki> can be used to filter syslog records (received by NetXMS built-in syslog server) by facility code. The following facility codes can be used:
 
{| class="wikitable"
|-
! Code !! Facility
|-
| 0 || kernel messages
|-
| 1 || user-level messages
|-
| 2 || mail system
|-
| 3 || system daemons
|-
| 4 || security/authorization messages
|-
| 5 || messages generated internally by syslogd
|-
| 6 || line printer subsystem
|-
| 7 || network news subsystem
|-
| 8 || UUCP subsystem
|-
| 9 || clock daemon
|-
| 10 || security/authorization messages
|-
| 11 || FTP daemon
|-
| 12 || NTP subsystem
|-
| 13 || log audit
|-
| 14 || log alert
|-
| 15 || clock daemon
|-
| 16 || local use 0 (local0)
|-
| 17 || local use 1 (local1)
|-
| 18 || local use 2 (local2)
|-
| 19 || local use 3 (local3)
|-
| 20 || local use 4 (local4)
|-
| 21 || local use 5 (local5)
|-
| 22 || local use 6 (local6)
|-
| 23 || local use 7 (local7)
|}
 
You can specify either single facility code or facility code range (by using two numbers separated by minus sign). For example:
 
<syntaxhighlight lang="xml">
<facility>7</facility>
</syntaxhighlight>
 
will match records with facility code equal 7, and
 
<syntaxhighlight lang="xml">
<facility>10-20</facility>
</syntaxhighlight>
 
will match records with facility code in range from 10 to 20 (inclusive).
 
This tag has no effect for text log files, and can be used as a synonym for <nowiki><id></nowiki> tag for Windows Event Log monitoring.
 
 
=== <tag> Tag ===
Tag <nowiki><tag></nowiki> can be used to filter syslog records (received by NetXMS built-in syslog server) by content of "tag" field. You can specify exact value or pattern with * and ? metacharacters.
 
Some examples:
 
<syntaxhighlight lang="xml">
<tag>httpd</tag>
</syntaxhighlight>
 
will match records with tag "httpd" (case-insensetive), and
 
<syntaxhighlight lang="xml">
<tag>X*</tag>
</syntaxhighlight>
 
will match records with tag started from letter "X".
 
This tag has no effect for text log files, and can be used as a synonym for <nowiki><source></nowiki> tag for Windows Event Log monitoring.
 
 
=== <severity> Tag ===
Tag <nowiki><severity></nowiki> can be used to filter syslog records (received by NetXMS built-in syslog server) by severity level. Each severity level has it's own code, and to filter by multiple severity levels you should specify sum of appropriate codes. Severity level codes are following:
 
{| class="wikitable"
|-
! Code !! Severity
|-
| 1 || Emergency
|-
| 2 || Alert
|-
| 4 || Critical
|-
| 8 || Error
|-
| 16 || Warning
|-
| 32 || Notice
|-
| 64 || Informational
|-
| 128 || Debug
|}
 
Some examples:
 
<syntaxhighlight lang="xml">
<severity>1</severity>
</syntaxhighlight>
 
will match all records with severity level "Emergency", and
 
<syntaxhighlight lang="xml">
<severity>6</severity>
</syntaxhighlight>
 
will match all records with severity level "Alert" or "Critical".
 
This tag has no effect for text log files, and can be used as a synonym for <nowiki><level></nowiki> tag for Windows Event Log monitoring.
 
 
=== <description> Tag ===
Tag <nowiki><description></nowiki> contains textual description of the rule, which will be shown in parser trace.
 
 
=== <event> Tag ===
Tag <nowiki><event></nowiki> defines event to be generated if current log record match to regular expression defined in <nowiki><match></nowiki> tag. Inside <nowiki><event></nowiki> tag you should specify event code to be generated (or event name if you configure server-side syslog parsing). If you wish to pass parts of log record text extracted with regular expression as event's parameters, you should specify correct number of parameters in '''params''' attribute.
 
 
=== <context> Tag ===
Tag <nowiki><context></nowiki> defines activation or deactivation of contexts. It has the following format:
 
<syntaxhighlight lang="xml">
<context action="action" reset="reset mode">context name</context>
</syntaxhighlight>
 
Possible actions are:
 
{| class="wikitable"
|-
! Action !! Description
|-
| clear || Deactivate (clear "active" flag of) given context.
|-
| set || Activate (set "active" flag of) given context.
|}
 
Reset mode determines how context will be deactivated (reset). Possible values for reset mode are:
 
{| class="wikitable"
|-
! Reset mode !! Description
|-
| auto || Deactivate context automatically after first match in context (match rule with '''context''' attribute set to given context).
|-
| manual || Context can be deactivated only by explicit <nowiki><context action="clear"></nowiki> statement.
|}
 
Both '''action''' and '''reset''' attributes can be omitted; default value for '''action''' is '''set''', and default value for '''reset''' is '''auto'''.
 
== Examples of Parser Definition File ==
1. Generate event with code 100000 if line in the log file /var/log/messages contains word '''error''':
 
<syntaxhighlight lang="xml">
<parser>
<file>/var/log/messages</file>
<rules>
<rule>
<match>error</match>
<event>100000</event>
</rule>
</rules>
</parser>
</syntaxhighlight>
 
2. Generate event with code 200000 if line in the log file '''C:\demo.log''' contains word '''process:''' and is immediatelly following line containing text '''process startup failed'''; everything after word '''process:''' will be sent as event's parameter.
 
<syntaxhighlight lang="xml">
<parser>
<file>C:\demo.log</file>
<rules>
<rule>
<match>process startup failed</match>
<context action="set" reset="auto">STARTUP_FAILED</context>
</rule>
<rule context="STARTUP_FAILED">
<match>process:(.*)</match>
<event params="1">200000</event>
</rule>
</rules>
</parser>
</syntaxhighlight>
 
 
= Passing parameters to events =
The log parser can send parameters to events.<br>
Anything matched by the regular expression within () will be sent to the event as a parameter.
 
Consider the following line is received via syslog, or added to a monitored file:
<pre>
24.04.2015 12:22:15 1 5 system,error,critical login failure for user testUser from 11.2.33.41 via ssh
</pre>
We can extract username and login method from the syslog message, and pass it as parameters to an event with the following rule:
<pre>
<match>system,error,critical login failure for user (.*) from .* via (.*)</match>
<event params="2">10000</event>
</pre>
Username will be sent to the event as %1, IP address will not be sent, and login method will be sent as %2.

Revision as of 17:03, 24 November 2017

Information moved to documentation:

https://www.netxms.org/documentation/adminguide/log-monitoring.html#log-monitoring

Retrieved from "https://wiki.netxms.org/index.php?title=UM:Log_Monitoring&oldid=1635"