Event Management

How does Log-Message Normalization work?

NormalizationThe process of standardizing log messages for use by TLC. Standardized messages are known as Normalized Messages. is the process of standardizing a log messageA data record generated by a Log SourceAny log-generating application, operating-system service, database instance, or device from which TLC collects log messages. and collected by TLC. for use by Tripwire Log Center (TLC). A log message that has been normalized by TLC is known as a Normalized MessageA log message that has been normalized by TLC., and a Normalized-Message fieldA field in a Normalized Message. is a field that appears in a Normalized Message.

To normalize a log message, your Primary ManagerEach TLC environmentConsists of all TLC software, Managers, Log Sources, Monitored Assets, Collectors, and data in your TLC installation. has a single Primary Manager that controls: 1. The storing of log messages in the Audit LoggerThe TLC Console1. Tripwire Log Center Console is the software for the TLC graphic user interface (GUI), or 2. The Tripwire Log Center GUI. Through the TLC Console, you can configure TLC, oversee your TLC environment, and manage log and event data. component in which you can work with the log messages collected by TLC. File Store and Events in Event1. Either a log message that a Manager has standardized (i.e. normalized) for use by TLC (a.k.a. Normalized Messages), or an event or vulnerability imported from a scanner. 2. An 'event message' collected from a Log Source.-Management Databases, 2. The configuration settings for your TLC environment, and 3. User access and license management for TLC.'s Normalization EngineThe component of your Primary Manager responsible for normalizing log messages. uses a Normalization RuleDefines a regular expression that can be used to normalize log messages generated by a specific type of Log Source.. Each Normalization Rule defines a .NET regular expression that can only be used to normalize log messages generated by a specific type of Log Source.

To simplify the process of creating and editing Normalization Rules, TLC provides an extensive library of pre-defined regular expressions that may be added to your rules. In addition, you can create custom variables (known as Normalization Aliases) to represent partial or complete regular expressions of your own (see Working with Aliases).

Each Monitored AssetAn object in the TLC Configuration Manager that represents a Log Source from which TLC collects log messages directly. in TLC specifies one or more Output Destinations for log messages generated by the AssetA Log Source in your TLC environment. See Monitored Assets and Discovered Assets.'s Log Source (see Changing the Output Destinations for a Monitored Asset). An Output DestinationAssigned to a Monitored Asset, an Output Destination is either the Audit Logger, an Event-Management Database, or a CorrelationThe examination of Normalized Messages for events of interest, along with the ability to initiate appropriate responses; for example, sending an email notification to specified recipients. Engine that correlates Normalized Messages. is either the Audit Logger, an Event-Management DatabaseAn optional component of your TLC environment, an Event-Management Database stores Events. Types of Event-Management Databases include Event Databases, IDS Databases, and Firewall Databases., or a Correlation EngineThe component of your Primary Manager responsible for identifying events of interest. To correlate events, the Correlation Engine applies Correlation Rules to the Normalized Messages received from the Normalization Engine. that correlates Normalized Messages. For further details about Output Destinations, see:

Where does TLC store Data?

How does Event Correlation work?

Figure 34 shows how TLC normalizes a log message. The following TLC components play a role in the normalization process.

The Parsing UtilityA component of the Normalization Engine that parses each name-value pair in log messages. parses each field/value pair in a log message.

A Clean-Up UtilityA component of the Normalization Engine that standardizes the format of each name-value pair in log messages. standardizes the format of each field/value pair. TLC includes one Clean-Up Utility for Normalized Messages to be stored as Events in IDS Databases, and another Clean-Up Utility for all other Output Destinations.

A Normalized-Message FilterA TLC object that defines a condition(s) to prevent TLC from forwarding some log messages to a specified Event-Management Database(s) or Correlation Engine(s). prevents TLC from forwarding some log messages to a specified Output Destination(s). If a message matches the condition(s) specified by an enabled Normalized-Message Filter, and the message originated with a Monitored Asset that has an Output Destination specified by the filter, TLC will not forward the message to the Output Destination. For example, consider a Monitored Asset that has the default Event DatabaseA type of Event-Management Database that stores Events from any Log Source and/or scannerA device that monitors systems in your TLC environment (for example, a vulnerability scanner).. as an Output Destination, and a Normalized-Message Filter that specifies the database and the Asset's IP address in a condition. If TLC normalizes a message from the Asset, the filter will prevent TLC from saving the Normalized Message in the default Event Database.

The steps below describe the decision-making process involved in the normalization of log messages (outlined in red in Figure 34). For more information about related TLC features, see Log-Message Normalization & Classification.

Figure 34.  Normalizing a log message (click to enlarge)

Does TLC have a Normalization Rule for the log message?

Yes = If the Configuration ManagerIn the Configuration Manager, you can create and configure TLC Resources (Monitored Assets, Asset Groups, Managers, Locations, Event-Management Databases), normalization objects (Normalization Rules, Aliases, and Normalized-Message Filters), and correlation objects (Correlation Engines, Rules, Lists, and Actions). contains an enabled Normalization Rule for the type of Log Source that generated the log message, the Parsing Utility parses the field/value pairs.

No = Otherwise, the Normalization Engine ignores the message.

Output = IDS, ED, and/or CE?

TLC sends the message to the appropriate Clean-Up Utility for the Monitored Asset's Output Destination(s). The Clean-Up Utility then standardizes the field/value pairs in the message.

Output = ED and/or CE?

CE = If a Correlation Engine is an Output Destination for the Monitored Asset, TLC forwards the Normalized Message to the Correlation Engine (see How does Event Correlation work?).

ED = If an Event Database is an Output Destination, TLC applies all enabled Normalized-Message Filters to the message.

Does the Normalized Message match Norm. Message Filter?

Yes = If the Normalized Message matches one of the Normalized-Message Filters, TLC ignores the message.

No = Otherwise, TLC saves the message as an Event in the specified IDS DatabaseA type of Event-Management Database that stores Events from IDS and IPS devices. or Event Database.