Working with Find-and-Replace Values in Regular Expressions

In the Replace tab of a Normalization RuleDefines a regular expression that can be used to normalize log messages generated by a specific type of Log SourceAny log-generating application, operating-system service, database instance, or device from which TLC collects log messages..'s properties, you can define find-and-replace values for columns in the content of log messages normalized by the rule. To access this tab, see:

Working with Normalization Rules

Defining a Regular Expression for a Normalization Rule

If a specified column in a log messageA data record generated by a Log Source and collected by TLC. contains the Original Text value, TLC overwrites the value with the New Text value in the Normalized MessageA log message that has been normalized by TLC. (see Table 92). To insert a value without overwriting an existing value, simply enter a null value in the Original Text column (i.e. leave the field empty).

To add a new Column, clickAdd.

To change the find-and-replace values for a Column, select and edit the Column's Original Text and/or New Text values.

To delete a Column, select the Column's row, click the arrow to the left of the row, and clickDelete.

Table 92. The Replace tab

Column	Description
Column	The name of a column in log messages.
Original Text	The column content to be replaced. Tip: The Original Text column supports the use of simple .NET regular expressions. However, the following characters are not supported: | (OR)
New Text	The content with which the rule will overwrite the Original Text.

Creating Integers in Normalized Messages

Some columns require the use of integers in the New Text fields; for example, the ActionA TLC object that initiates a response to Correlated Events created by Correlation Rules. column in a firewall schema. If a Column in a log message contains a hexadecimal number that you want stored as an integer in an Event-Management DatabaseAn optional component of your TLC environmentConsists of all TLC software, Managers, Log Sources, Monitored Assets, Collectors, and data in your TLC installation., an Event1. Either a log message that a Manager has standardized (i.e. normalized) for use by TLC (a.k.a. Normalized Messages), or an event or vulnerability imported from a scanner. 2. An 'event message' collected from a Log Source.-Management Database stores Events. Types of Event-Management Databases include Event Databases, IDS Databases, and Firewall Databases., insert "0x" in front of the column name in the New Text field. For example:

0x<dstport>

The Convert Function

With the Convert function, you can convert the type of data in a log message (see Table 93).

Table 93. Using the Convert function

Field Values	Converts ...
Original Text = Convert:<text> New Text = Base64	... a text value to a Base 64 value
Original Text = Convert:<text> New Text = Hex	... a text value to a hexadecimal value
Original Text =Convert:<host_name> New Text = IP	... a host name to an IP address Note: Host1. A Log Source or a system involved in an Event. 2. A system on which TLC Manager, TLC Console, or Event-Management Database software is installed. names cannot be saved in Event-Management Databases. If the rule's Normalized Messages will be saved as Events in a database, you should use this format to convert any host names in log-message columns.
Original Text =Convert:<ip_address> New Text = Rotate	... a rotated IP address to standard format Note: IP addresses in reverse notation cannot be saved in Event-Management Databases. If the rule's Normalized Messages will be saved as Events in a database, you should use this format to rotate any reversed IP addresses in log-message columns (for instance, changing 1.0.168.192 to 192.168.0.1).

Converting Timestamps

TLC supports a variety of formats for timestamps. To format the timestamps in Events created by a Normalization Rule, use the following variables when defining the variable's .NET regular expression (see Defining a Regular Expression for a Normalization Rule).

<year> - The year can be in either 2-digit or 4-digit format.

<month> - The month as a number or a name.

<day> - Numerical day of the month.

<time> - The time of day in 24-hour time. Format: 00:00:00.

<hour> - The numerical hour of the day in 24-hour time.

<minute> - The numerical value from 0 - 60.

<second> - A numerical value from 0 - 60.

Examples:

(?<month>[A-Za-z]+) (?<day>[\d]+) (?<year>\d+) (?<time>[0-9:]+)

(?<month>[0-9]+) (?<day>[\d]+) (?<year>\d+) (?<time>[0-9:]+)

(?<month>[0-9]+) (?<day>[\d]+) (?<year>\d+) (?<time>[0-9:]+)

(?<month>[0-9]+) (?<day>[\d]+) (?<year>\d+) (?<hour>[0-9]+):(?<minute>[0-9]+):(?<second>[0-9]+)

With the <epochtime> variable, you can convert timestamps formatted in Epoch time to standard time. For example, if a log message contains the following timestamp ...

1200820429.002: %PIX-4-106023: Deny udp src dmz:192.168.0.1/2967...

... you might insert the following content in the rule's regular expression:

(<epochtime>[0-9.]+): %PIX....