Firegen Command Line Interface Analyzer

 

Firegen provides a command-line version of the analysis engine that can be used to scripting or scheduling with special requirements that exceeds the default scheduler service. Firegen40CLI will also provide a more verbose output and it may be helpful for troubleshooting.

The use the command-line analyzer, open a command-line prompt, navigate to the location where Firegen is installed (default: C:\Program Files\Altair Technologies\Firegen 4.0) and type the command:

Firegen40CLI

This command will initiate the analysis of all the analysis profiles that have the “Include in scheduled analysis” option set.

To list the available options, use the -h argument:

Firegen40CLI -h

FireGen 4.0 Command Line Interface
Version: 4.2.1.0

Copyright: Firegen Analytics Inc. – www.firegen.com

Options:
-n – Runs the analysis for all the scheduled profiles and does NOT open the report in the default browser.
-a analysis_profile_name – Runs the analysis just for the specified profile name. If the profile name contains spaces, please use quotations marks for the name.
-r number – Runs the analysis repeatedly for the specified number of times, incrementing the day at each iteration. Typically used to generate stats for anomaly detection.
-s date – The start date for repeat analysis (used in conjunction with option r and a in order to build stats for anomaly detection.
-i – Initialize the stats (removes previous stats).

By default, Firegen40CLI will open the report in the default browser as soon as the analysis is finished. For scheduled report this may not be the desired behaviour. To disable the opening of the report in the browser use the following command:

Firegen40CLI -n

Firegen40CLI can also be used to analyze just on analysis profile, regardless of the “Include in the scheduled analysis” option, by using the “a” argument. For example, if the name of the defined analysis profile is CiscoASA5585, to run the analysis use the following command:

Firegen40CLI -n -a CiscoASA5585

The CLI can also be used to build a baseline for the Firegen anomaly detection engine. Typically, 30-45 days of analysis has to take place before Firegen can have enough data to detect anomalies. With the CLI analyzer, the baseline can be built right after the installation. See the Firegen Log Analyzer Anomaly Detection for more details.

Using cron expressions for Firegen scheduled reports

 

The Firegen scheduler settings provide the option to specify the analysis time and interval using a cron expression. Cron is long used Unix-based command line job scheduler that provides a wide range of options on scheduling a certain task. Quartz, the open-source, scheduling library used by Firegen allows the use of cron type expression in order to run the scheduled reports.

External links that can assist with the cron expressions:

Cron expression builder – www.cronmake r.com
Cron on Wikipedia – en.wikipedia.org/wiki/Cron
Quartz – www.quartz-scheduler.org/api/2.2.1/overview-summary.html
CronTab Guru – crontab.guru

Firegen Custom Log Patterns Explained

When you select one of your logs, if we did not ship a pre-configured log pattern for it, you get a “None of the known patterns matched any of the log entries.” message:

Test started on 8:35:50 AM
Lines analyzed: 1001
None of the known patterns matched any of the log entries.
Last analyzed line: <166>Jul 07 2015 00:00:33 : %ASA-6-302016: Teardown UDP connection 7980317 for dmz:10.2.174.218/1041 to inside:10.1.174.152/53 duration 0:00:00 bytes 183

Please send a small log sample (or just the line listed above) to support@firegen.com and the proper log pattern will be sent to you. If the log that you are trying to analyze is not the list of firewall logs supported by Firegen, again send a full log to mailto:support@firegen.com and we will add support for it.

Check http://www.firegen.com for updates!

The analyzer will display the last log entry that it tried to identify, in this case:

<166>Jul 07 2015 00:00:33 : %ASA-6-302016: Teardown UDP connection 7980317 for dmz:10.2.174.218/1041 to inside:10.1.174.152/53 duration 0:00:00 bytes 183

The log entry pattern syntax represents a regular expression that has to extract 7 details from the log entry:
– Firewall identification
– Year
– Month
– Day
– Time
– Log entry code
– Log entry detailed message

Some syslog servers are not recording all of these (most of the are able to but simply are not configured to do it). In the example above, the syslog server is not recording the IP address or the host name of the device (the firewall) sending the syslog messages. In this case, we will choose the <166> as the identifier (since there is nothing else).

So, now we have to create the regular expression that would extract the following:

1. Firewall: 166
2. Year: 2005
3. Month: Jul
4. Day: 07
5. Time: 00:00:33
6. Code: 6-302016 (for Cisco Pix/ASA/FWSM we discard the %ASA/%Pix/%FWSM code prefix)
7. Message: Teardown UDP connection 7980317 for dmz:10.2.174.218/1041 to inside:10.1.174.152/53 duration 0:00:00 bytes 183

A full tutorial in using regular expressions is beyond the scope of this article but there are many resources available on the Internet (as well as standalone tools). However, here is, shortly, how we wrote the regular expression for this log entry:

<(.*?)>(\D{3}) (\d\d) (\d{4}) (\d\d:\d\d:\d\d).*?(\d-\d{6}): (.+)

1. <(.*?)> – will match any set of characters between the < > tags – this will get us the “166” firewall id 2. (\D{3}) – will match any set of 3 non-digit characters (\D) – this will extract the month: Jul
3. (\d\d) – will match the next set of 2 digits (\d) – the day: 07
4. (\d{4}) – will extract a set of 4 consecutive digits – the year: 2005
5. (\d\d:\d\d:\d\d) – will match 3 pairs of digits separated by a colon – in our case the time: 00:00:33
6. .*? – will match some information that we do not need and that is everything between the time and the next piece of useful information (the code). Since is not in parenthesis, it will not be “extracted” in a match group
7. (\d-\d{6}): – will extract any set of characters that match a digit followed by a dash and then a set of 6 digits, in our case this would be the Cisco ASA entry code: 6-302016. Note that the : is outside the parenthesis so it will not be included in the “capture”
8. (.+) – will get us all the characters – a dot matches any character and the + indicates that everything like the one before should be included in the match. In our case is the rest of the message: Teardown UDP connection 7980317 for dmz:10.2.174.218/1041 to inside:10.1.174.152/53 duration 0:00:00 bytes 183

Once we enter this log pattern, save and click “Test pattern” we will get the following:

Test started on 8:56:04 AM
Lines analyzed: 1
Matching groups:
Matching group 1: 166
Matching group 2: Jul
Matching group 3: 06
Matching group 4: 2015
Matching group 5: 23:58:46
Matching group 6: 6-302016
Matching group 7: Teardown UDP connection 7979511 for outside:10.3.174.12/138 to inside:10.1.174.173/138 duration 0:02:03 bytes 176 (uvcs\profs)

Please verify if the following detected information is correct:
Firewall: 166
Year: Jul
Month: 06
Day: 2015
Hour: 23:58:46
Code: 6-302016
Message: Teardown UDP connection 7979511 for outside:10.3.174.12/138 to inside:10.1.174.173/138 duration 0:02:03 bytes 176 (uvcs\profs)

Last analyzed line: <166>Jul 06 2015 23:58:46 : %ASA-6-302016: Teardown UDP connection 7979511 for outside:10.3.174.12/138 to inside:10.1.174.173/138 duration 0:02:03 bytes 176 (uvcs\profs)

The first part of this message show the “matching groups”, that is the set of data that we extracted (by using parenthesis) from the log entry. As you can see it does extract the information that we need. However, Firegen needs to know which one is which, which group is the firewall id, which one the day, the year, etc… This is where the Field mapping setting gets to be used. The field mapping consists in the number of 1 to 7, comma separated (no spaces) that has to indicate which of the matching groups corresponds to the required fields, in this order: firewall id, year, month, day, time, code and message.

The first digit in the Field mapping should indicate which mapping group is the firewall id, the next one the year, then the month, day, time, code and message. Since the default Field mapping is 1,2,3,4,5,6,7, Firegen will consider that the first set of information captured is the firewall, the second one the year, third one the month, forth one the day and so on, and this will be displayed under the “Please verify if the following…” Looking back at the first section of the message, we need to create the Field mapping syntax to match the order required by Firegen: firewall,year,month,day,time,code,message:

Firewall: Matching group 1: 166
Year: Matching group 4: 2005
Month: Matching group 2: Jul
Day: Matching group 3: 06
Time: Matching group 5: 23:58:46
Code: Matching group 6: 6-302016
Message: Matching group 7:
Teardown UDP connection 7979511 for outside:10.3.174.12/138 to inside:10.1.174.173/138 duration 0:02:03 bytes 176 (uvcs\profs)

Considering the list above, the Field mapping syntax should be: 1,4,2,3,5,6,7. If we enter this in the “Field mapping” setting, click Save and then Test pattern we should now get:

Test started on 9:04:38 AM
Lines analyzed: 1

Matching groups:
Matching group 1: 166
Matching group 2: Jul
Matching group 3: 06
Matching group 4: 2015
Matching group 5: 23:58:46
Matching group 6: 6-302016
Matching group 7: Teardown UDP connection 7979511 for outside:10.3.174.12/138 to inside:10.1.174.173/138 duration 0:02:03 bytes 176 (uvcs\profs)

Please verify if the following detected information is correct:

Firewall: 166
Year: 2015
Month: Jul
Day: 06
Hour: 23:58:46
Code: 6-302016
Message: Teardown UDP connection 7979511 for outside:10.3.174.12/138 to inside:10.1.174.173/138 duration 0:02:03 bytes 176 (uvcs\profs) Last analyzed line: <166>Jul 06 2005 23:58:46 : %PIX-6-302016: Teardown UDP connection 7979511 for outside:10.3.174.12/138 to inside:10.1.174.173/138 duration 0:02:03 bytes 176 (uvcs\profs)

Now the information seems to match the fields required by Firegen so we are ready to analyze the log.

Note: Log entries that do not contain the year should be treated in a similar fashion, however the year will always be set to matching group 7 (so the second entry in the Field mapping should be 7 and the regular expression should only extract the remaining 6 groups).