Possibility to filter for Inspection Settings generated Logs

[MR_K]

Hi CheckMates,

I am not getting anywhere and TAC was unfortunately not able to help either, therefore I hope you can give me hints or a solution.

The question:

Is there a way to filter for logs generated by Inspection Settings (this)?

Background:
A customer requested the Log Exporting of all Threat Prevention logs and active querying how many logs are created and to look into them once the number of logs exceeds a certain threshold. (The reason and sense behind this is decided by someone above me.)

I configured the LogExporter after sk122323  with "filter-blade-in TP" and everyone was good so far.

Then we started to have a huge difference between SmartConsole/SmartView number of logs (filtering on all TP blades) and the exported number of logs, with the exported being about 10 times as many.

Digging deeper into the logs we found that we had huge amounts of logs created by Inspection Settings (like the ones described in sk36869). But since Inspection Settings are part of the Firewall Access Control Policy, all filters with the Threat Prevention Blades did not return these logs.

Since the Firewall Blade returns a lot of other Drops as well, I was searching for a way to filter on parts of the logs that are only present in the Inspection Settings Logs (like Confidence Level, Severity,...) but that did not work.

Do you have any ideas, hints or tips on how I can filter for these logs?
TAC case only lead to me being told to submit a RFE.

Thanks and BR
MR_K

[PhoneBoy]

What were the precise filters you tried?
What version/JHF level?
Can you send me the TAC SR in a PM?

[MR_K]

Hey PhoneBoy

The SMS I struggled on is running R80.40 Take 158, but I also tried it on a R81.10 MDM running Take 55 with no success.

The precise filter I used that worked best for me was this one from SmartView:

(((action:DROP) AND (NOT "Clean-Up Rule")) AND (NOT "Stealth Rule") AND (NOT "First packet isn't SYN")) AND (NOT "ICMP packet that matches an old session") AND (NOT "ICMP redirect packets are not allowed")

things I tried that did not work at all: filtering on all "confidence_level:" options, filtering on all "severity:" options (meaning one filter was (severity:Critical OR severity:High OR severity:Medium OR severity:Low OR severity:Informational). I chose this option since the severity field is not shown and I was hoping to filter out all normal Firewall Blade logs this way, but this did not work.

SR is coming in PM

Thanks

[RS_Daniel]

Hello,

We did somthing similar with one customer. In our case, customer decided to send only login/logout logs for remote access vpn users.

For your case i can think following steps are needed:

i) Create a mappingConfiguration file in XML format that includes all the fields you want to send to your syslog. In case you want to send all the fields on TP logs, you will have to add all fields in this file. You can find entire list of logs fields per product on sk144192. Do not forget to include desired inspection settings fields also.

ii) On fields that are specific to TP and Inspection settings, set the  ‘required’ attribute so only logs that contain this fields are sent.

iii) Set <exportAllFields> parameter to "false" on log exporter configuration file $EXPORTERDIR/targets/<Name of Log Exporter Configuration>/targetConfiguration.xml.

iv) Transfer the mapping configuration file to the $EXPORTERDIR/targets/<Name of Log Exporter Configuration>/ directory.

v) Edit $EXPORTERDIR/targets/<Name of Log Exporter Configuration>/targetConfiguration.xml file and put the name of your mapping configuration file on <mappingConfiguration> section.

v) Restart log exporter to apply new configuration.

HTH

[RS_Daniel]

PS: To create the mapping configuration file i referenced to the unofficial Log Exporter Guide

Look for "Example of the long XML format (more whitespace)", you can use spaces to make it clearer and easier to read. On the following example you can add parameter "required" to each field.

 

[MR_K]

Hi RS_Daniel,

Thanks for the response, but this is not what I am searching for.

I am not looking for a way to change the exported logs, but for a way to filter in SmartConsole or SmartView to see logs created by Inspection Settings.

[Alex- · Accepted solution]

Try maybe this filter: "blade:Firewall protection_type:"Engine Settings"

[MR_K]

Hi Alex-,

Thanks for this tip, it does indeed work on my R81.10 MDM, but on the R80.40 SMS it provides no Log when queried, even though some tcp_block_urg_bit_enable Logs are in the timeframe.

[Alex-]

Does it give you the logs you were looking for? I've only run this on R81/R81.10 SMS, it might be something linked to the version.

[MR_K]

In R81.10 it does give me the logs I am looking for. In R80.40 it does not give me any logs at all (for a time period where there are relevant logs).

[Alex-]

So it's likely linked to the DB improvements on R81.10; you might consider an upgrade of the SMS if feasible. The Blink process makes it actually quite fast and easy provided you take all the usual precautions like pre-upgrade backups and so on.

[PhoneBoy · Accepted solution]

For performance reasons, not every log field is indexed.
I suspect the log fields in question aren’t indexed in R80.40, which is why queries against them don’t work.
R81.10 has some significant under the hood performance improvements in this area and it seems the log fields you’re after are now indexed.

Getting the relevant log fields indexed in R80.40 is likely an RFE.
Upgrading to R81.10 is the recommended course of action here.