topic Re: Logging with limited log samples? in Firewall and Security Management

Logging with limited log samples?

Mille — Thu, 04 Apr 2024 22:05:11 GMT

Hi Mate,

Do you know any way to enable limited logging on an access rule, say 5% rule hits are logged and the rest is not logged?

Why?
A typical access rule will have log enabled and all matches for that rule will be logged.
Some types of trafic (like DNS, NTP, SNMP and NETBIOS) will generate a lot of hits and you may opt to disable log for this 'noise'.
However that will leave you 'blind' both in terms of the direct trafic and in terms of statical reporting and other data processing.

BR Mille

Re: Logging with limited log samples?

Amir_Senn — Sun, 07 Apr 2024 15:07:54 GMT

There is no partial logging.

Best suggestions I can offer are:

a. You can create build in filter that will filter unwanted logs from results

b. Identify all the features of the noisy logs and instead of the current rule, replace action with layer. Under this layer, create the first rules with a combo of source, destination and service and don't log those. If you keep it well defined I don't think you should have issues.

Re: Logging with limited log samples?

Mille — Mon, 08 Apr 2024 12:18:26 GMT

Amir Senn, Thank you for the suggestions. I was hoping for some kind of partial logging.

/Mille

Re: Logging with limited log samples?

Amir_Senn — Tue, 09 Apr 2024 11:45:41 GMT

You can also change the tracking options on layer suggestion to be session instead of connection. Should lower number of all logs that match the rule.

Re: Logging with limited log samples?

Tomer_Noy — Sun, 05 May 2024 07:57:36 GMT

Actually, there is a feature that is specifically meant for this use-case and allows you to reduce the noise of highly repeating logs.

It's not called "partial logging", but "Session Logs".

If you open the Track settings on your rule with logging, you will see the default logging configuration:

These default settings tell the gateway to create a new log for each connection attempt. In the case of DNS, indeed that means many logs that look the same as machines create a DNS request (and connection) for every DNS resolving.

You can of course switch that rule to Track=None, but that indeed leaves you blind to all this traffic and requests.

Instead, you can modify the Track settings to look like the below:

Deactivate "per Connection" logs and activate "per Session" logs.

Session logs are an aggregation of all connection logs that have the same significant parameters (source, destination, port, action, user, ...). Once a new connection is opened a session log will be sent. Subsequent connections (that share the same significant parameters) will not send another log, instead every 10 minutes a log update will be sent on the session to state how many connections were seen in that time period. That way, if you double click a session log, you can still see how many connection, without having a dedicated log row for each connection.

This reduces the noise and also reduces the load on your log server. We've seen customers reduce their total logging by 30%-60% by utilizing Session logs on noisy rules.

Re: Logging with limited log samples?

Mille — Tue, 07 May 2024 08:35:52 GMT

Thanks for the solution. It work for me.

In my case I also had to make the same adjustment in both the Network access rule and in the Application rule as the policy does not use Layes.

Re: Logging with limited log samples?

Tomer_Noy — Tue, 07 May 2024 08:41:20 GMT

Great to hear 😀

I'm curious, by how much did that reduce your logging on those rules?

For example, from 100K logs per day to 10K?

Re: Logging with limited log samples?

Mille — Tue, 07 May 2024 11:47:22 GMT

Work in progress. An early result is 30K DNS connection logs is reduced to 6K session logs for 24/hours. It's about 80% reduction.