Suspecting Checkpoint Smart Console forward duplicate log to SIEM

[loeunsokoeun]

Dear Sir/madam,

We suspect smart console forward duplicate checkpoint log to SIEM and make SIEM rule fire offense to event which supposed to not be detected.  Anyone ever face this issue? 

These 2 Smart Defense logs lead SIEM to have 2 duplicate event names with different action (one is prevented, and another is N/A)

LEEF:2.0|Check Point|SmartDefense|1.0|Prevent|devTime=1692599762

LEEF:2.0|Check Point|SmartDefense|1.0|Check Point Log|devTime=1692599762	

Warm regards,

Sokoeun

[PhoneBoy]

The only "duplicate" logs we should send are for accounting logs, and they occur every 10 minutes until the connection is closed.
Otherwise, we should not send duplicate logs.
Have you correlated this with the actual logs in SmartView? 

[loeunsokoeun]

On SmartView I can see there is only one log for the same period of time, but on SIEM there are 3 logs.

[PhoneBoy]

Can you provide a screenshot of the relevant log card (with sensitive details redacted)?
Depending on what the log is, it may very well be expected behavior, especially if the log you're seeing is correlated.

[loeunsokoeun]

Hello @PhoneBoy , please kindly check the attached screenshots. This is example of smartdefense log. On SIEM they have the same category and event ID. And there are other log types that we are seeing duplicate on our SIEM too. 

[G_W_Albrecht]

Forwarding to SIEM is done by SMS/Log server, not smart console. I see one event as Prevent and the other as Log. What does SmartLog show ? How to select what is exported to SIEM is explained in https://support.checkpoint.com/results/sk/sk122323

[loeunsokoeun]

On SmartView I can see there is only one log for the same period of time, but on SIEM there are 3 logs.

[Lloyd_Braun]

Verify that you have your cp_log_export read-mode config set to semi-unified.  Though with that enabled we still get some 'parts' of log cards that come through as different events at the SIEM.

There is a pretty good explanation in the r81 log exporter loguid log field documentation: 

Log Unification ID.
Some Check Point logs are updated over time.
Updated logs have the same Log UID value.
Check Point SmartLog client correlates those updates into a single unified log.
When the update logs are sent to 3rd party servers, they arrive as distinct logs.
Administrators can use the "loguid" field to correlate updated logs and get the full event chain.

Note - Log Exporter's new semi-unified mode correlates all previous logs into one, so the latest log always shows the complete data.
Examples of updated logs:
-The total amount of bytes sent and received over time.
-The severity field which is updated over time as more information becomes available.

[Tomer_Noy]

This is correct.

Just to clarify a bit further:

The gateway sends logs to the log server as soon as it has any meaningful information to share. As more information is available on a connection or attack, the gateway will send additional "update logs" using the same luuid (which is the unique identifier for a log). 

This behavior is very common for Accounting information, since more traffic is passed and we want to update the log server on the latest statistics. However, it also happens for other types of information, for example when a security verdict isn't taken in the first millisecond of identifying an attack/connection. An initial log will be sent with available information, and another update log will be sent later with the action.

Our log server has logic to unify these log updates so that customers will "experience" these as a single log with the latest information. When we export data to SIEM, we have less control over the SIEM vendor, so the log updates may appear as duplicates. For some vendors (such as Splunk) we provide our own dashboards that handle these duplications.

If you export using "semi-unified" mode, then each log update will contain all the information accumulated so far. So if you only look at the latest log, you will see the accurate info to that point. It's also possible to export in "raw" mode, which will send the update log data as it's coming in and each log will have just some of the fields.

We are considering a roadmap development in which you can specify that you only want the "last" log when all information has been accumulated. The benefit is having just one log, which is simpler to handle in SIEM. The drawback is that long-lived connections will only be visible when they are closed. I'm curious to hear your feedback if this is a desired solution.

[elotro]

How can I search in SmartConsole for an event received by our SIEM? Sometimes, our SIEM generate alarms from partial logs, but we can't search by luuid/loguid in SmartConsole

[PhoneBoy]

Not sure if this can be done in SmartConsole/SmartView, but it does appear you can use CPLogFilePrint (a CLI utility) to print the logs for a specific loguid.
From the arguments:

[Expert@R81.20_MGMT:0]# CPLogFilePrint 
Usage: CPLogFilePrint [-m raw|su|io] [-sf unification_schema_file_name] [-nr] [-pc] [-dp] [-sp start_position] [-ep end_position] [-hll_key key_number] [-kernel_hll_key key_number] [-conn_only] [-hll_only] [-hll_resource_only] [-hll_da_only] [-hll_hash] [-remove_line_sep][-f] [-sel selection_definition_file_name] [-back] [-deb] [-indent] [-join] [-resolve_flag] log_file_name
-m : unification mode
raw - raw mode (will print each log separately)
su - semi-unified mode (will iterate each entry in fw.logptr, and for each entry it will unify the log, from log pointed by the entry, until the first log with the same id)
io - initial order mode (will unify log completely - will iterate each entry in fw.loginitial_ptr, and unify all logs with that id)
indexer - indexer mode (will unify the log by raw or by semi-unified modes or will drop it according to the scheme file)
indexer_no_account - indexer mode no account logs
indexer_retrieve_marker - will ask you for the marker its should bring
-sf : unification schema file name (default i s $FWDIR/log/log/log_unification_scheme.C
-nr : don't resolve values
-pc : print the chains
-filter luuid : LUUID as printed, for example: "-filter 0x53ea03aa,0x8,0x7b1b17ac,0xc0000000"
-sp : start position (record number | tail)
-ep : end position (record number | tail)
-hll_key : filter by high level log key
-kernel_hll_key : filter by kernel high level log key
-conn_only : show only connection logs
-hll_only : show only high level logs
-hll_resource_only : show only resource high level logs
-hll_da_only : show only Data Awareness high level logs
-hll_hash : hash and count each hll key
-resolve_flag : resolve values like flags and LogId. this is not compatible with CPLogLogGenerator input text
-remove_line_sep : remove line separator from string_id, for grep to work well
-f : read online
-sel : selection definition file name
-back : read file backward
-deb : print debug info
-indent: will indent table fields. note that in such flavour, each log might take more than one line.
-join: will join table field by unification scheme.

The default log file is $FWDIR/log/fw.log.
fw.log only contains up to 2GB of logs or up to end of current day, whichever comes first.

[elotro]

Thank you very much, but we have tested it, and it can't find the logs associated to a loguid

[PhoneBoy]

I suggest involving TAC to see if there are other options here: https://help.checkpoint.com

[loeunsokoeun]

The current read-mode is semi-unified as I check the configuration. Is there any way to fix cp to not forward duplicate log to SIEM? I guess it is not only mentioned log above. Our SIEM vendor confirm that many same log are forwarded to SIEM by checking the tcpdump.