CPLogToSyslog Utility Now GA

[PhoneBoy]

Check Point has recently made available publicly a tool that allows you to export Check Point logs from the management to a syslog server.

Refer to the following SK: How to export Check Point logs to a Syslog server using CPLogToSyslog 

[PhoneBoy]

Worth noting this is not for R80.10 just yet, but it is planned for the near term.

[aner_sagi]

Hi team,

on sk117549 supprt explain how to send syslog to microsoft OMS. Those syslog messages need to be in BSD format. My customer have many gateways here and it dont want to specify for each gateway to send logs both to smartcenter and to syslog proxy on my lan.
i tried to use cplogtosyslog utility sk115392 but the logs are in the wrong format.
I am using a package for hotfix 205 i got from you. The issue is i can't specify BSD format on the CPLOGTOSYSLOG config.

thanks in advance
aner.

[PhoneBoy]

I don't believe cplogtosyslog supports this.

However, if you are using R77.30 management, you can configure gateways to send BSD Formatted syslog directly using the following SK: How to Configure R77.30 Security Gateway on Gaia OS to send Firewall logs to an external Syslog serv... 

Note that there are plans to improve syslog support in general during the next several months.

[PhoneBoy]

The R80.10 version of CPLogToSyslog tool is now available in the SK: How to export Check Point logs to a Syslog server using CPLogToSyslog 

[Vladimir]

Dameon, I have tried implementing it in a lab, using utility version provided for R80, but no dice. Can you confirm that it is working and, if at all possible, provide a sample of annotated config file?

In looking for solution to this issue, I have stumbled on suggestions located at using:

fw log -f -t -n -l  2> /dev/null | awk 'NF' | sed '/^$/d' | logger -p local4.info -t CP_FireWall &

which does seem to work, but I would like to understand how the logs on management server will be rotated and if the space allocated to them could be limited and reused.

Thank you,

Vladimir

[PhoneBoy]

I have not heard the tool not working for anyone.

The SK includes a sample annotated configuration file as well as installation and troubleshooting steps.

If you're not understanding something specific in the commands or the output of those commands, please post a more specific question.

For R80 in particular, there are two versions--make sure you're using the right one for your patch level. 

CPLogToSyslog is basically a LEA client that is reading from the Check Point logs similar to how third parties do it.

Those logs are then sent via syslog to the destination you've configured.

What you've presented there as a possible workaround has been well known for some time--I had it on my site back in the day.

As I recall, it's not particularly reliable. 

Management of the storage of logs on the log/manager is controlled by settings on the relevant object in SmartDashboard.

You can configure at what point logs are deleted there.

[Vladimir]

Dameon,

Thank you for prompt reply.

It is likely that I do not understand something, so any words of wisdom are appreciated.

I am attaching the lab notes, config file and the troubleshooting log and will be much obliged if you can take a look at those.

Regards,

Vladimir

[PhoneBoy]

The attachments you sent (assuming via email) did not come across.

You will need to come to the community website to attach those.

You may also wish to open a support ticket with TAC.

[Vladimir]

Weird, I have uploaded those from the community, can see them in the post above and was able to download and open all three files:

As to TAC, they are not keen on supporting lab environments, hence looking for suggestions in community.

[PhoneBoy]

Issue is on my end

Apparently they don't show in the "inbox" view on the community, which is...problematic. 

Looking at the logs you provided, it looks like CPLogToSyslog is sending the logs over TCP

Your syslog server is listening on UDP.

Looking at your configuration file, it seems you've configured it for UDP.

Clearly this setting is getting ignored.

You have two options:

1. Open a TAC case to investigate (it's a bug, whether lab or not is irrelevant)

2. Configure your syslog server to accept connections over TCP

Unrelated question: if this is lab, why are you using R80 instead of R80.10?

[Vladimir]

I think there are more issues with communities: I have replied to your last post via email, but it did not show-up in a thread (about 4 hours ago).

TAC is not keen on investigating lab reports of observed issues. I have forwarded my test results to four people at CP, but did not hear from them in a while.

I can use TCP in my lab, but am working on replicating client's environment, hence R80.

Thank you for help with the sanity check!

[PhoneBoy]

Can you forward the email you tried to send to checkmates AT checkpoint DOT com?

I’ll have someone take a look at it.

[Vladimir]

Apologies, false alarm: I've found it in my drafts.

[Hugo_vd_Kooij]

I can confirm that R80.10 works in the lab.

Now I need to do something intelligent with the entries in SPLUNK 😉

[Kristof_Vermael]

I'm trying to get this working, but the only logs I'm getting are anti-malware events, no drops nor accepts.

Can you share your config file ?

[ANTONIO_OPROMO1]

Hi Kristof,

I've also made in my lab some tests with management version R80 and gateway version R77.30 as in your lab, and also with the latest version R80.10 for the management and gateway.

In both cases the logs are exported to my syslog server (in my case as syslog server I'm using the Extreme Management Server for made a distributed IPS system because Extreme Networks has developed a module for this purpose), but you first need to force Check Point to export the logs via UDP instead TCP (by default use TCP, so follow the Check Point solution sk109016 for use UDP instead that I attach to) and then define the filter rules for the log events you want to export to the external syslog server in the local.cplogtosyslog_policy.C file.

The log structure is in some parts changed between version R80 and version R80.10 of Check Point management (for example in R80 the action was no a field mapped, instead it is in R80.10 version).

I attach my local.cplogtosyslog_policy.C file that I use in my lab for version R80.10 and the file that I use in version R80 as your lab is (remove the _r80.10 or _r80 from the file name if you want to use these files and change the ip address as in your environment).

[Bonnie_Self]

Does anyone know if this tool works on a log server running NGSE?

[PhoneBoy]

I recommend contacting the TAC to see if they can provide a version of this for NGSE: Contact Support | Check Point Software 

[Vladimir]

BTW, if you need the ability to differentiate gateways by name in the forwarded logs, (i.e. if you are shipping logs to external SIEM provider to be parsed):

Below are the steps to have the syslog contain the origin_sic_name field.

Perform these steps on the Log server that has CPLogToSyslog installed:

  1. Stop services on the Log Server.

  [Expert@HostName]# cpstop

2. Backup and modify $FWDIR/conf/log_fields.C

  [Expert@HostName]# cp -pv $FWDIR/conf/log_fields.C

$FWDIR/conf/log_fields.C_ORIGINAL

  [Expert@HostName]# vi $FWDIR/conf/log_fields.C

3. Search for "origin_sic_name" and under ":display_mode" there will be a

field, ":application_name (FWLog)". Change from:

: (

  :AdminInfo (

    :chkpf_uid ("{5DF46778-79F6-487B-AF90-8CE40333E117}")

    :ClassName (application_display_mode_object)

  )

  :application_display_mode (none)

  :application_name (FWLog)

)

TO:

: (

  :AdminInfo (

    :chkpf_uid ("{5DF46778-79F6-487B-AF90-8CE40333E117}")

    :ClassName (application_display_mode_object)

  )

  :application_display_mode (own_column)

  :application_name (FWLog)

)

4. Start services.

  [Expert@HostName]# cpstart

The logs will now contain the origin_sic_name field:

Nov 9 2017 14:39:50
Nov 9 2017 19:39:50 GMT
Thu Nov 9 14:39:50 Log host CPLogToSyslog: 0 16386 encrypt 52.184.158.74 >eth1 LogId: <max_null>; ContextNum: <max_null>; OriginSicName: <max_null>; log_sequence_num: 0; is_first_for_luuid: 131072; inzone: Internal; outzone: External; rule: 90; rule_uid: {AE182161-16C0-4367-A732-B036B35935E9}; rule_name: Internet access; service_id: domain-udp; src: 10.aaa.aaa.aaa; dst: 10.bbb.bbb.bbb; proto: 17; scheme: IKE; methods: ESP: AES-128 + SHA1; peer gateway: 62.aaa.bbb.ccc; community: Onprem-AzureCloud; fw_subproduct: VPN-1; vpn_feature_name: VPN; origin_sic_name: CN=gatewayname,O=managementserver.domain.com.d4w394; aba_customer: SMC User; date: 9Nov2017; hour: 14:29:57; type: log; Interface: < eth1; ProductName: VPN-1 & FireWall-1; svc: 53; sport_svc: 9679;

[Victor_Chang]

just adding to the general discussion, we tested the CPLogToSyslog function over TCP and UDP.

TCP seems to die after some time whereas UDP seems fine. Its possible that our parser on our SIEM was slowing things down and causing backlog that ultimately kills CPLogToSyslog. Still looking into this issue

Shoutout to Dameon Welch Abernathy‌

[PhoneBoy]

TCP is stateful, UDP is stateless.

Curious if you're losing logs on your SIEM as a result.

[Victor_Chang]

I suspect we were because the parser was inefficient. I'm guessing that was causing CPLogToSyslog to die when on TCP. We'll retry TCP and update everyone here

Also in case anyone is wondering there is a CPLogToSyslog HF equivalent for R77.30 HFA 286.

[Tom_Cripps]

I'm having issues in getting my utility to run over UDP, as it's using TCP. I've refered to sk109016 but we're still seeing it running over TCP then our utility stops. 

[Vladimir]

Have you rebooted your management server after implementing changes described in the sk?

[Tom_Cripps]

I have yes, rebooted it last night and checking the connection using netstat it's still using TCP. I thought that sk doesn't apply to R80.10? It doesn't say so.

[Vladimir]

Please paste the output of "netstat -anp | grep -E "PID|CPLogToSyslog" in reply.

[Tom_Cripps]

Here you go.

Tom

[Vladimir]

please let me know what is the host 172.20.1.9.

Additionally, please paste the pertinent section of the $FWDIR/state/SEAM/local.cplogtosyslog_policy.C file.

I am not sure if you've had a chance to read this document that I have posted few months ago:

https://community.checkpoint.com/docs/DOC-2205-cplog-to-syslog-for-r80-managed-environments 

But it may make sense to go over it to confirm that all is configured appropriately.

[Tom_Cripps]

172.20.1.9 is the IP of our Management server.  And is this the right part you want out of the configuration file?