- CheckMates
- :
- Products
- :
- Quantum
- :
- Management
- :
- Re: Log Exporter guide
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Are you a member of CheckMates?
×- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Log Exporter guide
Hello All,
We have recently released the Log Exporter solution.
A few posts have already gone up and the full documentation can be found at sk122323.
However, I've received a few questions both on and offline and decided to create a sort of log exporter guide.
But before I begin I’d like to point out that I’m not a Checkpoint spokesperson, nor is this an official checkpoint thread.
I was part of the Log Exporter team and am creating this post as a public service.
I’ll try to only focus on the current release, and please remember anything I might say regarding future releases is not binding or guaranteed.
Partly because I’m not the one who makes those decisions, and partly because priorities will shift based on customer feedback, resource limitations and a dozen other factors. The current plans and the current roadmap is likely to drastically change over time.
And just for the fun of it, I’ll mostly use the question-answer format in this post (simply because I like it and it’s convenient).
Log Exporter – what is it?
Performance
Filters
Filters: Example 1
Filters: Example 2
Gosh darn it, I forgot something! (I'll edit and fill this in later)
Feature request
- Labels:
-
Documentation
-
Integrations
-
Logging
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Su,
From your log (Current=13 Avg=139 MinAvg=10 Total=331020 ) as well as the status command it appears that logs are being exported.
If you want to actually see this you can use tcpdump command: 'tcpdump port 514 -A -s0' (if you are using port 514 for anything else, you can add other qualifiers to narrow down the output).
This will show you the actual data being exported in a readable format. For example:
[Expert@ypsa:0]# tcpdump port 514 -A -s0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
14:25:25.094828 IP ypsa.47206 > XX.XX.XX.XX.syslog: SYSLOG local0.info, length: 1044
E..0..@.@..xd P...f......<134>1 2018-08-14T14:25:23Z ypsa CheckPoint 17857 - [action:"Accept"; ifdir:"inbound"; ifname:"eth0"; [deleted the payload] product:"VPN-1 & FireWall-1"; proto:"6"; s_port:"35700"; service:"22"; service_id:"ssh"; src:"XX.XX.XX.XX"; ]1 packets captured
2 packets received by filter
0 packets dropped by kernel
[Expert@ypsa:0]#
(I deleted most of the payload since it just takes up space and not really relevant for this example - I just wanted to show that you can see and read the actual logs as they are being exported)
Since it looks like your logs are actually being exported, I would focus on the other end and try to see if it's being received and parsed correctly.
Use tcpdump or Wireshark on the other end. If it's not there, it's a connectivity issue, and if it's there it's probably a parsing issue.
HTH
Yonatan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey Yonatan,
Glad I found this post
Please let me know where it stands for supporting JSON format output.
My X-Pack charged ELK is waiting to search thru this data and I am currently working on parsing it myself which is not an easy task so far.
I believe JSON format will be friendly to work with in python scripts and rest API as well
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HI Bogdan,
I don't think that official json output will be added anytime soon - there are currently many other items ahead of it in the queue.
However, earlier in this thread, I showed how you can edit the settings to generate json like output.
It does have the disadvantage of potentially having duplicate keys, but a fast google search returned some answers on how to deal with that:
Elasticsearch, Kibana and duplicate keys in JSON -
Please let me know if this worked.
Yonatan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Product (Blade) names.
As you might have noticed the names that are exported are not the same as the ones which appear in the GUI.
This is because each blade has a display name which is shown in the GUI and an actual value which appears in the raw log. Those names are often the same but not always.
For technical reasons, it's very difficult to change the actual value that the gateway sends but easy to change the display name. So while the actual value and the display name usually start the same, the values might drift over time.
This has caused a bit of confusion among some customers.
So as a sort of public service here is the current mapping of raw log values to display name.
This list also includes some legacy names. Frankly, there were a few names on this that I've never heard of and didn't know existed
Field Name | Anti Malware | Core | Identity Awareness | SmartView MonitorMonitor |
Display Name | Anti-Bot | Core | Identity Awareness | SmartView Monitor |
Field Name | Anti-Malware | DefensePro | Identity Logging | Syslog |
Display Name | Anti-Malware | DDoS Protector | Identity Logging | Syslog |
Field Name | Anti-Exploit | DLP | Management Blade | System Monitor |
Display Name | Anti-Exploit | DLP | Management Blade | System Monitor |
Field Name | Anti-Ransomware | Content Awareness | MEPP | Threat Emulation |
Display Name | Anti-Ransomware | Content Awareness | Media Encryption & Port Protection | Threat Emulation |
Field Name | WIFI Network | Edge AV | Connectra | Threat Extraction |
Display Name | WIFI Network | Edge AV | Mobile Access | Threat Extraction |
Field Name | Mobile App | Compliance | Policy Server | Anti Virus |
Display Name | Mobile App | Endpoint Compliance | Policy Server | Traditional Anti-Virus |
Field Name | Network Security | Integrity | Web Filtering | UAG |
Display Name | Network Security | EndpointEndpoint Security | Legacy URL Filtering | UA Server |
Field Name | OS Exploit | Everest | CVPN | WebAccess |
Display Name | OS Exploit | FireWall-1 GX | CVPN | UA WebAccess |
Field Name | Device | Firewall | FG | URL Filtering |
Display Name | Device | Firewall | QoS | URL Filtering |
Field Name | Text Message | VPN-1 & FireWall-1 | rtm | VPN-1 Edge |
Display Name | Text Message | Security Gateway/Management | Real Time Monitor | UTM-1 Edge |
Field Name | iOS Profiles | Forensics | SecureClient | VPN |
Display Name | iOS Profiles | Forensics | SecureClient | VPN |
Field Name | Cellular Network | FDE | Server | VPN-1 |
Display Name | Cellular Network | Full Disk Encryption | Server | VPN |
Field Name | Anti Spam | Capsule Docs | SmartConsole | VPN-1 Embedded Connector |
Display Name | Anti-Spam and Email Security | Capsule Docs | SmartConsole | VPN Embedded Connector |
Field Name | New Anti Virus | HTTPS Inspection | Eventia Analyzer Client | WebCheck |
Display Name | Anti-Virus | HTTPS Inspection | SmartEvent Client | WebCheck |
Field Name | Application Control | SmartDefense | SmartEvent | Zero Phishing |
Display Name | Application Control | IPS Software Blade | Eventia Analyzer | Zero Phishing |
Field Name | Compliance Blade | IPS-1 | SmartView | MTA |
Display Name | Compliance Blade | IPS-1 Sensor | SmartView | MTA |
HTH
Yonatan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It's scary that I recognize almost all these names.
The only one I didn't know was Everest.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think that was the code name for Connectra once
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yonatan, the top row has become a "header" row in the table you have posted.
Had a double-take reading offset entries under it for about 10 lines before it clicked
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Partially fixed. I removed the bold, but it won't let me remove the bottom 'header' border for some reason.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You probably need to edit the raw HTML (which you can do).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks - That seems to have done the trick.
I hate going into the HTML source code. I get flashbacks from trying to manually fix SKs.
The horror!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Yonatan,
Thank you for the detailed write-up, I found it very helpful!
Is there a way to rate limit the amount logs that are exported? We ran into an issue where our log exporter process was overloading our syslog server with requests.
Thanks!
Matt
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Matthew,
At this time there is no way to limit the number of logs sent.
This will probably be possible in the future once we implement advanced filter capabilities.
Improving the filters is an item on our roadmap but I don't know when this will be implemented.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi!
Nice writeup and nice tool!
I have small question
I need export only SmartEvent events width EN***** id in message body
Can i do it with LogExporter? Can u help me with configuration?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Alexandr,
This can be done to a limited degree, probably not good enough for your use case.
This will most likely be possible once we implement advanced filtering options - unfortunately, I don't know exactly when that will actually happen (it's on the roadmap but we are currently working on another feature ahead of filters).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Maybe i can use log exporter tool in custom script for sending messages?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I'm not sure I understand the question. I suspect the answer is no. The Log Exporter uses the indexing infrastructure (that Checkpoint log servers use). It reads *.fw log files, but instead of 'indexing' them it sends the logs to the interface send queue.
I don't see how you can insert scripts into this chain, nor can this run on a server without the indexing infrastructure (e.g. log servers)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Jonathan,
Something that is not completely clear to me, when we want to use TLS and we have a official cert at the other end, do I still need to created local files to allow this to be used? Or can I just tell log exporter to use TLS?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The Log Exporter uses mutual authentication - both sides need to authenticate each other.
When we were looking at the TLS implementation of other vendors we noticed that some of them use single sided authentication, but after considering the issue we decided to err on the side of more security and implemented mutual authentication.
Hope that answers your question. it's always a bit difficult for me to address TLS related questions as that's not my area of expertise, and while I was involved in the TLS discussions during the implementations, I was mostly on the sidelines of those discussions and left it to the relevant experts to do the heavy lifting.
HTH
Yonatan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Log_export can export all software blade log to external syslog server ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes.
The Log Exporter can export everything in the fw.log file regardless of the content.
It basically treats everything in the log payload as an alphanumeric string.
All the adaptations, mappings, filters, etc. are all based on string/text manipulation regardless of the content ("blade").
HTH
Yonatan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Yonatan, I just configured log exporter so send logs via syslog to SIEM server, however when log sent I don't see protocol field in the log, when sent vi LEA saw for example - protocol=UDP etc. but now I get only proto=6, proto=17 etc. How do I convert it to protocol name? Is there any proto(number) to protocol(actual protocol name) mapping exist?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Also the /etc/protocols file on Gaia/Linux.
--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com
Exclusively at CPX 2025 Las Vegas Tuesday Feb 25th @ 1:00pm
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
For those who missed it, Dan has officially announced the new Log Exporter update with better Splunk integration.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Will it work on Standalone machine?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have it running on a standalone logserver.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is it possible to run the LogExporter only on a Log Correlation Unit?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can run the Log Exporter on any server where you can enable the Logging blade (Management, Log server, SmartEvent, etc.).
The Log Exporter uses the Indexing infrastructure so that infrastructure has to be installed on the server. (it doesn't have to actually be active, just needs to be installed. So even if you aren't actively using the server as an Indexer, as long as you have the option to enable the blade that's good enough).
HTH
Yonatan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, I'd need to add a string at the beginning of the exported logs, is it possible?
Not working on RSA NetWitness / Security Analytics:
"CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Log|domain-udp|Unknown|act=Accept destinationTranslatedAddress=0.0.0.0 destinationTranslatedPort=0 deviceDirection=0 .....
Working fine on RSA NetWitness / Security Analytics:
"<1> CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Log|domain-udp|Unknown|act=Accept destinationTranslatedAddress=0.0.0.0 destinationTranslatedPort=0 deviceDirection=0 .....
the addition of the string "<1>" at the beginning of the exported log is needed in order to have the exported log correctly ingested and parsed in the RSA SIEM.
Many thanks
kind regards
Luca
