Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Dan_Zada
Employee Alumnus
Employee Alumnus

Log Exporter Filtering

Hello all,

I'm happy to inform you that we added a new feature to the log exporter - the ability to filter logs.

Starting today, you will be able to configure which logs will exported, based on fields and values, including complex statements.

More information, including basic and advanced filtering instructions, can be found in SK122323.

If you have any question or comment, let me know.

Thanks!

Dan.

72 Replies
Danny
Champion Champion
Champion

Great innovation!

0 Kudos
Maarten_Sjouw
Champion
Champion

Very good addition, many people have asked for it.

Is there anywhere a list of the field names can be found with possibly an explanation?

Regards, Maarten
Dan_Zada
Employee Alumnus
Employee Alumnus

We are working on one.

Stay tuned Smiley Happy

Tom_Cripps
Advisor

Hi Dan,

I'm trying to filter on Source IP, can the filter be of a network group at all? or even subnets?

 

Thanks,

Tom

Jerry
Mentor
Mentor

oh yes indeed great stuff! Thanks guys 🙂 well done!
Jerry
Dan_Zada
Employee Alumnus
Employee Alumnus

Hi @Maarten_Sjouw,
I'm happy to share with you (and with everyone else) that we released the full mapping for Check Point logs fields:
https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...
Maarten_Sjouw
Champion
Champion

Great Dan, that really helps to sort out what we can do with it.
Regards, Maarten
0 Kudos
Peter_Baumann
Contributor

Hi all,

A customer has tried to do a filtering for the origin so that he only get the logs sent from one firewall in the config.
It was not possible to use the following:

[Expert@mgmt1:0]# cp_log_export set name Filtered filter-origin-in "fw-test"
Error: Argument [filter-origin-in] is undefined for command: [set]

[Expert@mgmt1:0]# cp_log_export add name splunk domain-server fwadm.local filter-origin-in fw-test
Error: Argument [filter-origin-in] is undefined for command: [add]

[Expert@mgmt1:0]# cp_log_export set name splunk domain-server fwadm.local filter-origin-in fw-test
Error: Argument [filter-origin-in] is undefined for command: [set]

The documentation is very confusing without some adequate examples.

Does someone have examples for origin and/or action filtering?

Thanks

Garrett_DirSec
Advisor

would be nice to understand what is used to QA the "filtering" features.  at min, this would provide good examples for use.

Also, since sk122323 references "newly added feature" of filtering, this implies it requires a specific build of Log Exporter.  I don't see any references to build numbers and/or whether the "filter" feature will be added as part of HFA, etc. 

In other words:   will the "filter" feature Log Exporter require a patch even for platforms like R80.20 (where Log Exporter is native)?

 

PhoneBoy
Admin
Admin

Where Log Exporter is integrated (R80.20), the logical method for updating it is via the Jumbo Hotfix mechanism.
And, as it turns out, there's a note in the Log Exporter SK that says this is NOT included in R80.20 as of yet...
Garrett_DirSec
Advisor

thanks D. appreciate it.
Dror_Aharony
Employee Alumnus
Employee Alumnus

Hi Peter,

Your command is absolutely correct. It should work (as the examples are accurate).

Which CP-Version & Exporter-HF version are you using (if any)?

Keep in mind, that the new filtering options are currently only available for R77.30 & R80.10 (with the latest exporter-HF), so I'd guess you simply don't have the filtering feature on your server yet, that is probably the source of confusion here.

It should be available on R80.20/.30 JHFs pretty soon...

 

Peter_Baumann
Contributor

Hi Dror,

Thanks for the update.
The customer is using R80.20 Take 47.
It seems there's no version visible in cp_log_export, therefore here's the MD5 checksum:

11f4776c10b7b02d67ba72ee5cd00953 /opt/CPrt-R80.20/bin/cp_log_export

Dror_Aharony
Employee Alumnus
Employee Alumnus

Hi Peter,

as I suspected, the entire filtering feature for log-exporter isn't available in R80.20 yet. 

Coming soon...

 

0 Kudos
Garrett_DirSec
Advisor

hello @Dror_Aharony 

thanks for the updates and insight on JHA requirements for LogExporter filtering for R80.20/.30.     I would expect to find that information in SK122323.

The documentation enhancement request is valid to have examples inserted into filter section of SK122323.    I suggest not only what works, but more importantly, what doesn't work. 

Example for cp_log_export usages:

cp_log_export set name <name> filter-action-in "value1,value2"
cp_log_export set name <name> filter-origin-in "value1,value2"
cp_log_export set name <name> filter-blade-in "value2"

* The name of the field to filter on should be the mapped name in case it is changed in the mapping XML, or the original raw name if it is not mapped.

* The value of the field to filter on should be the raw field values.

Dror_Aharony
Employee Alumnus
Employee Alumnus

Hi Garrett,

We've added a few examples to the sk (SK122323) of using the log-exporter with the new filtering feature.

Hope it helps a bit.

Garrett_DirSec
Advisor

@Dror_Aharony thank you for the update!!   much appreciated.   -GA

Svetlana_Klasni
Explorer

Hi Dan,

I tried to filter logs on CP SMS R80.20, Gaia Kernel Version: 3.10, Jumbo Hotfix General availability (Take 87), but without success:

 cp_log_export set name RSYSLOG-TCP filter-action-in "Drop"
Error: Argument [filter-action-in] is undefined for command: [set]

The sending logs to our RSYSLOG server was configured earlier:

[Expert@CASCPSMS:0]# cp_log_export show

name: RSYSLOG-TCP
enabled: true
target-server: 10.0.9.99
target-port: 5519
protocol: tcp
format: syslog
read-mode: raw

Why I do not have  use filter-action-in argument for cp_log_export command?

Best regards,
                 Svetlana
0 Kudos
Maarten_Sjouw
Champion
Champion

Svetlana from SK 122323:
R80.20
Log Exporter is already integrated in R80.20. There is no need to install in it a dedicated package.

Note: Filtering ability is not integrated to R80.20 and R80.30 yet, this SK will be updated when it will be supported.
Regards, Maarten
genisis__
Leader Leader
Leader

Have a question.

We know that log exporter can be used to export logs to an external logging server, however what I would like to do is export IPS and Threat Prevention logs from originating from specific GWs or CMA.

 

Example:

Scenario I:

SmartEvent is used in our environment which clearly is already consolidating entries; however this is consolidating entries for every client our Provider.  We need to export a specific customers entries to any external collector such as ArcSight.

Is there a way to export logs from a specific CMA or GWs from here?

 

Scenario II:

From the CMA how can we extract IPS/Threat Prevention logs to an external logging server (ArcSight I believe).  Ideally logs that originate from specific GWs.

 

The SK122323  above does not really show me any examples that make sense to me.  We are running R80.20 with HFA103 as well.

0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

the log-exporter filtering ability allows you to do just that:

You can activate/use an exporter that runs on that specific CMA (as the Log-Server), that will only export logs received by it - that CMA.

 

to export logs from a specific GW only, simply use the origin field, as in this example:

cp_log_export set name <name> filter-origin-in "GW_name"
 

Good Luck!

0 Kudos
Maarten_Sjouw
Champion
Champion

@Dror_Aharony can this also be used for a group of gateways?
Regards, Maarten
0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

that depends, who's the origin GW that appears in the logs?

if they are separate, which I believe they are, then you need to write all of them, as in the sk's example ("GW1,GW2,GW3")

cp_log_export set name <name> filter-origin-in "GW1,GW2,GW3"

 

You can verify in the FilterConfiguration.xml that there's an OR in that field (origin) with all your matching values. Should look like that:

                <field name="origin" operator="or">
                        <value operation="eq">GW1</value>
                        <value operation="eq">GW2</value>
<value operation="eq">GW3</value> </field>

 

 

0 Kudos
Maarten_Sjouw
Champion
Champion

Ok, well that will be a bit of a hassle when you add multiple gateways every week, While a group of origin gateways would just be updated when the gateway is added in the SmartConsole.
Regards, Maarten
0 Kudos
genisis__
Leader Leader
Leader

In our case we only have a certainly set of GWs we require the logs from, however we only want IPS/TP logs be to exported.

<field name="origin" operator="or">
             <value operation="eq">GW1</value>
             <value operation="eq">GW2</value>

             <value operation="eq">GW3</value>

</field>

So the above is great! just need to expand on this to determine only IPS/TP logs from the above?

0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

Yea, simply add the IPS/TP as in the sk examples, like this for both TP & Multi-GWs.

cp_log_export set name <name> filter-blade-in "TP" filter-origin-in "GW1,GW2,GW3"

or

   cp_log_export set name <name> filter-blade-in "SmartDefense" filter-origin-in "GW1,GW2,GW3"

 change/combine them both if needed. 

0 Kudos
genisis__
Leader Leader
Leader

Will this same process work on a SmartEvent appliance?

ie. origin GWs only would be required as the SmartEvent should only be storing consolidated events.

 

 
0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

exactly the same for using log-exporter on a SmartEvent too, for any exporter.

0 Kudos
Maarten_Sjouw
Champion
Champion

@Dror_Aharony, @Dan_Zada any idea when the filtering will also be available on the R80.30 release?
Regards, Maarten

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events