Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Obiwan1968
Contributor

FilterConfiguration.xml gets overwritten at restart of cp_log_export

Hi There

Using R80.30 Take 191
I have seen the example in sk122323 about the FilterConfiguration.xml.

Using the example below by editing $EXPORTERDIR/targets/<target-name>/conf/FilterConfiguration.xml

<filters>
        <filterGroup operator="and">
                <field name="action" operator="and">
                </field>
                <field name="origin" operator="and">
                </field>
                <field name="product" operator="or">
                        <value operation="eq">SmartDefense</value>
                        <value operation="eq">Threat Emulation</value>
                </field>
                <field name="severity" operator="or">
                        <value operation="eq">3</value>
                        <value operation="eq">4</value>
                </field>
        </filterGroup>
</filters>

But when I restart the cp_log_export service, all changes are gone and the file "FilterConfiguration.xml" is "empty" again as before? Anymore any idea why this file gets "reset" be the restart of the service?

0 Kudos
26 Replies
Dror_Aharony
Employee Alumnus
Employee Alumnus

It means the file wasn't legal, therefore getting overwritten with the default file on restart.

Try quoting the Threat Emulation: "Threat Emulation".

*Copy/Save text of file in any-case before restarting for easier repeat editing attempts.

Let us know if it works.

 

Obiwan1968
Contributor

Is a "cp_log_export reconf" necessary or only doing a "cp_log_export restart"?

With "reconf" the file gets default again, with "restart" the FilterConfiguration.xml keeps its settings, but they do not work, nothing appears. I tried with 


<filterGroup operator="and">
<field name="action" operator="and">
</field>
<field name="origin" operator="and">
</field>
<field name="product" operator="and">
<value operation="eq">IPS</value>
</field>
</filterGroup>
</filters>

Without the IPS line I get logs with product IPS/Firewall etc, with the line nothing at all

Dror_Aharony
Employee Alumnus
Employee Alumnus

no, no. only restart

reconf is only meant to run once post upgrade to update the log-exporters to updated version (including base files)

Do not repeat it afterwards, or it will override your files, as you've already seen.

Great, then file is good:)

 

Not IPS, as it uses the raw/older name (SmartDefense).
see Log Fields Description sk144192 > https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...

IPS=SmartDefense.

and afterwards add "Threat Emulation" if you like.

You can also use the suggested Threat group (filter-blade-in "TP") for all threat logs.

see log-exporter sk122323 for more examples > https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...

 

Hope that solves it.

 

 

 

Obiwan1968
Contributor

Now it looks good, I get traffic this way

0 Kudos
Obiwan1968
Contributor

Can I do something like this? Does not work. The idea is having the SmartDefense with certain severity and confidance AND the AUDIT logs

When I kill the audit log, the SmartDefense in itselfe works, but I am not sure if I can use cases this way? or what the right way is?

 

<filters>
        <filterGroup operator="or">
                <filterGroup operator="and">
                        <field name="product" operator="or">
                                <value operation="eq">SmartDefense</value>
                                <value operation="eq">TP</value>
                        </field>
                        <field name="severity" operator="or">
                                <value operation="eq">2</value>
                                <value operation="eq">3</value>
                                <value operation="eq">4</value>
                        </field>
                        <field name="confidence_level" operator="or">
                                <value operation="eq">3</value>
                                <value operation="eq">4</value>
                                <value operation="eq">5</value>
                        </field>
                </filterGroup>
                <filterGroup operator="and">
                        <field name="product" operator="or">
                                <value operation="eq">SmartConsole</value>
                        </field>
                </filterGroup>
        </filterGroup>
</filters>
[

 

 

Dror_Aharony
Employee Alumnus
Employee Alumnus

There is no value TP, it's a set group filter for the filter-blade-in that translates to all Threat blades:
SmartDefense, "Threat Emulation" & more...

 

Sorry, I think you did the Audit, right.

Let's see & I'm also checking here.

Obiwan1968
Contributor

I am not sure is this structure with "filterGroup" is correct? Is there any big example how to do this correctly?

0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

I understand what you mean, but you configured it properly.  I verified it.

 

Top filterGroup "or" between the groups of FW-filtered & Audit.

Another filterGroup "and" between the fields (IPS, Severity,Confidence).

Another filterGroup "and" between the Audit field (even if it's just one).

filter of fields "or" between actual values in the same field (even if it's just one).

 

Are you sure you don't see any Audit logs being exported?

 

Anybody, @PhoneBoy  @Shay_Hibah?

 

 

Obiwan1968
Contributor

Ok I delete the "TP" line after "SmartDefense"

When I run this, I suddenly get plenty of log entries from the "product=Firewall" which I do not want, no idea why? I just need the SmartDefense and the Audit Logs, but no firewall logs

<filters>
        <filterGroup operator="or">
                <filterGroup operator="and">
                        <field name="product" operator="or">
                                <value operation="eq">SmartDefense</value>
                        </field>
                        <field name="severity" operator="or">
                                <value operation="eq">2</value>
                                <value operation="eq">3</value>
                                <value operation="eq">4</value>
                        </field>
                        <field name="confidence_level" operator="or">
                                <value operation="eq">3</value>
                                <value operation="eq">4</value>
                                <value operation="eq">5</value>
                        </field>
                </filterGroup>
                <filterGroup operator="and">
                        <field name="product" operator="or">
                                <value operation="eq">SmartConsole</value>
                        </field>
                </filterGroup>
        </filterGroup>
</filters>

 

 

0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

These may be IPS as FW logs, we have a few of these.

 

You can use the additional filter-out of FW connections.

Change to true in targetConfiguration.xml.

<filter filter_out_by_connection="true">

 

Any updates on the Audit logs?

 

 

0 Kudos
Obiwan1968
Contributor

With this minimal configuration it works, but here I have also "informal","low" and "low-medium" SmartDefense" events listed

<filters>
        <filterGroup operator="and">
                <field name="product" operator="or">
                        <value operation="eq">SmartConsole</value>
                        <value operation="eq">SmartDefense</value>
                </field>
        </filterGroup>
</filters>

 

 

 

0 Kudos
Shay_Hibah
Employee Alumnus
Employee Alumnus

Hey @Obiwan1968 

How are you?

 

Filter configuration supports (for now) only one filterGroup.

This file should be saved during restart and no additional command is needed to keep it with the new configuration.

Can you please let me know what exactly you wish to achieve and I will try to help?

 

Thanks,

Shay

0 Kudos
Obiwan1968
Contributor

Hi Shay

I wanna do two things:

1) all audit logs
2) all attacks with severity (2/3/4) and confidence_level (3/4/5)

 

0 Kudos
Obiwan1968
Contributor

Just comes to my mind, I could of course create two instances of cp_log_export services instead of putting all the settings into one?

0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

Exactly:)

0 Kudos
Paul_Hagyard
Advisor

Hi,

While I love the fact that it can be made to work with two instances, this presumably means there are now two processes parsing all of the log messages, rather than one with a more complex filter. I'm unsure of the performance implications of the two approaches, but I would have thought one process could be more efficient in terms of IO and performance.

It's a couple of years since this post, has there been any work to support more complex filters, through multiple filterGroup entries or some other mechanism?

Cheers,

Paul

0 Kudos
PhoneBoy
Admin
Admin

As far as I know, this hasn't changed and would require an RFE.
Having said that, running multiple instances is expected in some configurations independent of this use case.
At least on the community side, I haven't seen any specific complaints related to this.

0 Kudos
Shay_Hibah
Employee Alumnus
Employee Alumnus

You are right.
In case you want to export all audit logs and in addition, export all security logs with severity 2+ and confidence 3+, it is possible to create 2 exporters:
1. one of the exporters will export only audit logs (can be configured in targetConfiguration.xml file).
2. The second exporter will export all security logs with filtering configuration as mentioned above.

Do you need any assistance to do this?
0 Kudos
Obiwan1968
Contributor

No, I am fine and everything works now as expected 🙂

Thanks for your support!

0 Kudos
Cesar_Almada
Explorer

hello

could you share the xml, please

i need the same filter

thanks

Cesar A.

0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

One log-exporter filter for the IPS severity/confidence logs, like this:

<filterGroup operator="and">
                        <field name="product" operator="or">
                                <value operation="eq">SmartDefense</value>
                        </field>
                        <field name="severity" operator="or">
                                <value operation="eq">2</value>
                                <value operation="eq">3</value>
                                <value operation="eq">4</value>
                        </field>
                        <field name="confidence_level" operator="or">
                                <value operation="eq">3</value>
                                <value operation="eq">4</value>
                                <value operation="eq">5</value>
                        </field>
                </filterGroup>

Add Filters to start & to close as well.

<filters>
/filters>

 

2nd for audit only:

Configure your targetConfiguration.xml to audit only, without any filters, like this:
<log_types>audit</log_types><!--all[default]|log|audit/-->

 

Hope that's clear enough (as I used the current pictures).

omidrajaee
Explorer

Hello, Can you please let me know how did you fix the problem? I used the same template as yours but I am not getting everything!

0 Kudos
omidrajaee
Explorer

Hi, 

Only Smart Defense and Threat Emulation are enough how I can find out the other services or running on my CP? DO I need to use Confidence level this is what I've used along with severity but I missed all unknow logs I had before xml editing:

                <field name="product" operator="or">

                        <value operation="eq">SmartDefense</value>

                        </field>

                        <field name="severity" operator="or">

                                <value operation="eq">2</value>

                                <value operation="eq">3</value>

                                <value operation="eq">4</value>

                        </field>

                        <field name="confidence_level" operator="or">

                                <value operation="eq">3</value>

                                <value operation="eq">4</value>

                                <value operation="eq">5</value>

                </field>

 

0 Kudos
PhoneBoy
Admin
Admin

From the gateway itself? enabled_blades
From the management server, refer to the relevant gateway object.

0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

If I understood you correctly, you wanted the confidence_level unknown to be exported as well?
Add a line for 0 (=Unknown) as well.

<field name="confidence_level" operator="or">

                                <value operation="eq">0</value>

0 Kudos
omidrajaee
Explorer

Hello, 

It did not help! also I am only getting the SmartDefense event nothing else. Do you know how I can get Firewall, VPN, URL Filtering and etc? Before filtered out I got some unknown severity logs and they are very useful but now I don't have them anymore.

0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events