- Local User Groups
I'm happy to inform you that we added a new feature to the log exporter - the ability to filter logs.
Starting today, you will be able to configure which logs will exported, based on fields and values, including complex statements.
More information, including basic and advanced filtering instructions, can be found in SK122323.
If you have any question or comment, let me know.
Very good addition, many people have asked for it.
Is there anywhere a list of the field names can be found with possibly an explanation?
A customer has tried to do a filtering for the origin so that he only get the logs sent from one firewall in the config.
It was not possible to use the following:
[Expert@mgmt1:0]# cp_log_export set name Filtered filter-origin-in "fw-test" Error: Argument [filter-origin-in] is undefined for command: [set] [Expert@mgmt1:0]# cp_log_export add name splunk domain-server fwadm.local filter-origin-in fw-test Error: Argument [filter-origin-in] is undefined for command: [add] [Expert@mgmt1:0]# cp_log_export set name splunk domain-server fwadm.local filter-origin-in fw-test Error: Argument [filter-origin-in] is undefined for command: [set]
The documentation is very confusing without some adequate examples.
Does someone have examples for origin and/or action filtering?
would be nice to understand what is used to QA the "filtering" features. at min, this would provide good examples for use.
Also, since sk122323 references "newly added feature" of filtering, this implies it requires a specific build of Log Exporter. I don't see any references to build numbers and/or whether the "filter" feature will be added as part of HFA, etc.
In other words: will the "filter" feature Log Exporter require a patch even for platforms like R80.20 (where Log Exporter is native)?
Your command is absolutely correct. It should work (as the examples are accurate).
Which CP-Version & Exporter-HF version are you using (if any)?
Keep in mind, that the new filtering options are currently only available for R77.30 & R80.10 (with the latest exporter-HF), so I'd guess you simply don't have the filtering feature on your server yet, that is probably the source of confusion here.
It should be available on R80.20/.30 JHFs pretty soon...
Thanks for the update.
The customer is using R80.20 Take 47.
It seems there's no version visible in cp_log_export, therefore here's the MD5 checksum:
thanks for the updates and insight on JHA requirements for LogExporter filtering for R80.20/.30. I would expect to find that information in SK122323.
The documentation enhancement request is valid to have examples inserted into filter section of SK122323. I suggest not only what works, but more importantly, what doesn't work.
Example for cp_log_export usages:
cp_log_export set name <name> filter-action-in "value1,value2"
cp_log_export set name <name> filter-origin-in "value1,value2"
cp_log_export set name <name> filter-blade-in "value2"
* The name of the field to filter on should be the mapped name in case it is changed in the mapping XML, or the original raw name if it is not mapped.
* The value of the field to filter on should be the raw field values.
I tried to filter logs on CP SMS R80.20, Gaia Kernel Version: 3.10, Jumbo Hotfix General availability (Take 87), but without success:
cp_log_export set name RSYSLOG-TCP filter-action-in "Drop"
Error: Argument [filter-action-in] is undefined for command: [set]
The sending logs to our RSYSLOG server was configured earlier:
[Expert@CASCPSMS:0]# cp_log_export show
Why I do not have use filter-action-in argument for cp_log_export command?
Have a question.
We know that log exporter can be used to export logs to an external logging server, however what I would like to do is export IPS and Threat Prevention logs from originating from specific GWs or CMA.
SmartEvent is used in our environment which clearly is already consolidating entries; however this is consolidating entries for every client our Provider. We need to export a specific customers entries to any external collector such as ArcSight.
Is there a way to export logs from a specific CMA or GWs from here?
From the CMA how can we extract IPS/Threat Prevention logs to an external logging server (ArcSight I believe). Ideally logs that originate from specific GWs.
The SK122323 above does not really show me any examples that make sense to me. We are running R80.20 with HFA103 as well.
the log-exporter filtering ability allows you to do just that:
You can activate/use an exporter that runs on that specific CMA (as the Log-Server), that will only export logs received by it - that CMA.
to export logs from a specific GW only, simply use the origin field, as in this example:
cp_log_export set name <name> filter-origin-in "GW_name"
that depends, who's the origin GW that appears in the logs?
if they are separate, which I believe they are, then you need to write all of them, as in the sk's example ("GW1,GW2,GW3")
cp_log_export set name <name> filter-origin-in "GW1,GW2,GW3"
You can verify in the FilterConfiguration.xml that there's an OR in that field (origin) with all your matching values. Should look like that:
<field name="origin" operator="or"> <value operation="eq">GW1</value> <value operation="eq">GW2</value>
<value operation="eq">GW3</value> </field>
In our case we only have a certainly set of GWs we require the logs from, however we only want IPS/TP logs be to exported.
<field name="origin" operator="or">
So the above is great! just need to expand on this to determine only IPS/TP logs from the above?
Yea, simply add the IPS/TP as in the sk examples, like this for both TP & Multi-GWs.
cp_log_export set name <name> filter-blade-in "TP" filter-origin-in "GW1,GW2,GW3"
cp_log_export set name <name> filter-blade-in "SmartDefense" filter-origin-in "GW1,GW2,GW3"
change/combine them both if needed.