Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Herschel_Liang
Collaborator
Jump to solution

How clipping logs field to syslog server used log exporter?

requirement:

How clipping logs field to syslog server used log exporter? For example:

12-18-2019 12:38:35 Local0.Info 172.22.14.40 1
2019-12-18T04:38:33Z SMS CheckPoint 28309 - [action:"Accept"; flags:"411908";
ifdir:"inbound"; ifname:"eth1"; logid:"0";
loguid:"{0x5df9ad49,0x0,0x290e16ac,0xc0000001}"; origin:"172.22.14.41";
originsicname:"CN=GW,O=SMS..pekhdi"; sequencenum:"2"; time:"1576643913";
version:"5"; __policy_id_tag:"product=VPN-1 &
FireWall-1[db_tag={F8CCF5D5-A96C-FC47-89B5-DE562533C7A6};mgmt=SMS;date=1576562059;policy_name=Standard\]";
dst:"180.163.222.208"; log_delay:"1576643913"; layer_name:"Network";
layer_uuid:"c0264a80-1832-4fce-8a90-d0849dc4ba33"; match_id:"2";
parent_rule:"0"; rule_action:"Accept"; rule_name:"Cleanup rule";
rule_uid:"e5c72b7a-811f-4b4b-a6fe-2cf646e0b4c7"; product:"VPN-1 &
FireWall-1"; proto:"6"; s_port:"23031"; service:"80"; service_id:"http";
src:"172.22.14.56"; ]
----------------------------------->
12-18-2019 12:38:35 Local0.Info 172.22.14.40 1
2019-12-18T04:38:33Z SMS CheckPoint 28309 - [action:"Accept";
origin:"172.22.14.41"; dst:"180.163.222.208"; proto:"6"; s_port:"23031";
service:"80"; service_id:"http"; src:"172.22.14.56"; ]

The client just want to send useful fields to syslog server but not all fields.

sk122323

Resolver Parameters

ParameterDescriptionPossible/Default Values
<mappingConfiguration></mappingConfiguration>The XML file containing the log field mapping scheme. If left empty will use the default settings.Default values are based on the 'format'.
<exportAllFields>true</exportAllFields>

When this field is set to 'true' all log fields will be sent regardless of whether or not they appear in the mapping scheme, , except for specifically black-listed fields in the relevant log format mapping file (<exported>false</exported>).

When set to 'false' only those fields which appear in the relevant log format mapping file will be sent (with exported flag true: <exported>true</exported>)

true / false

If I set field as <exported>false</exported>, it seem that I can change export fields in mappingConfiguration, but I have no more information about grammar and syntax in that files(targetConfiguration.xml &FieldsMapping.xml). Can someone give me a good advice? THX!

2 Solutions

Accepted Solutions
Dror_Aharony
Employee Alumnus
Employee Alumnus

if those are the only fields you'd like to export, then yea.

but you must include the exact syntax of the xml, best including: <exported>true</exported> for each field.

like this:

<field>
<exported>true</exported><origName>src</origName><dstName>src</dstName>
</field>

 

See full fieldMapping.xml example for guide usage

<?xml version="1.0" encoding="utf-8"?>
<fields>
<!-- field names/types of tables/fields can be found in fw.lf file -->
<field>
<exported>true</exported><!-- optional, field that is not exported won't be sent / by default it's true-->
<origName>time</origName><!-- field name in the log, can be different between joined and not joined log -->
<dstName>start</dstName><!-- the field will be exported with this name -->
<required>true</required><!-- optional, traffic without this field is dropped/default is false-->
</field>
<field>
<origName>src</origName>
<dstName>cef_src</dstName>
</field>
<field>
<origName>dst</origName>
<dstName>cef_dst</dstName>
</field>
<table>
<tableName>match_table</tableName><!-- name of the table/ field of the table is changed during join, see log_unification_scheme.C -->
<tableFormat></tableFormat><!-- optional, the format of table export, by default flat-->
<exported>true</exported><!-- optional, field that is not exported won't be sent / by default it's true-->
<required>false</required><!-- optional, traffic without this field is dropped/default is false-->
<fields>
<field>
<origName>layer_name</origName>
<dstName>cef_layer_name</dstName>
</field>
</fields>
</table>

 

View solution in original post

Dror_Aharony
Employee Alumnus
Employee Alumnus

he only needed to configure the fieldsMapping.xml properly in the targetConfiguration.xml.

like this:

<mappingConfiguration><MappingFile_Path-relative-to-exporter-folder></<mappingConfiguration>

# Example:

<mappingConfiguration>fieldsMapping.xml</<mappingConfiguration>

View solution in original post

9 Replies
Dror_Aharony
Employee Alumnus
Employee Alumnus

See the log-exporter sk122323 referring to the fields description sk144192 (https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...)

It states all fields raw names – that should help you to rather easily avoid exporting specific fields by clipping the exported logs to your syslog server.

Herschel_Liang
Collaborator
Off course, it will help but not change xml grammar and syntax.
Dror_Aharony
Employee Alumnus
Employee Alumnus

Not sure I understand what you mean regarding grammar/syntax.

Can you show an example or elaborate?

Herschel_Liang
Collaborator
just a example, such as:
if I change targetConfiguration.xml
<exportAllFields>false</exportAllFields>
and change FieldsMapping.xml
<fields>src</fields>
<fields>dst</fields>
<fields>proto</fields>
<fields>port</fields>
<fields>action</fields>
<fields>sxlate</fields>
I can implementation my requirement through exporting what I want to by changing xml files
Dror_Aharony
Employee Alumnus
Employee Alumnus

if those are the only fields you'd like to export, then yea.

but you must include the exact syntax of the xml, best including: <exported>true</exported> for each field.

like this:

<field>
<exported>true</exported><origName>src</origName><dstName>src</dstName>
</field>

 

See full fieldMapping.xml example for guide usage

<?xml version="1.0" encoding="utf-8"?>
<fields>
<!-- field names/types of tables/fields can be found in fw.lf file -->
<field>
<exported>true</exported><!-- optional, field that is not exported won't be sent / by default it's true-->
<origName>time</origName><!-- field name in the log, can be different between joined and not joined log -->
<dstName>start</dstName><!-- the field will be exported with this name -->
<required>true</required><!-- optional, traffic without this field is dropped/default is false-->
</field>
<field>
<origName>src</origName>
<dstName>cef_src</dstName>
</field>
<field>
<origName>dst</origName>
<dstName>cef_dst</dstName>
</field>
<table>
<tableName>match_table</tableName><!-- name of the table/ field of the table is changed during join, see log_unification_scheme.C -->
<tableFormat></tableFormat><!-- optional, the format of table export, by default flat-->
<exported>true</exported><!-- optional, field that is not exported won't be sent / by default it's true-->
<required>false</required><!-- optional, traffic without this field is dropped/default is false-->
<fields>
<field>
<origName>layer_name</origName>
<dstName>cef_layer_name</dstName>
</field>
</fields>
</table>

 

Herschel_Liang
Collaborator
<?xml version="1.0" encoding="utf-8"?>
<fields>
<field>
<exported>true</exported>
<origName>time</origName>
<dstName>start</dstName>
<required>true</required>
</field>
<field>
<exported>true</exported>
<origName>src</dstName>
<dstName>src</required>
<required>true</required>
</field>
<field>
<exported>true</exported>
<dstName>dst</dstName>
<required>dst</required>
<required>true</required>
</field>
<field>
<exported>true</exported>
<dstName>proto</dstName>
<required>proto</required>
<required>true</required>
</field>
<field>
<exported>true</exported>
<dstName>protocol</dstName>
<required>protocol</required>
<required>true</required>
</field>
<field>
<exported>true</exported>
<dstName>xlatesrc</dstName>
<required>xlatesrc</required>
<required>true</required>
</field>
<field>
<exported>true</exported>
<dstName>xlatedst</dstName>
<required>xlatedst</required>
<required>true</required>
</field>
<table>
<tableName>match_table</tableName><!-- name of the table/ field of the table is changed during join, see log_unification_scheme.C -->
<tableFormat></tableFormat><!-- optional, the format of table export, by default flat-->
<exported>true</exported><!-- optional, field that is not exported won't be sent / by default it's true-->
<required>false</required><!-- optional, traffic without this field is dropped/default is false-->
<fields>
<field>
<origName>layer_name</origName>
<dstName>cef_layer_name</dstName>
</field>
</fields>
</table>
</fields>
Is that right? but it also fail
[Expert@SMS:0]# cp_log_export status
name: test1
status: Not running
last log read at: 19 Dec 16:35:40
debug file: /opt/CPrt-R80.30/log_exporter/targets/test1/log/log_indexer.elg
[Expert@SMS:0]# tail /opt/CPrt-R80.30/log_exporter/targets/test1/log/log_indexer.elg
[log_indexer 4251 4128230208]@SMS[23 Dec 18:33:25] SourceConfiguration::init: - Reading logs mode is raw
[log_indexer 4251 4128230208]@SMS[23 Dec 18:33:25] SourceConfiguration::init - read log files from directory: [/opt/CPsuite-R80.30/fw1/log]
[log_indexer 4251 4128230208]@SMS[23 Dec 18:33:25] ExportConfiguration::setLinkIP - export_link_ip is set to default
[log_indexer 4251 4128230208]@SMS[23 Dec 18:33:25] No default mapping file found.
[log_indexer 4251 4128230208]@SMS[23 Dec 18:33:25] The mapping configuration must be defined in case not all fields are exported!
0 Kudos
Herschel_Liang
Collaborator

I have tried many times, but it seem failed in change xml(fieldsMapping.xml ).

Always errors:

[log_indexer 44884 4128398144]@SMS[23 Dec 22:27:01] No default mapping file found.

[log_indexer 44884 4128398144]@SMS[23 Dec 22:27:01] The mapping configuration must be defined in case not all fields are exported!

Could you please give me a detail example for this case? Many THX!

 

0 Kudos
Dror_Aharony
Employee Alumnus
Employee Alumnus

he only needed to configure the fieldsMapping.xml properly in the targetConfiguration.xml.

like this:

<mappingConfiguration><MappingFile_Path-relative-to-exporter-folder></<mappingConfiguration>

# Example:

<mappingConfiguration>fieldsMapping.xml</<mappingConfiguration>

Herschel_Liang
Collaborator

Meanwhile, I found that it can not defind export filedmapping about "information" like screenshots and  logs&monitor(new version) is no information fileds, quite ddifferent from old version. Is it expected or can it come true?

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events