Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
DeletedUser
Not applicable

Log Exporter LEEF Field Mappings

Please use this discussion as a guide to understand how Check Point syslog Log Exporter maps Check Point logs to the LEEF format. This discussion is based upon R80.30 and may change in future versions.

LEEF fields have their own names such as cat, devTime, url, etc. Check Point fields such as src and dst that already match a LEEF field name do not need to be mapped from a Check Point to a LEEF name so are not covered in this discussion.

Note: in this discussion we refer to the raw Check Point field value. Check Point may translate the raw field name to show a different display name in the user interface like Tracker in R77.30 or SmartConsole in R80.x. 

LEEF Event Components, IBM Knowledge Center

The Log Event Extended Format (LEEF) is a customized event format for IBM Security QRadar. The LEEF format consists of the following components.

LEEF Header Mapping

The LEEF header is a required field and is composed of a pipe delimited (|) set of values that identifies Check Point events to QRadar.

Check Point fields are added to the header as specified in assign_order in the file $EXPORTERDIR/conf/LeefFormatDefinition.xml:

  • first (use first added value - default)
  • last (use last added value)
  • join (join between values)
  • init (set value once to header formatted string on init and do not generate per every log)

LEEF Format Header Definition Examples (note: a space is added between the “|” delimiter to make it easier to see the values).

  • LEEF:Version | Vendor | Product | Version | Event ID |
  • LEEF: 2.0 | Check Point | Log Update | 1.0 | Check Point Log |

 

LEEF Version: assign_order is set to init

LEEF: 2.0

Vendor: the assign_order is set to init

Check Point

Product: the assign_order is set to first

This default is Log Update, but may also be the value from the fields; product or productname.

Version: the assign_order is set to init

1.0

Event ID, the assign_order is set to init

The default is Check Point Log, but may also be the value from the fields protection_name, appi_name, action.

 

Event Attributes

The event attributes identify the payload information of the event in a set of key value pairs that provide detailed information about the security event. Each event attribute is separated by the tab delimiter character.

 

Examples

  • key=value<tab>key=value<tab>key=value<tab>key=value<tab>
  • src=192.0.2.0     dst=172.50.123.1    sev=5 cat=accept    srcPort=81     dstPort=21    usrName=joe.black

 

Predefined LEEF Event Attributes, IBM Knowledge Center

The LEEF format contains a number of predefined name=value pairs event attributes, which allow QRadar to categorize and display the event. Log Exporter uses these keys when possible.

 

Custom Event Keys, IBM Knowledge Center

Vendors and partners have the option to define their own custom event keys and include them in the payload of the LEEF format.

 

Check Point events that do not fall into one of the pre-defined LEEF event attributes are non-normalized, which means they are not displayed by default on the Log Activity tab of QRadar. To view custom attributes and non-normalized events on the Log Activity tab of QRadar, you must create a custom event property. Non-normalized event data is still part of your LEEF event, is searchable in QRadar, and is viewable in the event payload.

 

Mapping of Check Point Fields to Pre-defined LEEF Event Attributes

The below is the Log Exporter LEEF Field Mapping from R80.30 from $EXPORTERDIR/conf/LeefFieldsMapping.xml where origName is the Check Point raw field name and dstName is the LEEF attribute sorted by the LEEF dstName field name.

 

Callback Functions

In the name column are example uses of the callback functions where the value is replace_value. This function swaps values based on a key:value chart. We use this to map the Check Point severity (and other fields) to known LEEF values.

origName

dstName

name

key

value

tableName

attack_information

attackInformation

       

attack_name

attackName

       

status

attackStatus

       

business_impact

businessImpact

       

action

cat

       

industry_reference

cve

       

resource

destinationDnsDomain

     

resource_table

time

devTime

       

client_dst

dst

       

ipv6_dst

dst

       

received_bytes

dstBytes

       

server_inbound_bytes

dstBytes

       

client_inbound_bytes

dstBytes

       

mac_destination_address

dstMAC

       

d_port

dstPort

       

destination_port

dstPort

       

xlatedst

dstPostNAT

       

xlatedport

dstPostNATPort

       

recipient-recipients

emailRecipient

       

from

emailSender

       

sender

emailSender

       

subject

emailSubject

       

supress_logs

eventsCoalesced

       

extracted_files

extractedFiles

       

extracted_file_types

extractedFileTypes

       

extracted_hash

extractedHash

       

file_MD5

fileHash

       

file_id

fileID

       

file_id

fileId

     

file_table

file_name

filename

       

file_size

fileSize

       

filetype

fileType

       

file_name

fname

     

aggregated_file_table

file_name

fname

     

file_table

file_size

fsize

     

file_table

OU_group

identGrpName

       

source_machine_name

identHostName

       

malware_family

malware

       

malware_activity

malwareActivity

       

packet_capture

pcap

       

performance_impact

performanceImpact

       

phone_number

phoneNumber

       

policy_name

policy

       

policy

policyName

       

profile

profileName

       

protocol

proto

       

remediated_files

remediatedFiles

       

app_risk

sev

replace_value

default

0

 

app_risk

sev

replace_value

0

0

 

app_risk

sev

replace_value

1

2

 

app_risk

sev

replace_value

2

4

 

app_risk

sev

replace_value

3

6

 

app_risk

sev

replace_value

4

8

 

app_risk

sev

replace_value

5

10

 

severity

sev

replace_value

default

0

 

severity

sev

replace_value

0

0

 

severity

sev

replace_value

1

2

 

severity

sev

replace_value

2

5

 

severity

sev

replace_value

3

8

 

severity

sev

replace_value

4

10

 

app_risk

sev

replace_value

default

Unknown

match_table

app_risk

sev

replace_value

0

Unknown

match_table

app_risk

sev

replace_value

1

Low

match_table

app_risk

sev

replace_value

2

Low

match_table

app_risk

sev

replace_value

3

Medium

match_table

app_risk

sev

replace_value

4

High

match_table

app_risk

sev

replace_value

5

Very-High

match_table

app_risk

sev

replace_value

default

Unknown

primary_application

app_risk

sev

replace_value

0

Unknown

primary_application

app_risk

sev

replace_value

1

Low

primary_application

app_risk

sev

replace_value

2

Low

primary_application

app_risk

sev

replace_value

3

Medium

primary_application

app_risk

sev

replace_value

4

High

primary_application

app_risk

sev

replace_value

5

Very-High

primary_application

sha1

sha1

       

sha256

sha256

       

protection_name

signature

       

client_ip

src

       

ipv6_src

src

       

sent_bytes

srcBytes

       

server_outbound_bytes

srcBytes

       

client_outbound_bytes

srcBytes

       

imsi

srcMAC

       

mac_source_address

srcMAC

       

s_port

srcPort

       

xlatesrc

srcPostNAT

       

xlatesport

srcPostNATPort

       

proxy_source_ip

srcProxyIP

       

suspicious_events

suspiciousEvents

       

destination_dns_hostname

url

       

resource

url

       

orig_from

usrName

       

user

usrName

       

 

10 Replies
Sven_Glock
Advisor

Hi Bob,

 

thanks for your nice post.

As I am facing exactly the problem you described.

Does this documentation also fit to R80.10?

If not what are the differences?

 

Thanks for your support.

 

Regards

Sven

DeletedUser
Not applicable

Diffed an R80.10 LeefFieldsMapping.xml with the same file from R80.30. The major differences seem to be that the app_risk and severity callbacks numeric values in R80.10 have string values in R80.30, e.g. 0 replaced with a value of Low risk. Header field mapping may be handled differently. 

What problems are you running into?

Lubomir
Explorer

Hello Bob,
we recently ran into a problem when forwarding logs to IBM QRadar. I know this is mainly IBM's issue, but according to step 4 of their troubleshooting guide (https://www.ibm.com/support/pages/node/876650), "product" needs to be mapped to "cat" (instead of action) to make it work with their parser. This workaround worked well with version R80.10, but one of our customers recently updated to R80.40 and since then we are unable to map product (or productname) to this field. It still works as part of header definition in LeefFormatDefinition.xml, so I believe it's still one of the available parameters, but it doesn't seem to work when processing LeefFieldsMapping.xml. Were there any changes that could affect this behavior? Is there any other field that contains product name information and could be used in LeefFieldsMapping.xml? I know that product is not listed in origName column in your table, but it worked anyway. I don't know why IBM ignores product name in the header, but changing mapping was way easier than writing own parser. Thank you in advance for any answer.

 

Best Regards,

Lubomir

Yaakov_Ohayon
Employee
Employee

Hi,

There are few known issues in the integration between log exporter and Qradar:

1. The Leef format, as log exporter forms, is not the exact format Qradar expects. the main issue is that CP field "product" is not mapped to "cat".

Solution: change LeefFormatDefinition.xml and LeefFieldsMapping.xml to adjust the header\mapping accordingly.

2. CP logs can have multiple blades reported in a single log, for instance, "Application Control" and "Content awareness". As we map the product field to "cat", Qradar fails to parse those logs and leaves them out of the structured data.

Solution: Log exporter will send only the first blade name in the product field.

3. In R80 and up, a layered rulebase was intruduced. Now logs are being reported with the match table, that details all the layers, rule names, rule numbers matched along the way. it causes logs to be exported with duplicate fields (with different values) as we "flatten" the table to single fields.

Solution: Log exporter will send only the first line from the match table.

 

All those solutions are currently in development and will be integrated into R80.X JHFs soon.

 

Thanks.

Lubomir
Explorer

Hello Yaakov,

we had working workarounds for first two issues (on R80.10). The problem is, that the solution for the first issue is not working for us after upgrade to R80.40. After upgrade we have following mapping:

<?xml version="1.0" encoding="utf-8"?>
<fields>
<!-- Filter out fields -->
		<field><origName>flags</origName><exported>false</exported></field>
		<field><origName>__policy_id_tag</origName><exported>false</exported></field>
<!-- End of filter out -->
                <field><origName>product</origName><dstName>cat</dstName></field>
		<field><origName>time</origName><dstName>devTime</dstName></field>
		<field><origName>protocol</origName><dstName>proto</dstName></field>
...

but our logs still look like this (we restarted exporter):

LEEF:2.0|Check Point|VPN-1 & FireWall-1|1.0|Accept|devTime=1595317396	proto=HTTP	...

When we tried to map another field to cat, it worked (for example we mapped protocol and it showed cat=HTTP). But we were unable to map product. I thought you removed it because it wasn't used by default, but according to sk144192, it should be still available.

 

We solved second problem by adding following escape section(s), but if it will be done automatically in the future it would be great.

<char>
  <orig>Application Control URL Filtering</orig>
  <escaped>Application Control</escaped>
</char>	

 

Anyway, thank you for your response. In the end it looks like we use right solutions and according to CP documentation it should still work on R80.40. Now lets hope JHF will help.

 

Best Regards,

Lubomir

0 Kudos
Yaakov_Ohayon
Employee
Employee

Hi,

The issue you describe is known to us and we already fixed it in R80.30 and R80.20 JHFs. 

This issue has no WA, please open a support ticket so we can provide the known fix, and I will make sure it will be integrated into R80.40 JHF also.

Yaakov_Ohayon
Employee
Employee

Hi,

Small update, the issue is fixed in R80.40 since take 38.

You can safely install and use it.

 

Kobi Ohayon

0 Kudos
Beja
Contributor

Hi Yaakov

Currently we are facing same problem with a QRADAR you said in point number 3. "duplicated fields due to layers.

Do you know how we could send only first line from match table?

Regards.

Yaakov_Ohayon
Employee
Employee

Hi @Beja ,

Currently you cannot, we are working to integrate a solution where you will get only the first line to all jumbo versions

 

dehaasm
Collaborator

Hi I am using version R81.10 latest jumbo and we configured the Log Exporter to send the correlated events to QRADAR and are being send. They are in fact not yet (readable) by QRADAR which is under investigation because we do send the LEEF formatted data to it. Inside the data there is no severity field present how is that possible and how to include this field? Shouldn't this be send by default as well to SIEM/QRADAR?

0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events