Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Ankur_Datta
Collaborator

Log exporter send logs in UTC time zone to syslog server

Hi All,

 

We configured log exporter on our MLM and send logs of all CLM to syslog server. On checking on syslog server we found out logs are coming in UTC Time zone. I gone through sk133472 and not able to understand the last two lines.

 

"The CP Log Server can send logs to syslog in different timezone, and it needs to be UTC so target Log Server can convert it to the local timezone.

This is also how Splunk and other SIEM vendors behave."

Is this means we need to send logs through log exporter in UTC time zone

We are using IST time zone. 

 

Please guide.

6 Replies
PhoneBoy
Admin
Admin

What it means is that Log Exporter will always send logs in UTC format as this is what other vendors are expecting to receive it in.
0 Kudos
Ankur_Datta
Collaborator

Is there any way we can change the timezone and syslog get logs in timezone we are using?

0 Kudos
PhoneBoy
Admin
Admin

While @Dan_Zada can confirm, I believe this is not currently possible and would require an RFE.
The SK indicates Log Exporter sending syslogs in UTC is expected, RFC-compliant behavior.

0 Kudos
Ivan_Svinoukhov
Explorer

 

Old post, but could be useful for people who haven't found a way to change the time stamps from log exporter.

I had to deal with the same issue recently where logs are shipped in UTC and it's always a pain to sort through the logs and having to convert timestamp manually. 

I've found a solution to this issue by modifying syslog format definitions.

Go to $EXPORTERDIR/targets/$target_syslog_server/conf

Make a backup of current SyslogFormatDefinition.xml 

open file in text editor and find a section that starts with <!-- TIMESTAMP-->, should look something like below:

 <!-- TIMESTAMP-->                                                                                                                                                                                                                                              
<header>
<default_value>-</default_value>
<assign_order>first</assign_order>
<field>
<name>time</name>
</field>
<callback>
<name>format_timestamp</name>
</callback>
</header>

you can delete this entire section from the config file and restart your log exporter process

# cp_log_export restart

This will get rid of the UTC time stamps and will only include timestamp that are generated by your gateways in your respective timezone. 

It's also good idea to modify the header_format to exclude fields that are not required in the logs as your syslog SEIM could be addition additional header information such as host names and etc.

The section looks like the following 

<!-- Log Header will be generated according this format string. Every (header_format_replacment_string val) "{}" will be replaced with header value -->                                                                                                                
<header_format>{}{} {} {} {} {} {} </header_format>

Here's what it looks like before the modifications (I'm using rsyslog as my SEIM) timestamp in UTC: 

<134>1 2021-02-19T17:03:00Z RH-SMS-02 CheckPoint 17769 - [action:"Accept"; flags:"411908"; ifdir:"inbound"; ifname:"eth2"; logid:"0"; loguid:"{0x602fef44,0x0,0x49bcb9cb,0xd5bff1ea}"; origin:""; originsicname:"CN=FW-5100-S01-S2S-MGMT,O=RH-SMS-01..y8sqgb"; sequen
cenum:"2"; time:"1613754180"; version:"5"; __policy_id_tag:"product=VPN-1 & FireWall-1[db_tag={64437249-B834-954A-BD81-B1FC6A733BB8};mgmt=RH-SMS-02;date=1613751112;policy_name=PolicyPackage-TR1-IMP\]"; dst:""; inzone:"Internal"; layer_name:"TR1"; layer_uuid:"3b06
02c8-80c9-44df-9385-87b7456bc8d1"; match_id:"0"; parent_rule:"0"; rule_action:"Accept"; rule_name:"Implied Rule "; rule_uid:"0E3B6801-8AB0-4b1e-A317-8BE33055FB43"; outzone:"Internal"; product:"VPN-1 & FireWall-1"; proto:"6"; s_port:"60053"; security_inzone:"Interface_s-m
gmt"; service:"18192"; service_id:"CPD_amon"; src:""; ]

Time stamp in EST after removing TIMESTAMP section

<134>1 2021-02-19T12:04:01.219674-05:00 RH-SMS-02 CheckPoint - - -  18036 - {} {} [action:"Accept"; flags:"411908"; ifdir:"inbound"; ifname:"eth2"; logid:"0"; loguid:"{0x602fef7e,0x0,0xfb64ed58,0xfb2725e8}"; origin:""; originsicname:"CN=FW-5100-S01-S2S-MGMT,O=R
H-SMS-01..y8sqgb"; sequencenum:"3"; time:"1613754238"; version:"5"; __policy_id_tag:"product=VPN-1 & FireWall-1[db_tag={64437249-B834-954A-BD81-B1FC6A733BB8};mgmt=RH-SMS-02;date=1613751112;policy_name=PolicyPackage-TR1-IMP\]"; dst:""; inzone:"Internal"; layer_nam
e:"TR1"; layer_uuid:"3b0602c8-80c9-44df-9385-87b7456bc8d1"; match_id:"0"; parent_rule:"0"; rule_action:"Accept"; rule_name:"Implied Rule "; rule_uid:"0E3B6801-8AB0-4b1e-A317-8BE33055FB43"; outzone:"Internal"; product:"VPN-1 & FireWall-1"; proto:"6"; s_port:"59331"; secur
ity_inzone:"Interface_s-mgmt"; service:"18192"; service_id:"CPD_amon"; src:""; ]

as you can see  above I have some additional field now - {} {} which are just values that no longer exist from log exporter, so that's where you have to modify your header_format section in order to get rid of the extra fields in the logs. So I changed mine from 

<header_format>{}{} {} {} {} {} {} </header_format>

to

<header_format>{}{} {} {} </header_format>

and this is the result:

<134>1 2021-02-19T12:11:36.007271-05:00 RH-SMS-02 CheckPoint - - -  19916 [action:"Encrypt"; flags:"411908"; ifdir:"inbound"; ifname:"bond1"; logid:"0"; loguid:"{0x602ff158,0x0,0x1b1a89ba,0xd17d4c8a}"; origin:""; originsicname:"CN=FW-5100-D01-S2S-MGMT,O=RH-SMS-
01..y8sqgb"; sequencenum:"1"; time:"1613754712"; version:"5"; __policy_id_tag:"product=VPN-1 & FireWall-1[db_tag={0A65B690-0B26-A845-9F63-78237EB9CBAF};mgmt=RH-SMS-02;date=1613670734;policy_name=PolicyPackage-DL1\]"; community:"TR1-DL1"; dst:""; fw_subproduct:"
VPN-1"; inzone:"External"; layer_name:"DL1-SRX_policy_Opt Network"; layer_uuid:"b69886f5-1979-4e4f-8d62-6973f494244e"; match_id:"14"; parent_rule:"0"; rule_action:"Accept"; rule_name:"Inter-Cluster-D Traffic"; rule_uid:"64d66cc5-63d5-4a0c-8cc1-97486e3f3e43"; methods::"ES
P: AES-128 + SHA1 + PFS (group 5)"; outzone:"External"; peer_gateway:""; product:"VPN-1 & FireWall-1"; proto:"6"; s_port:"48158"; scheme::"IKE"; security_inzone:"NAME_COLLISION_RESOLVED_Interface_d-InterCluster"; service:"4369"; service_id:"NAME_COLLISION_RESOLVE
D_2_TCP_4369"; src:""; vpn_feature_name:"VPN"; ]

still have some additional - - - characters, but at least I have my timestamps in EST and looks a bit better than having extra fields. You can always play around with header_format to get the results you want.

Hope that helps! 

🙂

0 Kudos
Forsaken_61
Explorer

Hi @Ivan_Svinoukhov 

Thanks for your guidelines.

It seems like I'm missing the SyslogFormatDefinition.xml file. I don't have It under $EXPORTERDIR/targets/MYTARGET/conf.
I've a bunch of other .xml files there, but not the SyslogFormatDefinition.xml.

Under $EXPORTERDIR/conf,
I can see the SyslogFormatDefinition.xml file. But I dont think that file will be related to my Syslog Destination target that I've configured.

I'm running R81.20. Maybe there has been som changes related to your post which was year 2021.

Thankful If someone could help out with this.

0 Kudos
Ivan_Svinoukhov
Explorer

hi @Forsaken_61 . Yes, looks like you are correct. They seem to have updated the configuration. 

I've checked this in our lab also running r81.20 and created a new target and indeed there's no SyslogFormatDefinition.xml that is in the conf directory. 

If you change the settings in $EXPORTERDIR/conf this will be applied globally to all target servers. If you want to apply settings individually to different targets you have to follow these steps:

  1. Copy the SyslogFormatDefinition.xml from $EXPORTERDIR/conf to $EXPORTERDIR/targets/MYTARGET/conf
  2. make all the necessary adjustments to the file
  3. Then in the $EXPORTERDIR/targets/MYTARGET/targetConfiguration.xml file you have to point it to the new configuration in the header file section
    • so change  <formatHeaderFile></formatHeaderFile>  to point to the new file eg.: <formatHeaderFile>./conf/SyslogFormatDefinition.xml</formatHeaderFile>
  4. restart your target exporter cp_log_export restart name $mytarget

This should apply the settings to only that specific target server. You'll have to play around with the config settings inside the SyslogFormatDefinition.xml file to get the desired results of course.

Hope this helps! 🙂

0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events