Log Exporter stopped reading logs

[Peter_Baumann]

Hello again,
A new problem, this time with the log exporter:

[Expert@cplog01p:0]# date
Tue Jul 02 09:40:40 CEST 2019

[Expert@cplog01p:0]# cp_log_export status

name: fw.domain.com
status: Running (3986)
last log read at: 27 Jun 11:51:02
debug file: /opt/CPrt-R80.20/log_exporter/targets/fw.domain.com/log/log_indexer.elg

--> Log Exporter has stopped reading logs since some days but is still running.

We did a cp_log_export restart and it worked again.

Does someone know how to monitor the Log Exporter stopped working even when the process is still running?
Is this problem known?

Installed version of cplog01p:

[Expert@cplog01p:0]# cpinfo -y all

This is Check Point CPinfo Build 914000182 for GAIA
[IDA]
   No hotfixes..

[CPFC]
   HOTFIX_R80_20_JUMBO_HF_MAIN

[MGMT]
   HOTFIX_R80_20_JUMBO_HF_MAIN

[FW1]
   HOTFIX_R80_20_JUMBO_HF_MAIN

FW1 build number:
This is Check Point Security Management Server R80.20 - Build 007
This is Check Point's software version R80.20 - Build 047

[SecurePlatform]
   HOTFIX_GOGO_LT_HALO_JHF

[CPinfo]
   No hotfixes..

[DIAG]
   No hotfixes..

[Reporting Module]
   HOTFIX_R80_20_JUMBO_HF_MAIN

[CPuepm]
   HOTFIX_R80_20_JUMBO_HF_MAIN

[VSEC]
   HOTFIX_R80_20_JUMBO_HF_MAIN

[SmartLog]
   No hotfixes..

[MGMTAPI]
   No hotfixes..

[R7520CMP]
   No hotfixes..

[R7540CMP]
   No hotfixes..

[R76CMP]
   No hotfixes..

[SFWR77CMP]
   No hotfixes..

[R77CMP]
   HOTFIX_R80_20_JHF_COMP

[R75CMP]
   No hotfixes..

[NGXCMP]
   No hotfixes..

[EdgeCmp]
   No hotfixes..

[SFWCMP]
   No hotfixes..

[FLICMP]
   No hotfixes..

[SFWR75CMP]
   No hotfixes..

[CPUpdates]
   BUNDLE_R80_20_JUMBO_HF_MAIN_gogoKernel    Take: 47

[rtm]
   No hotfixes..

[Dror_Aharony]

I don't think such a status exists for this scenario (that shouldn't happen), but you can search the exporter's .elg for ERROR or last output:

/opt/CPrt-R80.20/log_exporter/targets/fw.domain.com/log/log_indexer.elg

cat $EXPORTERDIR/targets/fw.domain.com/log/log_indexer.elg | grep -i "ERROR"

tail -fn 1000 $EXPORTERDIR/targets/fw.domain.com/log/log_indexer.elg

Please send them to me personally (assuming they weren't already overwritten).

Next time it happens, copy them as quickly as possible or simply run: SmartEventCollectLogs

One manual way to notice if this happens again:

tail -fn 1000 $EXPORTERDIR/targets/fw.domain.com/log/log_indexer.elg | grep "rate \[log\]"

# if you stop receiving this line consistently for more than 1 minute (stating the estimated approximate exporter rate), then something is wrong.

Also, Any-chance your SIEM went down? (though the log-exporter should reconnect automatically).

Dror Aharony | Email: drora@checkpoint.com

[Peter_Baumann]

Hi Dror,
Thank you for the good tipps for troubleshooting the issue.
It is difficult to see the error since it shows up under rare conditions.

If I will catch once the error again I will see if I can get the error logs for it.

Thanks,
Peter

[Yaakov_Ohayon]

Hi,

Currently we don't know of any such problem in the GA version of log exporter.

Please open a support ticket and attach the log exporter's directory to it.

We really like to analyze this, it is important to us.

Thanks,

Kobi Ohayon

SmartEvent Core team leader