topic Re: How to quickly check log indexing backlog in Firewall and Security Management

How to quickly check log indexing backlog

Kaspars_Zibarts — Fri, 19 Feb 2021 09:22:02 GMT

We've been struggling with log search performance since upgrading to R80.10 (yes - we have upgraded VM HW, quadrupled it to 16 cores and 128GB RAM but still slow). Case is still open with CP and if anyone has good ideas let us know!

On the side I wrote this one-liner that gives you quick look at indexer backlog - it will show which files are being crunched apart from fw.log and fw.adtlog. This is MDS version so just take the outside loop away and it should work on SmartCentre server.

ls -1 $MDSDIR/customers | while read CLM; do mdsenv $CLM; echo; echo $CLM; cat $INDEXERDIR/data/FetchedFiles | while read line; do if [ `echo $line | awk '{print $7}'` -eq 0 ] && [ `echo $line | grep -c " fw."` -eq 0 ] && [ `echo $line | grep -c "serialization"` -eq 0 ]; then echo $line; fi done; done

Output will look like this, you can see that two files under CLM-x are being crunched apart from fw.log and fwadt.log

Good reading reference in sk112336 and sk97885

Re: How to quickly check log indexing backlog

PhoneBoy — Fri, 21 Jun 2019 09:05:08 GMT

Another one for the https://community.checkpoint.com/community/management/visibility-monitoring?sr=search&searchId=f1156ea7-6806-4811-b8e1-587324f74eb4&searchIndex=0‌ space

Thanks for sharing!

Re: How to quickly check log indexing backlog

Martin_Valenta — Tue, 06 Nov 2018 08:23:34 GMT

Is it normal that FetchedFiles file shows log files, which are not even existing on MLM at all?

Re: How to quickly check log indexing backlog

Kaspars_Zibarts — Tue, 06 Nov 2018 11:14:04 GMT

I would re-build FetchedFiles as described here

"SmartLog is not active or unreachable" warning when trying to connect with SmartLog GUI to SmartLog Server

Re: How to quickly check log indexing backlog

Yoel_Maaravi — Fri, 08 Feb 2019 15:32:12 GMT

nice