Log indexing delays and Correlation Unit dependency

[Michael_Horne]

Hello all,

I have been investigating log delays on our log server. The Logs themselves are present, if you open up the latest log file, but when you use the SmartLog with Log indexing there is a 10 - 20 minute delay.  I have been through some of the SK, but nothing seemed to help.

In the end I think it related to the sizing of the dedicated log server, which I will follow up on.

The question I have is, if there is a relationship between the SmartEventCorrelation unit and the log indexing?

We have a dedicated Management Server (running Smart Event server) and dedicated Log server.

Our log indexing delays were solved, by making sure there was a SmartEvent Correlation Unit running on the dedicated Log server as well as on the Management server.

I understand that there is log need to be processed by the correlation unit, and that if the correlation unit is not running on the log server, there is some extra communication between the log server and where the correlation unit is. Beyond the overhead of CPU and networking related to this, should there be an impact on the indexing of the logs?

Many thanks,

Michael

[_Val_]

It would help to state the version in use, and some HW parameters of the server.

Also if your management servers were upgraded in place, they still use EXT3 file system. You may want to check if this is the case. XFS, available with the clean install of 3.10 versions, is much more effective with the small DB transactions, which are used for indexing.

[Michael_Horne]

Hello We are running VM based and Log servers running the latest HF for R81.  I was basically just curious if there was s dependency between the correlation unit and the log_indexer function.   The last major upgrade to the log server was not via a fresh install, and we are looking at doing this at the next major upgrade.

[_Val_]

Uh, virtual machines... Please look into sk104848 for VM optimization, this might be your best shot.

For your question about relations between correlation units and log indexing, they are not connected directly. You can review the architecture of SmartEvent and SmartLog products in sk93970 and sk92769. I guess both engines were fighting for HDD read/write in a VM with non-optimal drive settings.

Hope this helps

[the_rock]

Had customer with similar issue and there was escalation TAC case for few months and since client was not keen on buying an actual dedicated physical appliance, TAC suggested memory upgrade and I believe that did make things better, but still not working way it should. I hate to say this, but your options are somewhat limited : - (

[Michael_Horne]

Hi,

I have ended up gathering some logging statistics for the logs using “cpstat” and “CPLogInvestigator”.  Based on the average log receive rate and total number of logs per day

I checked the dataCheck Point Smart-1 Security Management Platform Datasheet and saw that the matching physical appliance  would be the 6000-L given the number of FWs and the setup of logging and SmartEvent.

We will probably increase the resources on the Log server VM to match the same hardware specs and see if this fixes the problem with the log delay.

Still interested to know if there is a dependency between correlation unit and log_indexer.  If there is then simply updating assigned resources might not fix the issue and a redesign of who logs are processed  might need to be done.

[the_rock]

Im sure there is dependency, but I will let someone from CP confirm. I believe both would use process cpsead and it really boils down to cpsead being responsibly for correlation unit functionality.

[PhoneBoy]

R81.10 has some under the hood improvements that should improve log indexing as well.