Sanblast Appliance TE250x - High CPU

[Prabulingam_N1]

Dear All,

I have a scenario for customer as below:

Dedicated TE250x Appliance Gaia R77.30 with no JHF.

Per "top" command - we could see all cores and qemu were taking Very high CPU (~98%).

Due to this sometimes, users facing issue or error while Downloading/uploading of files.

Engine version is 57.990002630

Is there a way to find why qemu (vm's emulation) were using high CPU constantly? or it is normal behaviour for any TE appliance going High CPU when qemu's were emulating?

Regards, Prabu

[PhoneBoy]

Any particular reason you're not using the latest JHF?

Have you engaged the TAC to troubleshoot?

[Alexander_Eck]

Hi Prabu,

in my experience the CPU of the TE250x was going very high as the emulation of files tarted, too. After installing JHF the CPU calmed down for some time, but the increasing Internet Usage of our colleagues again resulted in high CPU of the TE250x after a few weeks. As the download of files was very very slow our management chose to use the ThreatCloud for HTTP and HTTPS Traffic and our TE250x is solely used as MTA and for SMTP Emulation. 

Even now I am sometimes receiving notification mails of High CPU (> 95%)  of our TE250x. That's why i guess it's save to say its normal behaviour for the TE250X to have high CPU while emulation is done. 

Cheers,

Alex

[Prabulingam_N1]

Dear Alex,

Thanks for your inputs.

Even I had opened TAC case and was informed that CPU will be going High since the Files were emulating,Pending and so on which is a Continuous process.

As long as there is no issue, then we can keep this high cpu as normal behaviour.

But customer had faced error in downloading of files hence we were concerned on High CPU.

Hence was looking for some suggestions.

Probably if customer still face any error or issue due to this High CPU - I may recommend them to enable SMT thru cpconfig - Atleast this will reduce the load on CPU due to more virtual cores being created by SMT.

Regards, Prabu

[Alexander_Eck]

Dear Prabu,

ok i understand.  When I'm remembering correctly we had several download errors when using our TE250X with R77.30 with no Jumbo Hotfix installed, too.  The download would just hang at 99% all day long  other downloads would still be completed. 

I'd really suggest to get a look at the Jumbo Hotfix for R77.30  the first few Takes already have several Performance and Stability Improvements integrated. 

Regards,

Alex

[Timothy_Hall]

Hi Prabu,

As mentioned earlier, high CPU usage is generally expected on a local Threat Emulation (TE) appliance.  However there were a few tips in my book to help avoid bottlenecks when performing local emulation, namely:

1) Enable Intel Virtualization Technology (VT) processor extensions if supported, see sk92374: Intel Virtualization Technology (VT) support compliance on Check Point appliances and sk92375: Enabling Intel Virtualization Technology (VT) in BIOS on Check Point appliances

2) By default if the CPU cores allocated for emulation are more than 90% busy, no more emulation VMs will start until the CPU load drops below that value, thus potentially causing a large backlog of emulation requests.

3) Enabling SMT can definitely help, but only if there is sufficient free RAM available, see the next item...

4) By default the emulation VM processes may not consume more than 70% of the system’s RAM. If there is not enough RAM available, startup of new emulation sessions will be delayed. A runaway memory leak in an unrelated process on the emulation system can potentially delay or bring emulation to a complete halt; keep up to date with the latest GA Jumbo HFAs on the TE appliance.

5)  Do NOT check the “Disable static analysis” checkbox in the Threat Emulation settings of the matching TP profile. Doing so will cause every single file encountered to be sent for emulation (even if it has been emulated previously), and should only be enabled in a lab environment or under the guidance of Check Point TAC.

6) Make sure the Protected Scope in the matching Threat Prevention (TP) rule invoking the TP profile for TE is defined as specifically as possible, and is not overly generous to avoid unnecessary amounts of emulation.  Note that it is possible to further clarify the Protected Scope in the TP profile itself under the Threat Emulation settings.

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

[Prabulingam_N1]

Dear Tim,

Thanks for your brief inputs.

I had advised for JHF and Static Analysis is not disabled as you stated.

No problem for enabling SMT as there is sufficient RAM.

Need to await if customer comes back if any further issue due to this High CPU.

Per TAC - we have reverted older Engine version 22.00xx and then again upgraded Engine version to 56.00xx.

Regards, Prabu