R77.20.85 performance issue on centrally managed SMB

[HristoGrigorov]

Guys,

That build is causing significant traffic delays and CPU load is higher than that of R77.20.81. 

Any of you experiencing similar problem ?

[Naftali_Oziel]

Hi, which appliance are you running this on 700/900/1400.  Plus which blades to you have enabled?

[HristoGrigorov]

Centrally managed 1470 appliance. FW, VPN, AC/UF, IPS blades. Same configuration is fine on R77.20.81. 

Currently one node of the cluster is running 81 and one 85. I flip them over from time to time and as soon as 85 becomes active load average increases well above 1.0 while on 81 it rarely goes up to 0.8.

[Naftali_Oziel]

Ok. Thanks.  Do you have any standalone (SMB)  and those to SMP but in a non-cluster setup?   Checking my end to compare stats but am running 1490 using the same blade setup.   However, are local managed none central or cluster.  

[HristoGrigorov]

Sorry, only those two 1470s in a cluster. I don't know why, but that build makes things run really slow. I am going to revert it back to 81 today which is as you know long and painful procedure because flashing previous firmware version is not supported. 

[Pedro_Espindola]

I have the same problem.

Centrally managed 1470. Build 990172731

Before upgrade, average CPU load was below 20% during work hours. After upgrade, average was at 35% on sunday and now, during work hours, it is peaking at 85%... I can't tell the average, because the gw stops responding to SNMP every few minutes.

[HristoGrigorov]

Yesterday, I gave it another try. And users immediately started to complain about network being slow. Switched over to the other member (77.20.81) and everything went back to normal. Btw, 77.20.81 build 541 has exactly the same problem. R77.20.81 build 525 is rock stable and performance is noticeably better. So I am staying with that for the time being. 

[Pedro_Espindola]

Noticed some instability periods in 541 for centrally managed appliances, but 77.20.85 was way worse and instability seemed to be permanent, not momentary. After reverting to 77.20.81 541 things definitely improved!

Had no issues with 541 in locally managed ones. I didn't test 77.20.85 GA on locally managed appliances.

[HristoGrigorov]

R77.20.85:

R77.20.81:

Time frame is: 08:30 AM - 09:30 AM (1 hour) on work days. Same policy.

[Pedro_Espindola]

Did you open tickets with TAC about this build? And what about Build 541? Did they say anything?

[HristoGrigorov]

I am sorry, I don't have time for that at the moment. I am tackling another SR with TAC at the moment. I just reverted back so not actually using it at the moment.

[G_W_Albrecht]

Strange - one of our customers has reported performance gains after changing from R77.20.81 to R77.20.85 on a single SMB GW.

[HristoGrigorov]

That's interesting. Is it locally or centrally managed ? 

[G_W_Albrecht]

That is a locally managed SMB unit, so there is much difference to your environment.

[HristoGrigorov]

Yes, it seems issue is only on centrally managed SMBs.

[Naftali_Oziel]

locally managed is also running into issues  from a GUI performance getting timeout error msg.   As for actual traffic don't believe there is an impact.

[HristoGrigorov]

Thanx for sharing. The poor GUI performance was one of the reasons I went for central management. I am so far very satisfied with 81. It performs beautifully here even with HTTPS inspection enabled. I checked again 85 release notes and there is nothing interesting for me that was changed. I'd love to see some features from R80.20 incorporated into SMB firmware but that is not likely to happen.

Once we solve together with R&D that nasty SecureXL problem with static routes I will be very happy with what SMB is  

[G_W_Albrecht]

Yes, it is not recommended to keep a GUI connection open without doing any configuration changes. WebGUI is not really a tool for monitoring as it uses too many ressources on a unit that does not have much...

[Naftali_Oziel]

GUI has latency/timeout issues even on configuration.  Believe this is all related to the new features added and simply will require optimization.  This issue occurred a few revisions back, can't recall which one.  Hopefully another build will be released to address all these shortly as I'd like to upgrade to production system. 

[Kaloyan_Kirchev]

Hi,

A client also has locally managed 1430 with R77.20.86 but performance is....low.

I had CPinfo here and looking at it with DiagnoseView but cannot find any solution.

Interesting is that I see that memory is 90% and most used proccess is "fw sfwd"(as I saw this is main 1430 proccess".

Any Ideas? Any suggestions? 

[KAPIL_RANA]

I have the same problem. 77.20.80 ruining well  but after updating 77.20.81 or 77.20.85  hang networ and

cpu   load  90 or 100 %

Before update  cpu load 20 %

Locally  managed 1470

[Naftali_Oziel]

Suggest you open SR asap.  Never had issues with r77.20.81 on CPU load. 

[HristoGrigorov]

Don't think CheckPoint is going to do something about it until someone opens SR. 

[Pedro_Espindola]

Yeah, but there is no way I can keep a production gateway in that version while R&D investigates.

Best I can do is try to replicate in a lab and then reach TAC.

I have never seen such an unstable build. IMO this should still be in EA or internal testing.

[HristoGrigorov]

Yep, same here. Only that I do not have test lab at my disposal. So, not much of a help really. 

[Naftali_Oziel]

Fully understandable not impacting production and agree that this build is not GA ready.  I would open an SR of the issue at hand providing as much info as possible and CP to than attempt and replicate the issue in their lab.  This is what I've done in the past by upgrading giving the logs/cpinfo snap shots, processes etc. and than revert back.  This way they have a way to run with it.   Just my two cents.

[G_W_Albrecht]

My first suggestion when encountering an issue that does not appear at other customers is a fresh firmware install using USB medium.

[KAPIL_RANA]

I have already install fresh firmware 77.20.80 using medium USB but after upgrade to 77.20.85 facing same problem cpu load 80 to 90 %

[Pedro_Espindola]

It does appear in other costumers. We have at least 4 different cases here in this topic with the same behavior.

[G_W_Albrecht]

I have also seen that a WebGUI firmware update made the unit unstable, and only USB firmware install resolved that. But of course, if the firmware is buggy this will not help!