Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Jan_Kleinhans
Collaborator

Who had outages with the geo_location.C error in sk174783

Hello everybody,

we experienced a 5 hour full internet outage because of the error in geo_location.C created by Checkpoint.

As our Cluster failed over on thursday morning because of another bug we had these full outage as the services could not be started because of dynamic_objects process utilizing 100% CPU and not coming to an end.

Has anybody experienced the same problem in this hard impact? How did you handle it? What is your experience with checkpoint support in this moment?

Jan

0 Kudos
21 Replies
G_W_Albrecht
Legend
Legend

Can you please mention your version and jumbo take ?

0 Kudos
Jan_Kleinhans
Collaborator

R80.40 T119 as requested to install by Checkpoint Support because of latency with SSL Inspection. Take 119 produces the cluster crashes.

0 Kudos
Ruan_Kotze
Advisor

Thanks for the heads-up.  Busy troubleshooting a perplexing IPSEC tunnel issue and TAC had us deploy T119.

0 Kudos
Jan_Kleinhans
Collaborator

Keep my fingers crossed for you.

0 Kudos
Ruan_Kotze
Advisor

OK so we got bitten hard as well.  All tunnels on the cluster failed - traffic dropped with local interface spoofing.  Gateways also randomly stopped passing DHCP traffic between VLANs.  Rolled back to T118 and all issues resolved.

Lesson learned for the nth time - be wary of installing ongoing takes as part of troubleshooting.  And only update on one cluster member so that you can roll-back quickly.

0 Kudos
PhoneBoy
Admin
Admin

0 Kudos
Peter_Thome
Participant

Any information, why this sk can't be accessed with expert level as partner?

Especially, as this is linked in the supportcenter "Hot News" Slider and if you click the link, it's just showing: 

To view this solution, higher access level is required.

 

0 Kudos
PhoneBoy
Admin
Admin

The only higher access level is internal, and yes, it is accessible to employees still.
Users shouldn't run into this issue anymore unless we happen to release a faulty update.
Why it's linked in "Hot News" is a separate question.
@Ronen_Zel ?

 

0 Kudos
Ronen_Zel
Mod
Mod

Good catch. Since the issue was indeed already resolved, it should no longer appear in Support Center's "Hot News" section. The item is now removed.

Thanks for reporting this!

0 Kudos
the_rock
Authority
Authority

We currently have a case with TAC where policy push takes 8 minutes when geo location to block all the countries, except maybe 7 or 8 is enabled, but as soon as you disable that rule, policy takes 3 minutes on cloud mgmt server. Engineer mentioned internal sk and something about geo_location.C file, but date shows September 1st, so not sure if its 100% related. They also mentioned dynamic update i=object issue, but customer does not even use those in the policy. I am still waiting for next steps, so will see what they advise.

0 Kudos
Jan_Kleinhans
Collaborator

Hello,

we have also the problem that installing the policy takes a long time (didn't measure it) with updateable_ojects process taking 100% CPU for this time. I don't know if it started directly after the case mentioned in this sk or if it started later.
I opened a case for this problem today.

Regards,

Jan

0 Kudos
the_rock
Authority
Authority

Would you mind sharing the status of that issue? Is it still ongoing?

0 Kudos
Jan_Kleinhans
Collaborator

Yes the slow installation process is ongoing. In our environment the installation takes 10 minutes. But then it's working so that it doesn't have a big impact. 
Case is opened a few minutes so we will have to wait till checkpoint requests some information or will have a clue.

 

the_rock
Authority
Authority

Ok, sounds good. In our case as well, policy does work, it just takes more than double time when geo block rule is enabled, but lets see what TAC says. As long as policy functions, I will take that : )

0 Kudos
Micky_Michaeli
Employee
Employee

Hi @Jan_Kleinhans , @the_rock 

We are delivering to JHFs an improvement to the loading process of Geo Updatable objects. It's improving the policy installation duration when Geo Updatable objects are used in policy.

You can open SR for port-fix in case you wish to get this as a private fix instead of waiting for the GA release of the different JHFs as it can take several weeks.

Thanks.

0 Kudos
the_rock
Authority
Authority

Thank you for the response. So is there an official solution for customers who still have this issue or do we have to wait until next jumbo hotfix? I get port-fix can be requested, but I am little apprehensive about it, because it causes issues later on when next general jumbo take has to be installed...

0 Kudos
G_W_Albrecht
Legend
Legend

As @Micky_Michaeli wrote: You can either wait a few weeks until this HF is included in GA Jumbo or request a port to the GA Jumbo you have installed. As next GA Jumbo should already contain this HF your thought is unnecessary...

the_rock
Authority
Authority

@G_W_Albrecht ...I get what he said, no argument there :). I was more actually wondering if there was some sort of workaround for the time being that may not require rebooting the box, due to the port fix install.

0 Kudos
Micky_Michaeli
Employee
Employee

Hi @the_rock,

I think that we don’t have other WA than installing the HF that requires a reboot.

Thanks.

0 Kudos
the_rock
Authority
Authority

Thanks Micky. So, will this be permanently fixed in jumbo take 121 for R80.40?

0 Kudos
PhoneBoy
Admin
Admin

Most likely a future one.

0 Kudos