Policy install tasks stop at 99%

[SimonMeadows]

Hi,

I have notice that when I stack up a lot of policy installs to the point that some get queued and wait for others to complete, some of them will complete but the task in the recent tasks list sticks at 99%.

It appears to only be cosmetic, the individual steps within the task details window all show complete but the task it's self stays at 99% indefinitely, or until I clear it from the list.

This only seems to happen when I queue up a lot of installs at the same time.

Is this a Smart Console or a management server problem and is there a fix?

Thanks

Simon

 

[G_W_Albrecht]

Try the lasted GA Jumbo and latest Dashboard build !

[SimonMeadows]

The management server is on 81.10 Take 61 but the same happens on Take 55.

Smart console 81.10.9600.406 and .407 both do the same thing.

I have previously tried triggering all the policy installs directly via Management API and the api responses for checking some of the task's progress also stick at 99% so it seems to be more management server than smart console.

[Timothy_Hall · Accepted solution]

At 99% the gateway has already loaded and is enforcing the new policy.

Once the gateway has completed loading the new policy by 98%, the Check Point management components (SMS, SMEvent, Log Servers) perform an object database sync at 99%.  If you are hanging for awhile there, check the configuration and network reachability/bandwidth for any of these other components. Do you have a Check Point management object still defined that doesn't exist any more or is unreachable?  That will cause a delay at 99%.  If these management components exist but have poor network connectivity to the SMS or a shortage of resources on them such as CPU/memory, that can also cause a delay at 99%.

[SimonMeadows]

No additional management objects that I can see. Just the four that are online and have no connection issues or resource issues that I can see.

It's not always the same gateways that have the issue.

I have left some for a day and they stayed at 99% for the whole day before I cleared the tasks.

How long would I have to wait for the object database sync step to complete?

It only seems to happen to installs that have been queued up. If I only install 2-3 policies it never happens.

[Timothy_Hall]

The object sync shouldn't take more than a few seconds assuming you don't have a gargantuan number of objects or resource issues on your management components.  I suppose it is possible if numerous object syncs are running simultaneously, that they are stepping on each other's toes somehow and perhaps deadlocking somewhere.  Probably would need to have a debug running while reproducing the situation.  I think dbsync would be the process to look at but it wouldn't surprise me if fwm was involved as well.

[SimonMeadows]

Thanks for the pointers.

I'll dig into the logs next time it happens and see if I can find anything.

[Gary_Scott]

I also have this problem on a domain within an MDS. What is the fix to clear this? Stopping and starting the domain did not have any impact.

[Liat_Cihan]

Hello Gary,

We are familiar with this issue. the immediate solution is to perform cpstop/cpstart and install policy again.

We have a fix for it in R81_10_jumbo_hf_main 45.

Good luck

Liat

[SimonMeadows]

Hi Liat,

We are not running MDS but are on R81.10 JHF Take 66 and it still happens.

Is there a fix to be released for the non MDS version?

Thanks,

Simon.

[Liat_Cihan]

Hello Simon.

In your case, since you are on JHF 66, It might be related to a different reason.

I suggest to open a ticket so we will be able to assist you.

Liat

[Gregg_Loraas]

We have the same issue doesn't happen if we stick to around 2-4 policy pushes at a time.