R80.10 Management Synchronization is not working

[Gabriel_Briones]

===========================================

Check Point Management Version (R80.10 with HFA 70)

===========================================

1. Primary/Active management's lv_current partition utilization becomes 100%

2. Secondary/Standby management was promoted to Secondary/Active and we've been using the secondary management to create firewall rules since.

3. Clear up files on Primary/Active (now Primary/Standby).

4. Tried synchronizing the Secondary/Active with Primary/Standby but fails with error "Synchronization error: NGM failed to retrieve last publish time"

5. Situation has been like this for a couple of months now and we can't synchronize the management.

6. A couple of steps has been carried out as advised by Check Point support including the following but none of them works
clear Smart Console Cache
clear $FWDIR/conf/mgha/*
reset SIC
install policy/database
cpstop and cpstart

7. Check Point support advised us to rebuild the primary which we did, we install a fresh R80.10, apply the relevant HFA and restore the database. Unfortunately migrate export on Secondary Management is not supported so we need to use what ever is the last backup on the primary. Tried synchronizing again but we still face the same issue.

8. Currently, we can still use the now Secondary/Active management but we can't create a backup of the database and it just a matter of time before this active management encounters issue.
9. From Gaia, i've created a snapshot of the secondary/active management and restore it in my lab setup, install another R80.10 management which acts as a primary and I've managed to replicate the issue.

10. On the lab, I've tried doing a "promote_util" on the Secondary/Active management but it keeps on generating a core dump. Tried resetting the SIC, clear cache, revoke certificate but none of them works. I even install the latest HFA (Take 154) and repeat all the steps I could think of to restore the synchronization but it doesn't work as well

11. The case is now with the R&D and we're still waiting for a possible solution to our issue. An OVF template of both the primary and secondary management has also been provided.

12. I seems to be running out of options and would like to seek some help from this forum

[PhoneBoy]

You're definitely in the territory where R&D needs to be involved.

What SR is this in relation to?

Feel free to send it to me in a private message.

Did you submit the core dump from promote_util via your TAC case?

[G_W_Albrecht]

Out of my experience, this is the big drawback of Management HA - if the primary is distroyed, the secondary may be unable to take the role of the primary. Only a snapshot / backup of the primary management will help. In R7x.xx you also did loose all database revisions that were stored on primary only. I assume that you have also tried the promote_util alternative from sk114933 ?

[Gabriel_Briones]

Hi Dameon / Gunther,

 Thank you for your response. I really appreciate it.

I did tried the "promote_util" only on the lab but it keeps on giving me a core dump. And i'm not too confident to try it on my production box.

I noticed though that when I do the command 'cpprod_util FwSetPrimary 1' on the Secondary/Active management, I am able to perform a 'migrate export' and able to restore the exported backup successfully to the Primary/Standby. But I notice though after the restoration that port 18190 is not running on the Primary/Standby and I can't connect using SmartConsole.

Dameon,  case number has been sent via private message. I haven't given the core dump to the support as it's from my lab.

[Gabriel_Briones]

It turns out, the trick of running 'cpprod_util FwSetPrimary 1' on the Secondary is being suggested by Check Point as per the sk65360.

But i'm still stuck at not being able to connect via SmartConsole when I restore the database from Seondary to my primary

[PhoneBoy]

I didn't see that particular suggestion in the TAC case you sent me.

But what that trick does is allows a migrate export to complete from the secondary (which is normally not supported).

It could be useful in rebuilding or troubleshooting.

[Kian_Ong_Tan]

May I know where this leads to in the end? I am facing the exactly same scenario here

Thanks. 

Regards. 

KianOng