Hello Checkmates,
I am upgrading a Check Point Management Server from R80.20 to R80.30
Everything works fine during upgrade. The Webui is restarted
But we can't connect to the Management Server. Turns out that CPM has not initialized properly.
[Expert@DCTSMS:0]# /opt/CPsuite-R80.30/fw1/scripts/cpm_status.sh
Check Point Security Management Server is during initialization
We see that in the $FWDIR/log/cpm.elg file, that there are several logs worth investigating.
One of them :
ERROR fts.solr.Jpa2SolrManagerImpl [main]: SOLR is completely out of sync!!! more than 5000 jpa2FtsRecords are out of sync.
... leads us to sk116014 : CPM process initialization is slow after backup restore
But this time, it's not slow, it's super slow.
3 hours and no progress (of the size of the cpm.elg file).
We find that in this file, there are lines like :
Caused by: CpmGeneralException{base='com.checkpoint.management.is.exceptions.CpmGeneralException: java.lang.SecurityException: Tried to open non existing session with id d16200d0-e68e-42b5-ad37-1a4da8f3b5de', errorCode='CP_ERR_UNSPECIFIED', errorFamily='null', messageForUser='null', message='java.lang.SecurityException: Tried to open non existing session with id d16200d0-e68e-42b5-ad37-1a4da8f3b5de'}
at com.checkpoint.management.object_store.fts.solr.Jpa2SolrManagerImpl.syncJpaDbWithFtsIndex(Jpa2SolrManagerImpl.java:688)
at com.checkpoint.management.object_store.ObjectStoreSessionImpl.syncJpaDbWithFtsIndex_aroundBody194(ObjectStoreSessionImpl.java:3600)
at com.checkpoint.management.object_store.ObjectStoreSessionImpl$AjcClosure195.run(ObjectStoreSessionImpl.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at com.checkpoint.management.dleserver.coresvc.internal.TransactionRetrySvcImpl.proceed(TransactionRetrySvcImpl.java:79)
at com.checkpoint.management.dle.aspects.TransactionRetryAspect.aroundOperation(TransactionRetryAspect.java:7)
at com.checkpoint.management.object_store.ObjectStoreSessionImpl.syncJpaDbWithFtsIndex(ObjectStoreSessionImpl.java:2500)
at com.checkpoint.management.object_store.ObjectStoreImpl.syncJpaDbWithFtsIndex_aroundBody14(ObjectStoreImpl.java:56)
at com.checkpoint.management.object_store.ObjectStoreImpl$AjcClosure15.run(ObjectStoreImpl.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at com.checkpoint.management.dleserver.coresvc.internal.TransactionRetrySvcImpl.proceed(TransactionRetrySvcImpl.java:79)
at com.checkpoint.management.dle.aspects.TransactionRetryAspect.aroundOperation(TransactionRetryAspect.java:7)
at com.checkpoint.management.object_store.ObjectStoreImpl.syncJpaDbWithFtsIndex(ObjectStoreImpl.java:83)
... 32 more
Caused by: java.lang.SecurityException: Tried to open non existing session with id d16200d0-e68e-42b5-ad37-1a4da8f3b5de
at com.checkpoint.management.object_store.ObjectStoreSessionImpl.isPublished_aroundBody192(ObjectStoreSessionImpl.java:542)
at com.checkpoint.management.object_store.ObjectStoreSessionImpl$AjcClosure193.run(ObjectStoreSessionImpl.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at com.checkpoint.management.dleserver.coresvc.internal.TransactionRetrySvcImpl.proceed(TransactionRetrySvcImpl.java:79)
at com.checkpoint.management.dle.aspects.TransactionRetryAspect.aroundOperation(TransactionRetryAspect.java:7)
at com.checkpoint.management.object_store.ObjectStoreSessionImpl.isPublished(ObjectStoreSessionImpl.java:1010)
at com.checkpoint.management.object_store.fts.solr.Jpa2SolrManagerImpl.syncJpaDbWithFtsIndex(Jpa2SolrManagerImpl.java:304)
========================
So it seems that session ID d16200d0-e68e-42b5-ad37-1a4da8f3b5de is non existent and causing problems regarding CPM initialization.
I try to suppress this session ID using the method I have seen on one of the forums :
mgmt_cli discard --port 443 uid d16200d0-e68e-42b5-ad37-1a4da8f3b5de
Username: sc-admin
Password:
code: "generic_server_error"
message: "Management server failed to execute command"
============================================================
It doesn't work.
Meanwhile, I have noticed that, indeed, there is a ghost session in the Smartcenter that we can't suppress using Smartconsole (or even GUIDBedit). See attached file.
I have tried to remove ghost session using the psql_client command... But I don't know how to proceed.
Any help ?
Thanks,
Gilles
