Checkpoint Management not communicating with gateways

[creativemind_CP]

Hello All

We have a problem with our checkpoint firewall.

OS : Gaia R80.20

When we try to push a new rule we get an error message “Authentication error [ SIC error no. 147 ] check that peer SIC is configured properly and that the system date and time on the security Management Server and peer are synchronized”.

We did not modify anything as regards the configuration and the time seems correct on all the devices involved (plus ntp is configured). Other than that, the gateways are operating normally for the moment.

For the time being, we do not have access to the Security Management Server GUI, when we click on the  cluster icon, nothing happens. We are afraid that if we re-initialization the communication from the gateway’s, we will not be able to complete the procedure and end up with a bigger problem than the one we are facing at the moment.

We have found that there is a multitude of certificates for “cp_mgmt” on the Management box that were recently created and valid.

However the expiry value is still the same “Mon Jun 12 11:13:34 2023”. As the renewal time is set to “0.75” , the renewal process will take place over and over again.

Is that something that you have encountered before ?

We also found out the trace of a “new_sic_cert.p12” in /opt/CPshrd-R80.20/conf from yesterday. Is that something normal ?

**********************************************************************

[Expert@ZSSAP2-MGMT:0]# ls -halt | grep .p12

-rw-rw---- 1 admin root   3.3K Mar 20 00:58 new_sic_cert.p12

-rw-rw---- 1 admin root   2.7K Jun 25  2019 old_sic_cert.p12

-rw-rw---- 1 admin config 3.3K Jun 25  2019 sic_cert.p12

-r--r----- 1 admin bin    2.4K Sep 20  2018 sic_local_cert.p12

********************************************************************

[Expert@ZSSAP2-MGMT:0]# cpca_client lscert -stat Valid -kind SIC

Operation succeeded. rc=0.

13 certs found.

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 1716   DP = 0

Not_Before: Fri Mar 11 22:13:39 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 9533   DP = 0

Not_Before: Tue Mar 15 00:38:40 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 19867   DP = 0

Not_Before: Thu Mar 17 00:48:40 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 30565   DP = 0

Not_Before: Fri Mar 18 00:53:40 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 30884   DP = 0

Not_Before: Sat Mar 12 00:23:39 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 33557   DP = 0

Not_Before: Sat Mar 19 00:58:40 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 67366   DP = 0

Not_Before: Wed Mar 16 00:43:39 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 85133   DP = 0

Not_Before: Fri Mar 11 23:18:39 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 89587   DP = 0

Not_Before: Sun Mar 13 00:28:39 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=cp_mgmt,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 94770   DP = 0

Not_Before: Mon Mar 14 00:33:39 2022   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=ZSCPM2,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 88487   DP = 0

Not_Before: Sat Oct  5 22:47:23 2019   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=ZSGATE2,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 53035   DP = 0

Not_Before: Wed Jun 26 13:27:13 2019   Not_After: Mon Jun 12 11:13:34 2023

Subject = CN=ZSGATE1,O=Zetes..qp4sad

Status = Valid   Kind = SIC   Serial = 99811   DP = 0

Not_Before: Wed Jun 26 16:14:28 2019   Not_After: Mon Jun 12 11:13:34 2023

[the_rock]

I would start with basics...can you even ping back and forth from mgmt / gateways? If so, then communication is there, so I would run cpwd_admin list fw side, as well as mgmt. There is way to reset SIC without doing cpstop on the firewall, but you would still need to initialize it on mgmt side.

https://korkutozcan.com/how-to-reset-sic-without-restarting-check-point-gw/

Btw, when did this problem happen? Any changes that may had caused it?

Andy

[creativemind_CP]

Hello Andy

I can ping from Management to gateway but not from gateway to management.

We noticed the problem when trying to push a rule yesterday evening (20 March). Before that no issue was reported, everything was running normally.

The change is just an added IP to an existing rule. It should not have that kind of effect I guess.

************************************************************************

Management side

[Expert@ZSSAP2-MGMT:0]# cpwd_admin list

APP        PID    STAT  #START  START_TIME             MON  COMMAND

CPVIEWD    7051   E     1       [11:43:53] 21/3/2022   N    cpviewd

HISTORYD   7054   E     1       [11:43:53] 21/3/2022   N    cpview_historyd

CPD        7063   E     1       [11:43:53] 21/3/2022   Y    cpd

FWD        7168   E     1       [11:43:57] 21/3/2022   N    fwd -n

FWM        7171   E     1       [11:43:57] 21/3/2022   N    fwm

STPR       7181   E     1       [11:43:57] 21/3/2022   N    status_proxy

SOLR       7357   E     1       [11:44:00] 21/3/2022   N    java_solr /opt/CPrt-R80.20/conf/jetty.xml

RFL        7439   E     1       [11:44:01] 21/3/2022   N    LogCore

SMARTVIEW  7458   E     1       [11:44:02] 21/3/2022   N    SmartView

INDEXER    7481   E     1       [11:44:02] 21/3/2022   N    /opt/CPrt-R80.20/log_indexer/log_indexer

CPM        7559   E     1       [11:44:03] 21/3/2022   N    /opt/CPsuite-R80.20/fw1/scripts/cpm.sh -s

SMARTLOG_SERVER 7636   E     1       [11:44:04] 21/3/2022   N    /opt/CPSmartLog-R80.20/smartlog_server

EXPORTER.SIEM_CSIRT 7733   E     1       [11:44:06] 21/3/2022   N    /opt/CPrt-R80.20/log_exporter/targets/SIEM_CSIRT/log_exporter -export /opt/CPrt-R80.20/log_exporter/targets/SIEM_CSIRT/targetConfiguration.xml

DASERVICE  7803   E     1       [11:44:07] 21/3/2022   N    DAService_script

LPD        2921   E     1       [11:47:25] 21/3/2022   N    lpd

CPSM       9442   E     1       [11:47:46] 21/3/2022   N    cpstat_monitor

[Expert@ZSSAP2-MGMT:0]#

**************************************************************

Gateway side

[Expert@ZSGate1:0]# cpwd_admin list

APP        PID    STAT  #START  START_TIME             MON  COMMAND

CPVIEWD    8879   E     1       [13:39:27] 12/7/2021   N    cpviewd

HISTORYD   8882   E     1       [13:39:27] 12/7/2021   N    cpview_historyd

SXL_STATD  8885   E     1       [13:39:27] 12/7/2021   N    sxl_statd

CPD        8892   E     1       [13:39:27] 12/7/2021   Y    cpd

MPDAEMON   8910   E     1       [13:39:29] 12/7/2021   N    mpdaemon /opt/CPshrd-R80.20/log/mpdaemon.elg /opt/CPshrd-R80.20/conf/mpdaemon.conf

CI_CLEANUP 9074   E     1       [13:39:36] 12/7/2021   N    avi_del_tmp_files

CIHS       9076   E     1       [13:39:36] 12/7/2021   N    ci_http_server -j -f /opt/CPsuite-R80.20/fw1/conf/cihs.conf

FWD        9102   E     1       [13:39:37] 12/7/2021   N    fwd

RAD        9503   E     1       [13:39:41] 12/7/2021   N    rad

CPHAMCSET  9988   E     1       [13:40:05] 12/7/2021   N    cphamcset -d

WSDNSD     10035  E     1       [13:40:08] 12/7/2021   Y    wsdnsd

DASERVICE  30660  E     1       [13:39:28] 30/1/2022   N    DAService_script

AUTOUPDATER 17082  E     1       [13:40:37] 12/7/2021   N    AutoUpdaterService.sh

LPD        25950  E     1       [03:08:16] 23/2/2022   N    lpd

[Expert@ZSGate1:0]#

We also have this message regularly in the logs of the management box (cpd.elg )

[CPD 7063 4133869312]@ZSSAP2-MGMT[21 Mar 13:59:14]

Renew_SIC_Cert_cb: CPD failed to renew sic certificate. status = 1, rc - -1.
Renew_SIC_Cert_cb: Cannot renew SIC certificate.
Try to restart all Check Point processes.

[creativemind_CP · Accepted solution]

Hello All,

Below CP article help to resolve the issue.

https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...