CPD showing "Terminated" status

[jboco]

Hi,

Is there someone that can help me with this problem? One of our gateways (R77.20) is having an issue with cpd.

When I checked the cpwd_admin list, it is showing that CPD is in T status. I already stopped/start and rebooted the gateway but still no luck.

cpstat showed that we can't established session with AMON and TCP ports 18192 and 18191 aren't listening.

[Expert@NGA-COM01-FWL01:0]# cpwd_admin list

APP        PID    STAT  #START  START_TIME             MON  COMMAND

CPD        0      T     6       [15:04:27] 3/10/2019   N    cpd

[Expert@NGA-COM01-FWL01:0]# cpstat os

Failed to establish session with AMON server at 127.0.0.1:18192

[Expert@NGA-COM01-FWL01:0]# netstat -an | grep 18192

[Expert@NGA-COM01-FWL01:0]# netstat -an | grep 18191

Regards,

J

[Timothy_Hall]

cpwd should attempt to restart cpd every 60 seconds, what type of error messages are being written into $CPDIR/log/cpd.elg?

If the cpd process is dead, SIC to that gateway won't work (policy pushes and logs) and status will show as Disconnected.  Traffic should still pass through the gateway, assuming there are not larger problems on the gateway (such as resource shortages) that are causing cpd to die.

[jboco]

Hi Timothy,

Thanks for helping. I'll provide you the logs I'm seeing.

Also, just an additional questions, should the IPSEC VPN should still be working on the gateway even if the cpd is terminated?

By resource shortages, which filesystems should I particularly look into?

[Timothy_Hall]

Site to site VPNs should still work, unless you are doing a so-called "Intranet" VPN between gateways that are using their SIC certificates to authenticate each other in IKE Phase 1.  cpd being dead could break that scenario, while VPNs using a pre-shared secret for authentication in IKE Phase 1 (like Extranet VPNs to externally managed gateways and interoperable device peers) should be fine.

Edit: If you have rebooted it is unlikely you have a memory shortage, run command df -h to see disk utilization.

[jboco]

Hi Timothy,

I was just able to get the logs from the CPD.elg. We are getting the same logs over and over

[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:38] CPD: Fri Oct 4 09:23:38 2019
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:38] SIC initialization started
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:38] cpsic_init: msg client name = cpd
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:38] cpsic_init: context id = 0
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:38] get_my_sicname_from_registry: This machine's sic name does not exist in registry
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] Initialized sic infrastructure
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] There is no valid SIC certificate on this machine. Cannot use sslca authentication yet
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] Initialized SIC authentication methods
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] CPSIC - Registering for messages. Client name 'cpd' (context id = 0)
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] Waiting for certificate from management ...
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] ---> Entering addon ctor [InstallPolicy, register_InstallPolicyAddon]
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] <--- Exiting addon ctor [InstallPolicy, register_InstallPolicyAddon]
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] ---> Entering addon ctor [FetchPolicy, register_FetchPolicyAddon]
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] <--- Exiting addon ctor [FetchPolicy, register_FetchPolicyAddon]
[CPD 10630 2013091520]@NGA-COM01-FWL01[4 Oct 9:23:39] ---> Entering addon ctor [amon, register_amon_addon]

[jboco]

When I'm doing a telnet 127.0.0.1 18191 and 18192 to the firewall itself. I'm getting connection refused.

[Timothy_Hall]

If that is all you are seeing in your cpd logs, it would appear that SIC needs to be reset on the gateway and reestablished with the SMS.

[jboco]

Hi Timothy,

We were able to fix the CPD being in Terminated status using this SK sk116356.

The SIC was then reestablished on both the SMS and on the gateway but it seems that the firewall encountered issue when we push the policy.

"Load on Module failed - problem with the Commit Function"

We did a fw -d fetch and this is what the logs says.

[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:42] fw_objects_download: Creating splitted objects.C files
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:42] IpcUnMapFile: unmapping file (handle=0xb460960)
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:42] file_diff_compare_data_to_file: returning 1 for file /opt/CPsuite-R77/fw1/database/globals_objects.C
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:42] file_diff_write_data_if_changed: not writing data. Identical to /opt/CPsuite-R77/fw1/database/globals_objects.C
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:42] IpcUnMapFile: unmapping file (handle=0xb460960)
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:42] file_diff_compare_data_to_file: returning 1 for file /opt/CPsuite-R77/fw1/database/setup_objects.C
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:42] file_diff_write_data_if_changed: not writing data. Identical to /opt/CPsuite-R77/fw1/database/setup_objects.C
[ 29563 145394576]@NGA-COM01-FWL01[8 Oct 9:37:42] hash_do_resize: Resizing hash from 16384 to 32768 (n_elements=32768)
[ 29563 145394576]@NGA-COM01-FWL01[8 Oct 9:37:42] hash_do_resize: Resizing hash from 32768 to 65536 (n_elements=65536)
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:43] IpcUnMapFile: unmapping file (handle=0xb460960)
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:43] file_diff_compare_data_to_file: returning 1 for file /opt/CPsuite-R77/fw1/database/netobj_objects.C
[ 29563 2012911312]@NGA-COM01-FWL01[8 Oct 9:37:43] file_diff_write_data_if_changed: not writing data. Identical to /opt/CPsuite-R77/fw1/database/netobj_objects.C