Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Sahil_Qos
Explorer

CPD-process going high on active member after up-gradation of a normal cluster

Hi all, We have recently upgraded a cluster to GAIA R80.10, but after updgradation CPD process is going high on the active member causing disconnection with the manager and we are not able to push policy.

0 Kudos
5 Replies
Marco_Valenti
Advisor

A bit more of info on your environment would be nice , do you have already installed the jhf take 70 for r80.10 ?

0 Kudos
Sahil_Qos
Explorer

Hi Marco,

JHF take_70 already installed on the Gateways.

0 Kudos
Timothy_Hall
Champion
Champion

Is the cpd process just consuming large amounts of CPU or is it constantly crashing and restarting?  Use cpwd_admin list to determine the latter, and take a look in $CPDIR/log/cpd.elg to figure out the former.

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
Sahil_Qos
Explorer

Hi Tim,

the cpd process is consuming more than 95% of CPU in production hours and fluctuating in lean-hours, no core-dumps generated. Even raised a SR with TAC for this issue. Followed some sk's as well as per the error-message in cpd.elg but still no luck.

0 Kudos
Timothy_Hall
Champion
Champion

Probably best to involve Check Point TAC to help diagnose this, but I did cover how to dig into the behavior of runaway or misbehaving processes in my book, here is a bit of it:

For certain processes there is a log file available which may prove helpful if the
process is logging any errors:

  • fwd: $FWDIR/log/fwd.elg
  • fwm: $FWDIR/log/fwm.elg
  • cpm: $FWDIR/log/cpm.elg
  • vpnd: $FWDIR/log/vpnd.elg
  • cpd: $CPDIR/log/cpd.elg


For example to look at the latest entries in the fwd process log, type tail
$FWDIR/log/fwd.elg. Further debugging of runaway or misbehaving processes can
be accomplished with these commands:

  • fw debug fwd on|off (Debugs go to $FWDIR/log/fwd.elg)
  • fw debug fwm on|off (Debugs go to $FWDIR/log/fwm.elg)
  • $FWDIR/scripts/cpm_debug.sh -t <TOPIC> -s <SEVERITY>
  • vpn debug on|off (Debugs go to $FWDIR/log/vpnd.elg)
  • cpd_admin debug on|off (Debugs go to $CPDIR/log/cpd.elg)


If you enable debug on a process, make sure to turn it off when done! To identify the
function of these or any other daemon process names that may be encountered, consult
sk97638: Check Point Processes and Daemons.


But suppose you see a strange-looking process hogging CPU in the top output:
Figure 4-7: A Process Consuming Excessive CPU


Huh? Why is the cat command eating so much CPU and what process launched this
beast? The built-in Gaia tool pstree can help give us the answer:
Figure 4-8: Using pstree to Help Identify a CPU-Hogging Process

As it turns out this process is related to the scrubd process which sk97638 lists as
part of the Threat Extraction blade feature. Using pstree can help us determine the
origin of a runaway process when the name of the process itself (or its arguments) is not
enough to determine its purpose and whether we should do something about it.


Another command that can provide some insight for a mystery process consuming
excessive CPU is lsof (LiSt Open Files). Once the process ID has been determined
(15215 in our example) run lsof -p 15215. The output will show all files currently
being accessed by the process, along with all open network connections and TCP/UDP
ports the process is currently listening on (if any).


A final option that is guaranteed to satisfy even the most voracious seeker of
information about a pesky process, is to install the strace binary (similar to the truss
command on Solaris) and dynamically attach it to the problematic process with the –p
option. This allows real-time monitoring of all system calls made by the process in an
attempt to determine what it is trying to accomplish. strace is not included by default
with Gaia (and its use is not officially supported) but you can install the precompiled
CentOS version of strace available here:


http://vault.centos.org/3.8/os/i386/RedHat/RPMS/strace-4.5.14-0.EL3.1.i386.rpm

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events