CPM is not running

[Alex_Wu]

Just upgraded R80.10 to R80.20, found...

CPM is not running.

[Expert@SmartCenter:0]# $MDS_FWDIR/scripts/server_status.sh
Checking server status. Please wait...
17:10:03,889 INFO com.checkpoint.management.cpm.Cpm.enableLocalSic:143 [main] - Enabling local sic. Setting cp.ssl_local.certificate.chec k=local
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /opt/CPsuite-R80.20/fw1/log/cpm.elg (No such file or directory)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:286)
at java.io.FileOutputStream.<init>(FileOutputStream.java:226)
at java.io.FileOutputStream.<init>(FileOutputStream.java:144)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:194)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:544)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:440)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:334)
at org.apache.log4j.PropertyWatchdog.doOnChange(PropertyConfigurator.java:717)
at org.apache.log4j.helpers.FileWatchdog.checkAndConfigure(FileWatchdog.java:89)
at org.apache.log4j.helpers.FileWatchdog.<init>(FileWatchdog.java:58)
at org.apache.log4j.PropertyWatchdog.<init>(PropertyConfigurator.java:709)
at org.apache.log4j.PropertyConfigurator.configureAndWatch(PropertyConfigurator.java:400)
at com.checkpoint.infrastructure.logging.TdLogConfig.setTdLogConfigFilePath(TdLogConfig.java:15)
at com.checkpoint.management.cpm.Cpm.setTdLogConfigFile(Cpm.java:84)
at com.checkpoint.management.cpm.Cpm.main(Cpm.java:110)
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /opt/CPsuite-R80.20/fw1/log/install_policy.elg (No such file or directory)
at java.io.FileOutputStream.open(FileOutputStream.java:286)
at java.io.FileOutputStream.<init>(FileOutputStream.java:226)
at java.io.FileOutputStream.<init>(FileOutputStream.java:144)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:194)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:334)
at org.apache.log4j.PropertyWatchdog.doOnChange(PropertyConfigurator.java:717)
at org.apache.log4j.helpers.FileWatchdog.checkAndConfigure(FileWatchdog.java:89)
at org.apache.log4j.helpers.FileWatchdog.<init>(FileWatchdog.java:58)
at org.apache.log4j.PropertyWatchdog.<init>(PropertyConfigurator.java:709)
at org.apache.log4j.PropertyConfigurator.configureAndWatch(PropertyConfigurator.java:400)
at com.checkpoint.infrastructure.logging.TdLogConfig.setTdLogConfigFilePath(TdLogConfig.java:15)
at com.checkpoint.management.cpm.Cpm.setTdLogConfigFile(Cpm.java:84)
at com.checkpoint.management.cpm.Cpm.main(Cpm.java:110)
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /opt/CPsuite-R80.20/fw1/log/dbsync.elg (No such file or directory)
at java.io.FileOutputStream.open(FileOutputStream.java:286)
at java.io.FileOutputStream.<init>(FileOutputStream.java:226)
at java.io.FileOutputStream.<init>(FileOutputStream.java:144)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:194)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:334)
at org.apache.log4j.PropertyWatchdog.doOnChange(PropertyConfigurator.java:717)
at org.apache.log4j.helpers.FileWatchdog.checkAndConfigure(FileWatchdog.java:89)
at org.apache.log4j.helpers.FileWatchdog.<init>(FileWatchdog.java:58)
at org.apache.log4j.PropertyWatchdog.<init>(PropertyConfigurator.java:709)
at org.apache.log4j.PropertyConfigurator.configureAndWatch(PropertyConfigurator.java:400)
at com.checkpoint.infrastructure.logging.TdLogConfig.setTdLogConfigFilePath(TdLogConfig.java:15)
at com.checkpoint.management.cpm.Cpm.setTdLogConfigFile(Cpm.java:84)
at com.checkpoint.management.cpm.Cpm.main(Cpm.java:110)
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /opt/CPsuite-R80.20/fw1/log/install_policy.elg (No such file or directory)
at java.io.FileOutputStream.open(FileOutputStream.java:286)
at java.io.FileOutputStream.<init>(FileOutputStream.java:226)
at java.io.FileOutputStream.<init>(FileOutputStream.java:144)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:194)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:334)
at org.apache.log4j.PropertyWatchdog.doOnChange(PropertyConfigurator.java:717)
at org.apache.log4j.helpers.FileWatchdog.checkAndConfigure(FileWatchdog.java:89)
at org.apache.log4j.helpers.FileWatchdog.<init>(FileWatchdog.java:58)
at org.apache.log4j.PropertyWatchdog.<init>(PropertyConfigurator.java:709)
at org.apache.log4j.PropertyConfigurator.configureAndWatch(PropertyConfigurator.java:400)
at com.checkpoint.infrastructure.logging.TdLogConfig.setTdLogConfigFilePath(TdLogConfig.java:15)
at com.checkpoint.management.cpm.Cpm.setTdLogConfigFile(Cpm.java:84)
at com.checkpoint.management.cpm.Cpm.main(Cpm.java:110)
Failed to check status, cpm server is probably down
[Expert@SmartCenter:0]#

[Jerry]

try to use CPUSE over the Internet and see if you have got some recent "TAKE" to apply. Do that and when you do the latest (don't remember which one sorry) you should be good to go and make sure you see fwm process in cpwd_admin list.

[Alex_Wu]

I have installed the latest TAKE, but it doesn’t work.

[Jerry]

show us:

1. cpinfo -y all

2.cpwd_admin list

[Alex_Wu]

Many thanks for your help.

1)

[Expert@SmartCenter:0]# cpinfo -y all

This is Check Point CPinfo Build 914000191 for GAIA
Local host is not a Gateway
[CPFC]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 33

[IDA]
No hotfixes..

[MGMT]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 33

[FW1]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 33

FW1 build number:
This is Check Point Security Management Server R80.20 - Build 005
This is Check Point's software version R80.20 - Build 026

[SecurePlatform]
HOTFIX_GOGO_LT_HALO_JHF Take: 33

[CPinfo]
No hotfixes..

[DIAG]
No hotfixes..

[PPACK]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 33

[CVPN]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 33

[SmartLog]
No hotfixes..

[Reporting Module]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 33

[CPuepm]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 33

[VSEC]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 33

[R7520CMP]
No hotfixes..

[R7540CMP]
No hotfixes..

[R76CMP]
No hotfixes..

[SFWR77CMP]
No hotfixes..

[R77CMP]
HOTFIX_R80_20_JHF_COMP Take: 33

[R75CMP]
No hotfixes..

[NGXCMP]
No hotfixes..

[EdgeCmp]
No hotfixes..

[SFWCMP]
No hotfixes..

[FLICMP]
No hotfixes..

[SFWR75CMP]
No hotfixes..

[MGMTAPI]
No hotfixes..

[CPUpdates]
BUNDLE_R80_20_JUMBO_HF_MAIN_SC Take: 47
BUNDLE_CPINFO Take: 0
BUNDLE_R80_20_JUMBO_HF_MAIN_gogoKernel Take: 33

2)

[Expert@SmartCenter:0]# cpwd_admin list
cpwd_admin: Failed to submit request to cpWatchDog

[Jerry]

you have got definitely something wrong Alex, 2nd output says it all. your cpWatchDog service failed to start.

does it happen all the timeyou boot the SMS? can you try to re-install take 33 and see if that fixes an issue?

as @Mark mentioned earlier can you do pwd_admin list instead of cpwd_admin list and see if your watchdog failes again? if it does look out for ps aux | grep fwm - does it have a pid compared to cpm or cpd ?

[Alex_Wu]

I have re-installed Take 33, it does not work.

[Mark_Mitchell]

It may be an idea to get a ticket raised with TAC whilst we are trying to assist. It may be possible that they have seen this exact behaviour and a "known Issue" to them. 

Regards

Mark

[Mark_Mitchell]

Hi Alex, 

Have you attempted to start the relevant CP services? cpstart?

If not as Jerry Szpinak‌ mentions, please provide output to "pwd_admin list" to enable diagnosing further.

Also, the below thread may help. 

https://community.checkpoint.com/thread/8054-cpwdadmin-list-overview-sms 

Regards

Mark

[Alex_Wu]

don't know why some files not found...

[Expert@SmartCenter:0]# cpstart
/opt/CPshrd-R80.20/bin/filesign: Error opening log file /opt/CPsuite-R80.20/fw1/log/filesign.elg
filesign: Manifest at /opt/CPsuite-R80.20/fw1/conf/cpmanifest.signed was properly signed at Tue May 17 16:56:53 2016.
filesign: $FWDIR/modules/fw_kern.o :
XXX File not found.

filesign: $FWDIR/modules/fw_kern_v6.o :
XXX File not found.

filesign: $PPKDIR/boot/modules/sim_kern.o :
XXX File not found.

filesign: $PPKDIR/boot/modules/sim_kern_v6.o :
XXX File not found.

filesign: All 20 files listed in the manifest were properly signed.
SVN Foundation: Starting cpWatchDog
Failed to start CPWD is not responding. Aborting.
FireWall-1: Unable to find CpWatchDog - run cpstart
SmartView Monitor: Not active
evstart: Unable to find CpWatchDog - run cpstart
UEPM: Endpoint Security Management isn't activated and will not be started
Can opening TdError log file /opt/CPcvpn-R80.20/log/CvpndAdmin.log: No such file or directory
Mobile Access service is disabled.
If you wish to start Mobile Access, please enable Mobile Access blade configure its policy.
Starting DAService...
cpstart: Power-Up self tests passed successfully

cpstart: Starting product - SVN Foundation

cpstart: Starting product - VPN-1

cpstart: Starting product - SmartView Monitor

cpstart: Starting product - Eventia Suite

cpstart: Starting product - UEPM

cpstart: Starting product - Mobile Access

cpstart: Starting product - Deployment Agent

[Expert@SmartCenter:0]#

[Mark_Mitchell]

Hi Alex, 

Thanks for the output. 

SK &apos;cpstart&apos; command does not start Check Point services  may be of some interest to you. Some of the symptoms seems to be what you are experiencing. 

Take a look and let us know if that helps. 

It may be safer to get some support from TAC though. 

Regards

Mark

[Jerry]

"Failed to start CPWD is not responding. Aborting."

for me it looks you've got a broken installation of CPM/FWM.

as Mark mentioned I would have rise that with TAC asap alternativally migrate db to the new installation or SMS (VM?).

unless this is a physical smart-1 appliance I would not hesitate a minute to make this work commencing asap.

[Alex_Wu]

Thanks.

I tried to revert to r80.10 with "snapshot management", but it still does not work.

The question is r80.10 worked well before upgrading, why it can not work any more? 

[Jerry]

create SR with TAC asap. seems odd indeed but I guess you're unable to bring the SMS back to live which is more than worrying in every corporate scenario (unless that is the LAB environment)

so unless this isn't the "supported platform" and licensed properly I'd not hesitate a minute and make the production SMS looked after by TAC/Diamond Support Team.

[Mark_Mitchell]

This is not a normal behaviour. I think at the minute your only option to recover the environment is to raise the TAC case. 

Something has gone seriously wrong with the installation. At this stage trying too many different things to try and fix it, may inadvertently make the situation worse. 

I know it doesn't really help now but for me completing any activity like this, I will also run the pre-upgrade adviser + migrate export to the target version as a test within a VM, this is a good test to make sure that there are no issues with the database and a lot of time can highlight things to be fixed before attempting the upgrade. 

Having a recovery plan also, will help revert in the event of a worst case scenario and you need to rebuild your management server. (i.e import snapshot, etc). 

Regards

Mark 

[Alex_Wu]

thanks for your comments.

actually, I am testing the upgrade on a VM. I would like to ask TAC for help.

[Timothy_Hall]

Core and memory specifications for the test VM?  My guess is way too low...

--
"IPS Immersion Training" Self-paced Video Class
Now Available at http://www.maxpowerfirewalls.com

[Alex_Wu]

2 CPUs, 8GB memory

[Timothy_Hall]

2 CPUs does not meet minimum requirements for an R80.10 SMS, increase to 4 cores and try again.  8GB memory meets the minimum, but a bit more certainly wouldn't hurt.  Not meeting these minimums will break stuff, there have already been numerous threads about this.

[Alessandro_Marr]

Alex, did you use what version of image (.ISO) to install?

[Alex_Wu]

I used CPUSE and found R80.20, I did a upgrade not clean install.

[Alex_Wu]

I finally fixed the issue.

below is my step:

1) revert to R80.10 using snapshot management, though it would not work.

2) according to "[Expert@SmartCenter:0]# cpstart
/opt/CPshrd-R80.20/bin/filesign: Error opening log file /opt/CPsuite-R80.20/fw1/log/filesign.elg"

    and 

Can opening TdError log file /opt/CPcvpn-R80.20/log/CvpndAdmin.log: No such file or directory

I checked the related directory, found log folder wasn't there, so I manually create log folder (it's empty)

3) then I tried to upgrade to R80.20 using CPUSE again, though it failed again, BUT, the installation fixed the corrupted files and services. 

4) with R80.10, I successfully upgraded to R80.20.

[Alessandro_Marr]

Alex, I suggest you to upgrade your SMS to R80.20 M2...

[idragoev]

Hello Folks, 

Is there official bug or something regarding this behavior. I have two management servers in HA and Primary one out of nowhere stopped syncing with standby unit. Login to Smart console is timing out and when I logged over ssh I'm getting the exact same outputs.

I'm running 80.10: 

This is Check Point CPinfo Build 914000191 for GAIA
[FW1]
HOTFIX_R80_JUMBO_HF Take: 76
HOTFIX_R80_10
HOTFIX_R80_10_JUMBO_HF Take: 225

FW1 build number:
This is Check Point Security Management Server R80.10 - Build 043
This is Check Point's software version R80.10 - Build 190

[SecurePlatform]
HOTFIX_R80_10
HOTFIX_R80_10_JUMBO_HF Take: 225

[CPinfo]
No hotfixes..

[SmartPortal]
No hotfixes..

[Reporting Module]
HOTFIX_R80_JUMBO_HF Take: 76
HOTFIX_R80_10
HOTFIX_R80_10_JUMBO_HF Take: 225

[CPuepm]
HOTFIX_R80_JUMBO_HF Take: 76
HOTFIX_R80_10

[SmartLog]
HOTFIX_R80_JUMBO_HF Take: 76
HOTFIX_R80_10
HOTFIX_R80_10_JUMBO_HF Take: 225

[MGMTAPI]
No hotfixes..

[R7520CMP]
HOTFIX_R80_10

[R7540CMP]
HOTFIX_R80_10

[R7540VSCMP]
HOTFIX_R80_10

[R76CMP]
HOTFIX_R80_10

[SFWR77CMP]
HOTFIX_R80_JUMBO_COMP
HOTFIX_R80_10
HOTFIX_R80_10_JHF_COMP

[R77CMP]
HOTFIX_R80_JUMBO_COMP
HOTFIX_R80_10
HOTFIX_R80_10_JHF_COMP

[R75CMP]
HOTFIX_R80_10

[NGXCMP]
HOTFIX_R80_10

[EdgeCmp]
HOTFIX_R80_10

[SFWCMP]
HOTFIX_R80_10

[FLICMP]
HOTFIX_R80_10

[SFWR75CMP]
HOTFIX_R80_10

[CPUpdates]
BUNDLE_R80_JUMBO_HF Take: 76
BUNDLE_R80_JHF_T76_SDB Take: 4
BUNDLE_R80_10_T462
BUNDLE_CPINFO Take: T42
BUNDLE_R80_10_JUMBO_HF Take: 225
BUNDLE_R80_10_JUMBO_HF_SC Take: 144

[DIAG]
No hotfixes..

[CPFC]
HOTFIX_R80_10
HOTFIX_R80_10_JUMBO_HF Take: 225

[IDA]
HOTFIX_R80_10

[VSEC]
HOTFIX_R80_10
HOTFIX_R80_10_JUMBO_HF Take: 225

Can I fix this without upgrading? Will reboot help, even temporary, and I assume that my standby device can go off at any time as well.

Regards, 

Ilian