Is there any one of this obstacle in the Checkup?

[TAEKBOM_Kim]

* Customer environment

- Traffic : 1Gbps

- User : 5000

* Checkup Platform/Version

- SG15600 / Product version Check Point Gaia R80.10 - OS build 462

* The point at issue

- FWD or FWM daemon is stopped within 2 to 3 days of checkup installation

[Expert@Checkup-Demo:0]# cpwd_admin list
APP        PID    STAT  #START  START_TIME             MON  COMMAND            
CPVIEWD    22325  E     1       [15:17:58] 14/6/2018   N    cpviewd            
HISTORYD   22328  E     1       [15:17:58] 14/6/2018   N    cpview_historyd    
CPD        22340  E     1       [15:17:58] 14/6/2018   Y    cpd                
MPDAEMON   22352  E     1       [15:17:59] 14/6/2018   N    mpdaemon /opt/CPshrd-R80/log/mpdaemon.elg /opt/CPshrd-R80/conf/mpdaemon.conf
CI_CLEANUP 22703  E     1       [15:18:06] 14/6/2018   N    avi_del_tmp_files  
CIHS       22705  E     1       [15:18:06] 14/6/2018   N    ci_http_server -j -f /opt/CPsuite-R80/fw1/conf/cihs.conf
FWD        0      T     0       [09:23:45] 18/6/2018   N    fwd                
FWM        22750  E     1       [15:18:07] 14/6/2018   N    fwm                
CPM        22971  E     1       [15:18:09] 14/6/2018   N    /opt/CPsuite-R80/fw1/scripts/cpm.sh -s
....
...
..

[Expert@checkup-demo:0]# cpview
CPView: Failed parsing the conf file
[Expert@checkup-demo:

- FWD, FWM daemon do not run when device is restarted after first failure

[Expert@checkup-demo:0]# ps -aux | grep fw
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ
admin     7387  0.0  0.0   1736   512 pts/2    S+   17:20   0:00 grep fw
admin    20857  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_0]
admin    20858  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_1]
admin    20859  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_2]
admin    20860  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_3]
admin    20861  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_4]
admin    20862  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_5]
admin    20863  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_6]
admin    20864  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_7]
admin    20865  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_8]
admin    20866  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_9]
admin    20867  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_10]
admin    20868  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_11]
admin    20869  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_12]
admin    20870  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_13]
admin    20871  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_14]
admin    20872  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_15]
admin    20873  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_16]
admin    20874  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_17]
admin    20875  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_18]
admin    20876  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_19]
admin    20877  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_20]
admin    20878  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_21]
admin    20879  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_22]
admin    20880  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_23]
admin    20881  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_24]
admin    20882  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_25]
admin    20883  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_26]
admin    20884  0.0  0.0      0     0 ?        S    11:40   0:00 [fw_worker_27]
[Expert@checkup-demo:0]#

- The same symptom occurred again after replacing the equipment.

* Questions

1. Is there any one of this obstacle in the Checkup?

   =>

2.  How to Enable the Failed Equipment to run FWD and FWM Daemons? (It is useless to reboot / cpstart)

   =>

3. How to solve this problem?

   =>

[Kaspars_Zibarts]

Generally this SK is a very good source of information where to look for logs for different processes:

Check Point Processes and Daemons 

From there you would find that fwm log is here $FWDIR/log/fwm.elg and fwd here $FWDIR/log/fwd.elg. Check those logs.

This SK is very helpful to explain how these processes relate to each other and how to debug them

R80.x Security Management server main processes debugging 

Is this a standalone deployment? Management and gateway running on the same box?

[TAEKBOM_Kim]

I will check "Check Point Processes and Daemons " and "R80.x Security Management server main processes debugging"

Yes, It is management and gateway running on the same box.

[Marco_Valenti]

You could you cpwd_admin start command , but the issue seems to be deeper here

> cpwd_admin start -name <process name> -path "<full path>" -command
"<executable name>"
Parameter Description
-name <process name> A name for the process to be watched by WatchDog.
-path "<full path>" The full path to the executable including the executable name
-command "<executable
name>"
The name of the executable file.
Example To start and monitor the fwm process.
> cpwd_admin start -name FWM -path "$FWDIR/bin/fwm" -command "fwm"

[TAEKBOM_Kim]

It doesn't work.

> cpwd_admin start -name FWM -path "$FWDIR/bin/fwm" -command "fwm"

[Expert@checkup-demo:0]# cpwd_admin start -name FWM -path "$FWDIR/bin/fwm" -command "fwm"
 cpwd_admin: Failed to submit request to cpWatchDog
[Expert@checkup-demo:0]# cpview
CPView: Failed parsing the conf file
[Expert@checkup-demo:0]#

[Kaspars_Zibarts]

Check if the watchdog is actually running 

[Expert@fwf1:0]# ps aux | grep cpwd
admin 5241 0.0 0.0 1736 508 pts/2 S+ 07:57 0:00 grep cpwd
admin 20182 0.0 0.0 14544 3416 ? Ss Apr14 2:22 /opt/CPshrd-R80/bin/cpwd

If you don't see it running, try starting it manually simply by running

/opt/CPshrd-R80/bin/cpwd

I'm sure it's not related but we had one really weird case where management server failed to start after reboot as watchdog failed to start. Didn't matter how many times we rebooted it. But then after running watchdog manually, it all fixed "itself". 

[TAEKBOM_Kim]

[Expert@checkup-demo:0]# ps -aux | grep cpwd
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ
admin    29789  0.0  0.0   1736   524 pts/2    S+   15:20   0:00 grep cpwd
[Expert@checkup-demo:0]# /opt/CPshrd-R80/bin/cpwd
[Expert@checkup-demo:0]# /opt/CPshrd-R80/bin/cpwd
[Expert@checkup-demo:0]# /opt/CPshrd-R80/bin/cpwd
[Expert@checkup-demo:0]# ps -aux | grep cpwd
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ
admin    29801  0.0  0.0   1732   520 pts/2    S+   15:20   0:00 grep cpwd
[Expert@checkup-demo:0]# cpwd_admin start -name FWM -path "$FWDIR/bin/fwm" -command "fwm"
 cpwd_admin: Failed to submit request to cpWatchDog
[Expert@checkup-demo:0]# cpview
CPView: Failed parsing the conf file
[Expert@checkup-demo:0]#

After FWD or FWM daemon is stopped within 2 to 3 days of checkup installation.

I tried to reboot the 15600 but both FWD and FWM don't run.

it is weird.

[Marco_Valenti]

This definitely look something for the Tac

[PhoneBoy]

How much memory is in the 15600?

R80.10 Management requires a lot more memory and it will certainly improve performance to have as much as possible.

Especially in a standalone configuration used in Security Checkups.

Also, if you haven't opened a TAC case, I recommend doing so.

[TAEKBOM_Kim]

The 15600 is installed 32GB Memory.

I will open the TAC case.

Thank you for your advice 🙂

[Bruno_Duarte · Accepted solution]

Isn´t that related with the sk 105510?

https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...

I faced a similar issue in 2 security Checkups in the same week,  I saw that there were more people complainning in a mailing list

[TAEKBOM_Kim]

Thank you so much 🙂
It helped me.

You faced a similar issue in 2 security Checkups.
Did you configure a standalone deployment(security management and security gateway running on the same box)?

[Bruno_Duarte]

Yes, it appears it is happenning in standalone deployments (to me, always with take 103), this week it happenned again to a team mate, and the stange it is that the issue re-occurres..