topic Re: SMS server showing error and can't see policy in Firewall and Security Management

SMS server showing error and can't see policy

Gombodorj — Mon, 12 Dec 2022 09:37:42 GMT

Hello,

I have Smart-1 S600 SMS running R81.10 with take 79. Everything was working perfectly last week. Today I just tried to click on policy on Smart Console, it says "Could not load the selected policy". Also SmartView seems to be not working because I cant monitor the license informations. SMS gaia portal is also not working, not connecting from web browser.

Also, it shows in the logs that bunch of Monitored processes restarting frequently or going down on SMS origin. (Smartview,cpsead etc...)

However, I can SSH into SMS and CPU,RAM,STORAGE usage seems to be perfectly fine. Storage at 50%.

What is the issue?

Re: SMS server showing error and can't see policy

G_W_Albrecht — Mon, 12 Dec 2022 11:59:56 GMT

Tried a reboot yet ?

Re: SMS server showing error and can't see policy

the_rock — Mon, 12 Dec 2022 12:04:54 GMT

I second what @G_W_Albrecht told you, reboot seems best in this case. But, BEFORE you do that, if you can wait a little bit, can you please send output of below commands?

evconfig

top

ps -auxw

free -m

api status

Andy

Re: SMS server showing error and can't see policy

Tal_Paz-Fridman — Mon, 12 Dec 2022 12:05:25 GMT

Hi - this seems to be a problem affecting many important processes.

As G_W_Albrecht suggested, if possible, try rebooting the machine. A few minutes after it comes up, run the following command to see that all the processes are running properly:
cpwd_admin list

If all the processes are up, try repeating the actions you listed and see if everything is working properly.

If not, I suggest opened a ticket with TAC but we could also continue investigating it directly.

Thanks

Re: SMS server showing error and can't see policy

Chris_Atkinson — Mon, 12 Dec 2022 12:34:57 GMT

What troubleshooting has been attempted so far?

cpstop;cpstart etc ...

Re: SMS server showing error and can't see policy

G_W_Albrecht — Mon, 12 Dec 2022 12:36:38 GMT

# Andy

did not work for me 😉

Re: SMS server showing error and can't see policy

the_rock — Mon, 12 Dec 2022 12:39:09 GMT

You lost me there @G_W_Albrecht lol. What did not work?

Re: SMS server showing error and can't see policy

G_W_Albrecht — Mon, 12 Dec 2022 13:01:23 GMT

The last command:

# Andy
-bash: Andy: command not found

😂

Re: SMS server showing error and can't see policy

the_rock — Mon, 12 Dec 2022 13:04:46 GMT

HAHAHAHAHA...I dont know man, worked fine for me!! Its layer 8 problem on your end brother...

Re: SMS server showing error and can't see policy

Gombodorj — Tue, 13 Dec 2022 06:43:00 GMT

I tried rebooting, after that I ran cpwd_admin list and everything seems to be in executing state. Problem still exactly the same. Also tried cpstop, cpstart as well.

Re: SMS server showing error and can't see policy

Gombodorj — Tue, 13 Dec 2022 06:51:03 GMT

Hi, I ran those commands.

Re: SMS server showing error and can't see policy

G_W_Albrecht — Tue, 13 Dec 2022 08:15:08 GMT

Restore last update and redo recent changes ?

Re: SMS server showing error and can't see policy

the_rock — Tue, 13 Dec 2022 11:21:31 GMT

K, can you confirm the following for us please? What is currently the status if you run api status command from expert mode? Also, can you send output of cpwd_admin list?

I actually had a weird issue couple of years ago with customer on R80.30 I believe and their mgmt server one day just "decided" to stop working and no matter what we did, we could never make smart console come up at all, even after working with TAC for few days. Finally, we decided not to spend more time on it and thank God they had working backup and after restore, we just loaded latest jumbo and all worked fine afterwards. I never recall seeing anything about policy not loading, first time I ever seen that was in my R81.20 lab, but it was standalone, which I could not make work after 5 tries.

Re: SMS server showing error and can't see policy

Gombodorj — Tue, 13 Dec 2022 11:36:57 GMT

Here are the results. SMS just decided to not work as you said. Can't click the Security Policies tab so I can't make new policies or even see them.

Re: SMS server showing error and can't see policy

Gombodorj — Tue, 13 Dec 2022 11:37:59 GMT

I do have snapshot and thinking of reverting it, but it's fairly old one so it will be real hassle to do vsx configs again

Re: SMS server showing error and can't see policy

the_rock — Tue, 13 Dec 2022 11:49:35 GMT

Ok, so we can see that cpm and fwm processes are up and running, so thats good, but whats NOT good is that api status shows failed. Can you also run this

cd $FWDIR/scripts

./cpm_status.sh

Send the output of that script please. At this point, I wont waste your time and ask you to try another PC, as I know 100% it wont work, as long as api status shows failed, that has to show successful.

Re: SMS server showing error and can't see policy

G_W_Albrecht — Tue, 13 Dec 2022 11:50:34 GMT

Is TAC involved already ? I would strongly suggest that with VSX managed...

Re: SMS server showing error and can't see policy

Gombodorj — Tue, 13 Dec 2022 11:53:17 GMT

[Expert@mn-dc1-r1c1-sec-fw.sms1:0]# cd $FWDIR/scripts
[Expert@mn-dc1-r1c1-sec-fw.sms1:0]# ./cpm_status.sh
Check Point Security Management Server is running and ready
[Expert@mn-dc1-r1c1-sec-fw.sms1:0]#

this is the output.

Seems like apache in API is not running but I dont know much about api status troubleshooting

Re: SMS server showing error and can't see policy

G_W_Albrecht — Tue, 13 Dec 2022 11:54:11 GMT

Looks fine. TAC needed.

Re: SMS server showing error and can't see policy

the_rock — Tue, 13 Dec 2022 12:05:48 GMT

As @G_W_Albrecht said, that looks good. If I were you, I would pick up the phone and call TAC and work on this right away. There is something seriously wrong here, what it is, Im not sure myself. For any movement here, you need to see api status as successful, so one thing you could try is maybe api restart, see if it does anything.