cancel
Showing results for 
Search instead for 
Did you mean: 
Post a Question

SFWD process crash

Hi,

has any of you ran into something like this:

[cpWatchDog 2765 1744478736]@RD6281[18 May  9:20:47] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[18 May  9:21:47] [SUCCESS] SFWD started successfully (pid=3551)
[cpWatchDog 2765 1744478736]@RD6281[18 May  9:35:25] [ERROR] Process SFWD terminated abnormally : Unhandled signal 6 (). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[18 May  9:36:25] [SUCCESS] SFWD started successfully (pid=4359)
[cpWatchDog 2765 1744478736]@RD6281[20 May  9:41:20] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[20 May  9:42:20] [SUCCESS] SFWD started successfully (pid=18894)
[cpWatchDog 2765 1744478736]@RD6281[20 May 16:12:09] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[20 May 16:13:09] [SUCCESS] SFWD started successfully (pid=20929)
[cpWatchDog 2765 1744478736]@RD6281[21 May  8:50:27] [ERROR] Process SFWD terminated abnormally : Unhandled signal 6 (). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[21 May  8:51:27] [SUCCESS] SFWD started successfully (pid=25168)
[cpWatchDog 2765 1744478736]@RD6281[21 May  9:14:11] [ERROR] Process SFWD terminated abnormally : Unhandled signal 6 (). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[21 May  9:15:11] [SUCCESS] SFWD started successfully (pid=25918)
[cpWatchDog 2765 1744478736]@RD6281[21 May 10:54:58] [ERROR] Process SFWD terminated abnormally :

Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[21 May 10:55:58] [SUCCESS] SFWD started successfully (pid=27458)
[cpWatchDog 2765 1744478736]@RD6281[21 May 12:35:14] [ERROR] Process SFWD terminated abnormally : Unhandled signal 6 (). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[21 May 12:36:14] [SUCCESS] SFWD started successfully (pid=29000)
[cpWatchDog 2765 1744478736]@RD6281[22 May 14:53:05] [ERROR] Process SFWD terminated abnormally : Unhandled signal 6 (). Core dumped.
[cpWatchDog 2765 1744478736]@RD6281[22 May 14:54:05] [SUCCESS] SFWD started successfully (pid=5887)

17 Replies

Re: SFWD process crash

The old dumping issue - process SFWD needs some time off i think . In the shown timespan it crashes at least twice a day, so i would use USB Medium firmware update to rule out flash issues. This is from /var/log/log/cpwd.elg, what about /var/log/log/sfwd.el* and /var/log/messages ?

0 Kudos

Re: SFWD process crash

Nothing suspicious there. But there is a core dump of the process that I send to R&D for examination.

0 Kudos

Re: SFWD process crash

Yes, that is the thing to do here - as you can not replicate the crashing sfwd issue.

0 Kudos
Admin
Admin

Re: SFWD process crash

Please open a TAC case so we can investigate.

Contact Support | Check Point Software 

0 Kudos

Re: SFWD process crash

Yes, sorry I forgot to mention SR is currently under investigation.

0 Kudos

Re: SFWD process crash

Hello Hristo, how is the investigation going? Did you receive any news yet?

I found out I have 3 appliances in which the issue does not happen and 7 in which it happens. I am opening an SR to try to find out what is special about those 3 and maybe identify what is wrong in the other 7.

Re: SFWD process crash

Hello Pedro,

Sorry for the late reply, I have somehow missed your question.

I opened SR on 18.09.2018 and since then there was not much progress with it. I have provided core dump to TAC but they said they cannot find anything interesting in it and asked for remote session few days ago. So far, nobody contacted me yet about it. In the mean time, situation is still the same and some times it is getting real bad. I mean just look at this:

[cpWatchDog 2158 1744364432]@RD6281[8 Oct 8:23:37] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 8:24:37] [SUCCESS] SFWD started successfully (pid=11666)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 9:52:31] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 9:53:31] [SUCCESS] SFWD started successfully (pid=14153)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 10:23:28] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 10:24:28] [SUCCESS] SFWD started successfully (pid=15008)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 10:28:48] [ERROR] Process SFWD terminated abnormally : Unhandled signal 6 (). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 10:29:48] [SUCCESS] SFWD started successfully (pid=15353)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 10:34:56] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 10:35:56] [SUCCESS] SFWD started successfully (pid=15773)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 10:51:45] [ERROR] Process SFWD terminated abnormally : Unhandled signal 6 (). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 10:52:45] [SUCCESS] SFWD started successfully (pid=16393)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 11:05:51] [ERROR] Process SFWD terminated abnormally : Unhandled signal 6 (). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 11:06:51] [SUCCESS] SFWD started successfully (pid=16965)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 11:12:31] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 11:13:31] [SUCCESS] SFWD started successfully (pid=17364)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 16:42:06] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 16:43:06] [SUCCESS] SFWD started successfully (pid=22701)
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 21:29:02] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744364432]@RD6281[8 Oct 21:30:02] [SUCCESS] SFWD started successfully (pid=23911)

Just prior the crash the system load average will go real high, 3.00 - 5.00 or so. I wonder what is difference between those 3 appliances (where you do not have such issue) and the other 7....

0 Kudos

Re: SFWD process crash

Btw, do you have Site-to-Site VPNs on those 3 appliances? 

0 Kudos

Re: SFWD process crash

Yes, I do have 3 tunnels configured in one of the 3 that work, with Azure and AWS.

Did they send you any builds? I opened a SR and they sent me a new build for 700/1400. Number of crashes were drastically reduced. I can now go a whole day without a crash.

Check with them if there is any update.

Re: SFWD process crash

No, no builds for me yet. Perhaps they want to sort it out for you first and then sent me the same build Smiley Happy 

Do not agree on half-working solution!!! Ask for super-duper-rock-stable build that lasts at least a week (think it is anyway good to reboot these gnomes every weekend).

Re: SFWD process crash

My SR has just been "escalated to Tier 3" (don't know if that is good or bad) and I am now waiting for TAC to contact me. In the mean time there us a new record on the number of crashes today:

[cpWatchDog 2158 1744515984]@RD6281[11 Oct 12:52:38] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744515984]@RD6281[11 Oct 12:56:06] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744515984]@RD6281[11 Oct 13:02:37] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.
[cpWatchDog 2158 1744515984]@RD6281[11 Oct 13:06:55] [ERROR] Process SFWD terminated abnormally : Unhandled signal 11 (SIGSEGV). Core dumped.

0 Kudos

Re: SFWD process crash

Well, don't want to sound over-optimistically here, but R77.20.81 seems to have solved sfwd crashes for me.  


@Pedro: Have you tried it already? Is it the same for you?

Admin
Admin

Re: SFWD process crash

The feedback is always useful even if not 100% definitive 

Re: SFWD process crash

Good to know Hristo,

I did not try this one yet, but I am using the build 451 of R77.20.80, which solved it for me. It probably has the same fix shipped in the new version.

I will try R77.20.81 as soon as possible and leave my feedback here.

0 Kudos

Re: SFWD process crash

Yeah, probably it is just like that. I was just asked by the devs to try latest firmware and see if it fixes the problem for me as well. 

0 Kudos
Highlighted

Re: SFWD process crash

SFWD crashes are now history here. Good job CheckPoint! It took a while to fix it but end result is satisfying. One more step of having more stable SMBs.

Re: SFWD process crash

Surprise!!! Crashed again today. Right after policy install. But I do not think that is the same "region" of code that was causing the crash before. Most impressive was that system load average sky rocketed to ~14 right before it.

I will see if it happens again. Definitely something very specific did it because it is otherwise very stable.

0 Kudos