1550 Appliance unexpected reboots

[BLD]

Hi.

We have had the appliance for a few weeks.

In the past 5 days our notification logs show 3 "unexpected reboot" notices. We have had no power or other issues in our site. How can we get more information to find the cause of these reboots? We have found nothing in the logs. Do logs survive a reboot?

Firmaware version is R80.20 (992000668)

Thanks.

[G_W_Albrecht]

If you use the 1500 as locally managed and without a external Log server, you can only save the Security Logs to a SD card and a snapshot of the System Logs to the flash disk in regular intervals (set it to 30mins). This should give you System Logs for 30mins before the reboot.

And you should consult sk96189    How to debug random reboot issues on 600/700/900/1100/1200R/1400 Locally or Centrally Managed appliance

[HristoGrigorov]

Don't know if it is valid for 1500 series but on 1400 kernel panics are saved in /logs directory and they often give a clue what went wrong. 

Best way however is to involve TAC and let them debug it  for you.

[BLD]

Thanks!

The sk just suggests to change ips mode to something less strict and the additional instructions are not meant for an embedded management unit, it seems.

So for now I will set up an external log server and capture more information.

Regards!

[G_W_Albrecht]

All instructions from sk96189 are meant for SMB - no matter how managed... I would do as shown here to get debug level information that is not generated otherwise! Next step had been installing a debug firmware in older times.

To set up an external log server and capture more information is a good idea, but is limited as it shows all System Log levels except for the debug level..

[BLD]

Right. Thanks for correction!

Will do both then, and get back to this thread when I get some more data.

Regards.

[G_W_Albrecht]

The collected Debug data is meant for CP TAC - so open a ticket and let them look at it...

[BLD]

Hi.

With firmware R80.20.01, when attempting to boot in debug mode as per sk96189, the device freezes up. Only way to recover is manually turning off and back on.  We have reported it to TAC.

Has anybody else experienced this problem?

[HristoGrigorov]

You can invest in one of these until firmware is stabilized 🙂

http://www.elexim.com/producten.php?p=152&SESID=qqruk3i0v89hokv94pvntj7vk4

[Martin_Seeger]

Yes, I have currently one reboot per day (R80.20.05).

My setup is rather complex: Internal network is a Bridge and I have three external networks (3 different ISPs). A lot of the blades are enabled.

[Amir_Ayalon]

Hi Martin,

Thanks for your feedback

We are constantly working to improve the stability and quality of the firmware.

We had a big infrastructure change with the move to R80, and under rare circumstances we had indeed encountered unexpected reboots. R80.20.02 included many fixes, so the majority of the issues were resolved.

in R80.20.05 we again saw a few issues, and we will release this week a new image to fix a these issues.

So, we are committed to deliver a firmware with zero unexpected reboots and will continue to improve the performance, stability and security of the firmware.

thanks again for providing your feedback,

there

[BorisL]

Hello Amir.

Will this new version be available from the device or does it have to be request or downloaded from somewhere?

Thank you.

[Amir_Ayalon]

Hi,

it will be released as any new firmware.

we first release it in an SK, and in parallel start gradual field deployment (the GW popup)

that's why it takes time until all the GW see recommendation for new firmware.

so if you have an issue, you can download it from the SK, and if everything is working properly , you can wait for the GW to "suggest" it.

thanks

[Chris_Atkinson]

Per above Build 992001169 is now available via sk164912.

[Chris_Atkinson]

Note there is also a new firmware version available for the 1500 series, please see sk163612 for details.

[John_Fleming]

I'm noticing the same. Kernel crash info in stored in /logs/.

# ls -l vmcore*
-rw-r--r-- 1 root root 129286 Dec 31 1969 vmcore-dmesg.2019-12-30-11-28-59
-rw-r--r-- 1 root root 130467 Dec 31 1969 vmcore-dmesg.2019-12-31-13-52-16

Im upgrading to R80.20.01 to see what happens. I'll follow up with a ticket if it happens again.

[BorisL]

We upgraded to R80.20.01 firmware. But too soon to tell if unexpected reboots still happen. 

We have also tried to start the 1550 in debug mode as per sk96189 but device freezes up. We have opened a case with CP TAC for this.

[Kevin_Zeitler]

Same issue reboots once a day with panics, upgraded to 80.20.01 hopefully it helps

[BorisL]

Kevin,

We still get the reboots after the update.

[Kevin_Zeitler]

Hi Boris,

Same rebooted eight hours after upgrading.  Problems with 1400, now problems with 1500

[HristoGrigorov]

Developing software for device with such limited resources is not an easy task, believe me. And there are problems that cannot be spotted in a test lab, only in production environment. As we many times discussed it,  rarely there are dedicated SMB admins to report problems to CheckPoint.

There is a very positive response from R&D here in their willing to monitor this thread and engage directly in investigating such a serious problems as unexpected reboots and panics. In the past you had to go through TAC about everything and this was long and sometimes painful process.

Overall, I am confident that 15xx series will have stable builds faster than 14xx ones.

[Dafna]

Hi,

My name is Dafna, I'm team leader in SMB.  

If you still have the "unexpected reboot" notification I would like to connect to tour GW and check it

Thanks,

   Dafna 

[John_Fleming]

So far i've been up for 5 days without issue. Before I was seeing a lot of bad looking things in dmesg. Now i just have a few of the following messages. Besides that everything looks fine.

# dmesg -T | tail -20
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
[Tue Jan 21 20:45:29 2020] [fw4_3];fw_inspect_strings_get: invalid id (1) supplied
 

[BorisL]

Hi Dafna. We have an open ticket with TAC about this so we can coordinate through there. Please write us an email to sistemas@infoe.es and we will go form there. 

[MTS_DGR]

Hello, 

We are facing the same errors, unexpected reboots. Nearly daily and randomly.

This thread does not offer a Solution. Is the problem solved?

If yes, How?

[Dafna]

Hi,

Can you please upgrade your GW to R80.20.01 and if issue occurs again please send me the vmcore-dmesg.* files under /logs

Thanks,

   Dafna

[Kevin_Zeitler]

Hello Dafna,

I upgraded to .01 two days ago and its rebooted twice since then.  Please send me your email and I will send the logs

[BLD]

Hello Dafna,

Please also give us your email or write to us at sistemas@infoe.es with instructions to get those logs and to send them back to you

[ML]

Same 'Unexpected reboots' are happening to our new 1550 using R80.20.01 (992000899)

[BLD]

We have been working with TAC and issue seems to have been resolved with version (992000909), which we have tested for a week.  I think they just released it now.