R80.20 issue with fw monitor - all the buffers are full

[HeikoAnkenbrand]

Under R80.20 with the latest jumbo hotfix the following error occurs if I start fw monitor:

fwmonitor_kiss_add_to_global_buf: all the buffers are full

System 12000 appliance:

- 8 coreXl instances

- approximatly 25000 connections

- enough free memory

[kobil]

is this R80.20SP?

[HeikoAnkenbrand]

R80.20 on 12K appliance.

[Chris_Wilson]

I am getting the same issue when I do a fw ctl zdebug drop.

@;1709167040;[cpu_0];[SIM-206652832];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;

@;1709167040;[cpu_0];[SIM-206652832];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;

@;1709167040;[cpu_0];[SIM-206652832];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;

@;1709167040;[cpu_0];[SIM-206652832];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;

@;1709167041;[cpu_0];[SIM-206652832];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;

@;1709167041;[cpu_0];[SIM-206652832];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;

[HeikoAnkenbrand]

Hi @kobil,

any news in this case?

[Andreas_Aust]

same here R80.20 latest Jumbo

[Thomas_Eichelbu]

Hello, 

yes i also got in on R80.20SP with Take 50 ...

[1_03]@;1417195069;[cpu_0];[SIM-206251208];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;
[1_03]@;1417195269;[cpu_0];[SIM-206251208];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;
[1_03]@;1417195569;[cpu_0];[SIM-206251208];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;
[1_01]@;1472195669;[cpu_4];[SIM-206976200];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;
[1_01]@;1472195969;[cpu_4];[SIM-206976200];fwmonitor_kiss_add_to_global_buf: all the buffers are full ;

is this issue for TAC case ... normal operation with this debugs is not really possible ...

best regards.
Thomas.

[Thomas_Eichelbu]

hello, 

i also see this sort of messages all over /var/log/messages:

Sep 19 11:31:19 2019 NWATSBGFWCL01-ch01-04 last message repeated 439284 times
Sep 19 11:31:33 2019 NWATSBGFWCL01-ch01-04 last message repeated 199578 times
Sep 19 11:31:33 2019 NWATSBGFWCL01-ch01-04 cpviewd: Destroying the lists of sensors
Sep 19 11:31:33 2019 NWATSBGFWCL01-ch01-04 kernel: [SIM4];fwmonitor_kiss_add_to_global_buf: all the buffers are full
Sep 19 11:31:48 2019 NWATSBGFWCL01-ch01-04 last message repeated 221176 times
Sep 19 11:31:48 2019 NWATSBGFWCL01-ch01-04 cpviewd: Destroying the lists of sensors
Sep 19 11:31:48 2019 NWATSBGFWCL01-ch01-04 kernel: [SIM4];fwmonitor_kiss_add_to_global_buf: all the buffers are full
Sep 19 11:32:19 2019 NWATSBGFWCL01-ch01-04 last message repeated 366866 times
Sep 19 11:32:33 2019 NWATSBGFWCL01-ch01-04 last message repeated 152620 times
Sep 19 11:32:33 2019 NWATSBGFWCL01-ch01-04 cpviewd: Destroying the lists of sensors
Sep 19 11:32:33 2019 NWATSBGFWCL01-ch01-04 kernel: [SIM4];fwmonitor_kiss_add_to_global_buf: all the buffers are full
Sep 19 11:32:48 2019 NWATSBGFWCL01-ch01-04 last message repeated 168848 times
Sep 19 11:32:48 2019 NWATSBGFWCL01-ch01-04 cpviewd: Destroying the lists of sensors
Sep 19 11:32:48 2019 NWATSBGFWCL01-ch01-04 kernel: [SIM4];fwmonitor_kiss_add_to_global_buf: all the buffers are full
Sep 19 11:33:19 2019 NWATSBGFWCL01-ch01-04 last message repeated 324457 times

maybe this SK helps out:
"/var/log/messages shows 'Log buffer is full'
sk52100
i will try this in a maintenance window!

best regards
Thomas

[Chris_Wilson]

try running this command.  Fixed it for me.

run   fw monitor -U on the gateway

[3315cd11-6366-4]

I also have this issue, can you tell me what the -U switch does before I try it?

thanks

Chris

[Chen_Muchtar]

This is a known issue,  affecting fw monitor only, errors can be ignored

We will make sure relevant fix is to be included in future  JHF

[Chen_Muchtar]

A fix was released as part of R80.20 JHF Take 117