Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
RemoteUser
Advisor

Unusual zdebug Output

Hi everyone,
I hope you're all doing well.

The client recently reported a rather unusual error that I have to admit I've never seen before. After running a zdebug drop on the firewall to monitor dropped packets, the following output was observed:

Has anyone come across something similar?

I'm wondering if this behavior could be related to the fact that SecureXL is running in UPPAK mode. Could this debugging method possibly interfere or produce unexpected results in such a configuration?

fw ctl zdebug + drop | grep x.x.x.x
^C
Next time perform for exit: "fw ctl debug 0"

 

cpdev_user_ioctl: ioctl failed to device /vs0/cp/usim/main
: Interrupted system call
*** Error in `fw': corrupted size vs. prev_size: 0x0879db68 ***
======= Backtrace: =========
/lib/libc.so.6(+0x78eaf)[0xefbaaeaf]
/lib/libc.so.6(__libc_calloc+0xb3)[0xefbad113]
/opt/CPshrd-R81.20/lib/libOS.so(+0x464c6)[0xf06344c6]
/opt/CPshrd-R81.20/lib/libOS.so(+0x46a07)[0xf0634a07]
/opt/CPshrd-R81.20/lib/libOS.so(CpSystemEx+0x161)[0xf0634f31]
/opt/CPshrd-R81.20/lib/libOS.so(CpSystemWithFlags+0x2b)[0xf063513b]
/opt/CPshrd-R81.20/lib/libkiss_apps.so(kiss_debug_command+0x14f7)[0xf1649eb7]
/opt/CPshrd-R81.20/lib/libkiss_apps.so(kiss_zdebug_command+0x426)[0xf164e046]
/opt/CPsuite-R81.20/fw1/lib/libfw1.so(+0x118f3f)[0xf38b3f3f]
/opt/CPsuite-R81.20/fw1/lib/libfw1.so(main_fwctl+0x4b4)[0xf38b7674]
/opt/CPsuite-R81.20/fw1/lib/libfw1.so(fw_cmain+0xccf)[0xf38fca2f]
fw(main+0xa8)[0x8052b48]
/lib/libc.so.6(__libc_start_main+0xde)[0xefb4a3ae]
fw[0x8052be5]
======= Memory map: ========
08048000-08086000 r-xp 00000000 fd:01 17622430                           /opt/CPsuite-R81.20/fw1/bin/fw_full
08086000-08087000 r--p 0003d000 fd:01 17622430                           /opt/CPsuite-R81.20/fw1/bin/fw_full
08087000-08089000 rw-p 0003e000 fd:01 17622430                           /opt/CPsuite-R81.20/fw1/bin/fw_full
08089000-080cc000 rw-p 00000000 00:00 0
085fe000-0884c000 rw-p 00000000 00:00 0                                  [heap]
e9e00000-e9e21000 rw-p 00000000 00:00 0
e9e21000-e9f00000 ---p 00000000 00:00 0
ea000000-ea200000 rw-p 00000000 00:00 0
ea3b8000-ea3c6000 r-xp 00000000 fd:01 2002                               /usr/lib/libnss_files-2.17.so
ea3c6000-ea3c7000 r--p 0000d000 fd:01 2002                               /usr/lib/libnss_files-2.17.so
ea3c7000-ea3c8000 rw-p 0000e000 fd:01 2002                               /usr/lib/libnss_files-2.17.so
ea3c8000-ea3ce000 rw-p 00000000 00:00 0
ea3da000-ea3ec000 rw-p 00000000 00:00 0
ea3ec000-ea3ee000 r-xp 00000000 fd:01 653867                             /opt/CPshrd-R81.20/lib/libsicOverSslCrl.so
ea3ee000-ea3f0000 rw-p 00001000 fd:01 653867                             /opt/CPshrd-R81.20/lib/libsicOverSslCrl.so
ea3f0000-ea3f1000 rw-p 00000000 00:00 0
ea3f1000-ea404000 r-xp 00000000 fd:01 653870                             /opt/CPshrd-R81.20/lib/libsic_conf.so
ea404000-ea406000 rw-p 00012000 fd:01 653870                             /opt/CPshrd-R81.20/lib/libsic_conf.so
ea406000-ea43a000 r-xp 00000000 fd:01 50687953                           /opt/CPsuite-R81.20/fw1/lib/libcpc_compiler.so
ea43a000-ea43c000 rw-p 00033000 fd:01 50687953                           /opt/CPsuite-R81.20/fw1/lib/libcpc_compiler.so
ea43c000-ea44d000 rw-p 00000000 00:00 0
ea44d000-ea451000 r-xp 00000000 fd:01 653868                             /opt/CPshrd-R81.20/lib/libsicOverSslExternalDomains.so
ea451000-ea452000 ---p 00004000 fd:01 653868                             /opt/CPshrd-R81.20/lib/libsicOverSslExternalDomains.so
ea452000-ea454000 rw-p 00004000 fd:01 653868                             /opt/CPshrd-R81.20/lib/libsicOverSslExternalDomains.so
ea454000-ea50e000 r-xp 00000000 fd:01 50687777                           /opt/CPsuite-R81.20/fw1/lib/libCPSet2SQL.so
ea50e000-ea511000 r--p 000b9000 fd:01 50687777                           /opt/CPsuite-R81.20/fw1/lib/libCPSet2SQL.so
ea511000-ea514000 rw-p 000bc000 fd:01 50687777                           /opt/CPsuite-R81.20/fw1/lib/libCPSet2SQL.so
ea514000-ea515000 rw-p 00000000 00:00 0
ea515000-ea521000 r-xp 00000000 fd:01 50687824                           /opt/CPsuite-R81.20/fw1/lib/libExtDAO.so
ea521000-ea522000 ---p 0000c000 fd:01 50687824                           /opt/CPsuite-R81.20/fw1/lib/libExtDAO.so
ea522000-ea524000 rw-p 0000c000 fd:01 50687824                           /opt/CPsuite-R81.20/fw1/lib/libExtDAO.so
ea524000-ea533000 r-xp 00000000 fd:01 50687798                           /opt/CPsuite-R81.20/fw1/lib/libCpmixServerAdapter.so
ea533000-ea535000 rw-p 0000e000 fd:01 50687798                           /opt/CPsuite-R81.20/fw1/lib/libCpmixServerAdapter.so
ea535000-ea590000 r-xp 00000000 fd:01 50688002                           /opt/CPsuite-R81.20/fw1/lib/libcpmix.so
ea590000-ea591000 ---p 0005b000 fd:01 50688002                           /opt/CPsuite-R81.20/fw1/lib/libcpmix.so
ea591000-ea593000 rw-p 0005b000 fd:01 50688002                           /opt/CPsuite-R81.20/fw1/lib/libcpmix.so
ea593000-ea594000 rw-p 00000000 00:00 0
ea594000-ea5dd000 r-xp 00000000 fd:01 653856                             /opt/CPshrd-R81.20/lib/libobjlibclient.so
ea5dd000-ea5de000 r--p 00048000 fd:01 653856                             /opt/CPshrd-R81.20/lib/libobjlibclient.so
ea5de000-ea5e0000 rw-p 00049000 fd:01 653856                             /opt/CPshrd-R81.20/lib/libobjlibclient.so
ea5e0000-ea5ee000 rw-p 00000000 00:00 0
ea5ee000-ea5fa000 r-xp 00000000 fd:01 850539                             /opt/CPshrd-R81.20/lib/libcpopenssh.so
ea5fa000-ea5fc000 rw-p 0000b000 fd:01 850539                             /opt/CPshrd-R81.20/lib/libcpopenssh.so
ea5fc000-ea5fe000 r-xp 00000000 fd:01 603062                             /opt/CPshrd-R81.20/lib/libcpssh_statistics.so
ea5fe000-ea600000 rw-p 00001000 fd:01 603062                             /opt/CPshrd-R81.20/lib/libcpssh_statistics.so
ea600000-ea601000 rw-p 00000000 00:00 0
ea601000-ea62a000 r-xp 00000000 fd:01 653771                             /opt/CPshrd-R81.20/lib/libgsoapcpp.so
ea62a000-ea62c000 rw-p 00028000 fd:01 653771                             /opt/CPshrd-R81.20/lib/libgsoapcpp.so
ea62c000-ea635000 r-xp 00000000 fd:01 653866                             /opt/CPshrd-R81.20/lib/libsicOverSsl.so
ea635000-ea637000 rw-p 00008000 fd:01 653866                             /opt/CPshrd-R81.20/lib/libsicOverSsl.so
ea637000-ea64c000 r-xp 00000000 fd:01 50687961                           /opt/CPsuite-R81.20/fw1/lib/libcpc_subr.so
ea64c000-ea64e000 rw-p 00014000 fd:01 50687961                           /opt/CPsuite-R81.20/fw1/lib/libcpc_subr.so
ea64e000-ea690000 rw-p 00000000 00:00 0
ea690000-ea692000 r-xp 00000000 fd:01 50687958                           /opt/CPsuite-R81.20/fw1/lib/libcpc_interface_usr.so
ea692000-ea694000 rw-p 00001000 fd:01 50687958                           /opt/CPsuite-R81.20/fw1/lib/libcpc_interface_usr.so
ea694000-ea6e0000 r-xp 00000000 fd:01 50687952                           /opt/CPsuite-R81.20/fw1/lib/libcpc_builtins.so
ea6e0000-ea6e3000 rw-p 0004b000 fd:01 50687952                           /opt/CPsuite-R81.20/fw1/lib/libcpc_builtins.so
ea6e3000-ea706000 rw-p 00000000 00:00 0
ea706000-ea70e000 r-xp 00000000 fd:01 50687957                           /opt/CPsuite-R81.20/fw1/lib/libcpc_interface_kvm2.so



Any insight would be appreciated.

Best regards,

0 Kudos
20 Replies
G_W_Albrecht
MVP Silver
MVP Silver

I would suggest to involve CP TAC ! I only found a similar error here: sk182173: Linux Endpoint computer cannot connect to Security Gateway using SSL Network Extender (SNX...

CCSP - CCSE / CCTE / CTPS / CCME / CCSM Elite / SMB Specialist
0 Kudos
RemoteUser
Advisor

Hi @G_W_Albrecht , i've already involved TAC,But I don't understand how this SK can be related to this error

0 Kudos
G_W_Albrecht
MVP Silver
MVP Silver

Because of the error message in SK "corrupted size vs. prev_size". You have *** Error in `fw': corrupted size vs. prev_size: 0x0879db68 *** But i do not think this SK will help as you do not have a SNX issue...

CCSP - CCSE / CCTE / CTPS / CCME / CCSM Elite / SMB Specialist
0 Kudos
RemoteUser
Advisor

I think i found the solution:

PRJ-60852

Security Gateway

In some scenarios, when working with SecureXL in User Mode (UPPAK) and enabling a debug filter, the Security Gateway may crash.

the_rock
MVP Gold
MVP Gold

That definitely sounds the same.

0 Kudos
G_W_Albrecht
MVP Silver
MVP Silver

Did you upgrade to JT 101 already and did that resolve the issue ?

CCSP - CCSE / CCTE / CTPS / CCME / CCSM Elite / SMB Specialist
0 Kudos
Lesley
MVP Gold
MVP Gold

Other commands still work? I think issue is not only with fw ctl zedbug but bigger. Maybe check stuff like cphaprob stat , hcp -r all

-------
If you like this post please give a thumbs up(kudo)! 🙂
0 Kudos
RemoteUser
Advisor

hey buddy,

I found this running hcp:
Result: ERROR |
| |
| Description: This test checks if there are any user mode core dumps and if possible, prints their backtrace. |
| |
| Summary:Found 1 coredump on the machine |
| |
| +-----------------------------------------------------+ |
| | User Mode coredumps | |
| +--------------+------+---------+---------------------+ |
| | Process Name | PID | Size | Creation Date | |
| +==============+======+=========+=====================+ |
| | fw_full | 3265 | 6.46 MB | 2025-05-22 15:23:49 | |
| +--------------+------+---------+---------------------+ |
| |
| Finding: |
| Process fw_full crashed on 2025-05-22 15:23:49 |
| |
| Suggested solutions: |
| - Please contact Check Point Support and provide the file /var/log/dump/usermode/fw_full.3265.core.gz |
| - If you want to see core dumps backtraces, please ask Check Point Support |
| to install GDB under bin folder |
|

0 Kudos
Bob_Zimmerman
MVP Gold
MVP Gold

This is the immediate problem. The process crashed when you ended the debug. You need to open a ticket and send the core dump and a cpinfo to the TAC. They should be able to tell pretty quickly if the core dump is a known issue or not.

0 Kudos
RemoteUser
Advisor

I have already sent everything, I am waiting for a response....

0 Kudos
Chris_Atkinson
MVP Gold CHKP MVP Gold CHKP
MVP Gold CHKP

Can you please share additional context i.e. appliance model and JHF take?

CCSM R77/R80/ELITE
RemoteUser
Advisor

Platform: RS-20-00
Model: Check Point 9200
CPU Model: 13th Gen Intel(R) Core(TM) i3-13100E
CPU Frequency: 3300.000 Mhz
Number of Cores: 8
CPU Hyperthreading: Enabled

R81.20
JHF > 99 + hotfix

0 Kudos
Timothy_Hall
MVP Gold
MVP Gold

What happens if you try just fw ctl zdebug drop without the "+".  The + enables logging of drops in SecureXL/UPPAK.

Gaia 4.18 (R82) Immersion Tips, Tricks, & Best Practices Video Course
Now Available at https://shadowpeak.com/gaia4-18-immersion-course
RemoteUser
Advisor

It seems to be working normally. This firewall uses UPPAK mode, so maybe using the "+" option during debugging could be causing some issues?

0 Kudos
RemoteUser
Advisor

The + in the command fw ctl zdebug drop  does not enable drop logging in SecureXL when operating in  (UPPAK).

According toHow to use the fw ctl zdebug command to view drops on the Security Gateway, the + drop option in fw ctl zdebug enables logging of drop messages from the kernel (INSPECT) and also from PPAK (Post-Processing Acceleration Kernel) not from UPPAK.

Or am i wrong???

0 Kudos
the_rock
MVP Gold
MVP Gold

Hey bro,

Lets take a step back, as they say...what is the actual issue?

Andy

0 Kudos
RemoteUser
Advisor

Hey Buddy
| Finding: |
| Process fw_full crashed on 2025-05-22 15:23:49 |
The real issue is that, as you can see from the output, it looks like the FW module crashed when we tried to run  fw ctl zdebug drop

0 Kudos
the_rock
MVP Gold
MVP Gold

Weird indeed...does reboot help?

Andy

0 Kudos
RemoteUser
Advisor

Hey Buddy
Unfortunately, I don't think a reboot will help, because there's a module, fw_full, that has crashed, although I haven't tried rebooting yet.

0 Kudos
the_rock
MVP Gold
MVP Gold

Personally, if I were you, would do it.

Andy

0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events