fw ctl zdebug - this is wrong...

[_Val_]

I see just too much of "fw ctl zdebug..." flying around these days. To explain my issue with this practice, I have posted an entry in my blog:

https://checkpoint-master-architect.blogspot.ch/2017/11/kernel-debug-best-practices-or-why-fw.html

Feel free to read and comment

debug‌ kernel_debug best practices

[Moti]

So in theory if there was ‘fw ctl lzdebug’ (“l” for Loukine) macro , what would u have it do ?

[_Val_]

Do not get me wrong, as internal R&D tool, zdebug is just fine. My problem is that you guys do not discourage using it on the field and even post SK articles of HOWTO kind to promote it.

My issues are:

• zdebug is way too simple to use, and it can be dangerous in inexperienced hands
• it is not flexible and does not allow adding or removing flags when running
• most importantly, the buffer is way too limited for live production

Fixing the buffer is no brainer, Tamir could fix it with a blink of an eye. The other two points are a bit more tricky. Ideally, if you really want CP users to run debug in production (which is questionable by itself), do a GUI based tool. Because, if you don't someone else will. Actually, there is already something for the matter: Check Point debugging GUI 

[Tomer_Sole]

Valeri Loukine wrote:

Ideally, if you really want CP users to run debug in production (which is questionable by itself), do a GUI based tool. 

like Packet-Mode search and SmartLog search?

[_Val_]

When I think about it, yes. SmartConsole extension with particular limited kernel debug abilities would be an ideal solution. You can call it ldebug, where L stands for "light"

[Tomer_Sole]

so can we open this discussion to at which cases would you choose to kernel debug rather than search a log or a matched rule? just rules with Track=none or other cases?

[_Val_]

There is not much to discuss. kernel debug should be only employed when other means of troubleshooting, mentioned above included, are exhausted. This is my main personal issue with zdebug: people use it instead of other means to find their config errors. 

However, there are troubleshooting scenarios, where log and policy search may not produce conclusive results, if any. In these cases kernel debug, if performed in educated and controlled manner, might help

[PhoneBoy]

So what you're saying is you're against the idea of putting fw ctl zdebug on a t-shirt

[_Val_]

As part of wider subject, yes. why should check point promote a debug command in the first place, I might ask? Some people say, a good product does not need to be debugged on the field after being released to GA 🙂

[Tomer_Sole]

I'd much rather see this T-Shirt instead

[Hugo_vd_Kooij]

I guess it has more to do with the Geek factor.

[cezar_varlan1]

You are not debugging the product - you are debugging traffic. 
You are not promoting the debug command but the tool to check issues - and having this visibility is what makes a tool good. 

In my opinion you are "apple-ing" all your customers and considering them to be morons that can only use GUI. This is the same approach as thinking that eliminating expert mode is a good idea , etc. (where it's one of the only product that still allows scripting and complex custom customer commands that you can't run on a Forti/Cisco)

Finally why this commands won't show TCP flags and why Check Point considers RFCs are trash and closes established fw sessions at RST/FIN then drops RST+ACK and FIN+ACK as "First Packet isn't SYN" so that you can't properly differentiate between 

[1] Actual Asymmetric routing

[2] Prematurely closed sessions and drops on the follow-up +ACK that would come anyway

[3] Other traffic issues

[4] Actual attacks such as ACK/FIN crafted packets etc.

[Lesley]

You have kicked now couple very old topics. Also this post seems not relevant tbh. If you want to compare TCP flags etc I would recommend tcpdump(cppcap) / Wireshark on both ends, so you can compare. Otherwise there is no point. You can even add interfaces to the Wireshark capture so you can spot asymmetric routing. As far as I know debug commands will be maybe moved to a gui, so still possible but then more accessible. 

[RickHoppe]

I've used zdebug a lot and never got into trouble. It also helped me fixing issues. Perhaps I was lucky.

On the other hand...more often I had to issue specific debug commands (received from TAC) that really did make things worse on production environments.

I understand your concerns about debugging but I guess you need to be carefull with every debug command. With or without 'z' in front of it.

[_Val_]

Absolutely correct, Rick. To execute kernel debug, one needs to understand and appreciate the implications. However, using zdebug shortcut saves you all those efforts of understanding and appreciation.