How to check debug command ?

[Sarm_Chanatip]

Hi Everyone,

I have been challenging from customer, they would like to know if there is any commands to check whether any debugging command is running at a time so that they will able to stop those command right away in case of some admins system or TAC forget to turn it off after running debug.

As my understanding from Top commands would be possible but not quite sure.

Really appreciate every comments

Regards,

Sarm

[G_W_Albrecht]

There is not "one" debug - starting from the kernel debug (see sk98799: Kernel Debug) over VPN debug (sk63560 - How to run complete VPN debug on Security Gateway to troubleshoot VPN issues?.) to various daemons (sk86321: How to debug FWD daemon, sk86320: How to debug CPD daemon, skI2821: How to debug RTM daemon, sk31404: How to Debug SecureXL, sk43443: How to debug CoreXL) and, of course sk112334: How to debug SmartConsole / SmartDashboard, but also see sk97443: Support Debug Tools.

[Sarm_Chanatip]

Hi Günther W. Albrecht,

Thanks for sharing, but I would like to get the commands that can display lists of debugging is running at that time.

Regards,

Sarm

[G_W_Albrecht]

Sorry, but i find that this is going nowhere fast ! Every Debug has to be planned in detail and scheduled, and every production gateway / SMS needs a maintenance window for debugs (maybe excluding e.g. policy install debugs). If you can imagine a situation with multiple administrators connected to the same device and one is debugging VPN and second one debugs CoreXL, you are not in the security business but purely into show business 😉

[Petr_Hantak]

You are right about service window no doubt. You should have it always for debugs. If you have service window, then you should notice anyone else about it, steps should be reviewed annd correct and so on. But what if somebody ends debug incorrectly no matter on reason? For example 

[Expert@HostName]# vpn debug trunc
[Expert@HostName]# vpn debug on TDERROR_ALL_ALL=5

runs on background until you turn it off properly.

I know it is pure theory but it could be possible. 

Chanatip Adisaktrakool‌ I think you should try to explain customer that situation shouldn't occur. I have experience that TAC is very careful with debugs and they are alway ending it and it is responsibility of administrator to run and end debug properly in agreed service window like Günther W. Albrecht‌ wrotes above.

[G_W_Albrecht]

I know that this can occur - but the question is about a command showing any debugs currently configured / running, and there is no such command. Except top, as a daemon under debug will need much more ressources 😉

[Petr_Hantak]

Yeah even it is tricky that we have so many debug types, it is still relevant question. I can imagine situation when you have multiple administrators connected to the same device and one is debugging VPN for example and second one reacting to some monitoring event for CPU and wants to debug CoreXL for example. In case both runs debugs in the same time, they could easilly kill the device just because they don't know about each other. 

[Sarm_Chanatip]

Hi Petr,

Yeah, you're right

Regards,

Sarm

[Timothy_Hall · Accepted solution]

In general there are two primary areas of debugging: Process Space and Kernel Space.  An SMS will only have debugs available in Process Space, while a gateway can have debugging active in Kernel Space and/or Process Space.  Kernel Space debugs are far more likely to cause gateway performance or stability effects if they get into a runaway state.

For kernel debugs you can see what debug flags are currently set with these commands: 

fw ctl debug

sim dbg list (R80.20+ - fwaccel dbg list)

To reset kernel debugs to default:

fw ctl debug 0

sim dbg resetall (R80.20+ - fwaccel dbg resetall)

Note: for SecureXL debugs (sim/fwaccel) it is extremely important to set a very specific filter with the -f option or the chances of cratering the system with a runaway debug are very high.

Because there are so many different tools and techniques for initiating Process Space debugs, figuring out if one is active is much more difficult.  Probably the best approach would be to run these commands and look for *.elg files rapidly increasing in size where the process debug files are typically written, and whether they are quickly being rotated (i.e. fwd.elg, fwd.elg.1, fwd.elg.2):

watch ls -ltr $FWDIR/log/*.elg

watch ls -ltr $CPDIR/log/*.elg

--
"IPS Immersion Training" Self-paced Video Class
Now Available at http://www.maxpowerfirewalls.com

[Matlu]

Hello,

I have a problem interpreting the following SK:
https://support.checkpoint.com/results/sk/sk171805

I have a problem with traffic blocked at 1 IP, because of the "HTTP Format Size" feature.

The SK invites me to "check" the "Kernel Debug", but I don't understand how I can "read" this debug?

I have applied the command that the SK says, directly on my GW, and the user generated traffic, but I didn't "get" any result.

[Expert@FW:0]# fw ctl debug -m WS all
Updated debug variable for module WS
[Expert@FW:0]# fw ctl debug -m all
Expert@FW:0]# [Expert@FW:0]#

What I want is a debug, that lets me know, how much is the size of the header in bytes, of the page that the IP of my LAN is trying to consume, to know, if I should or not increase the threshold of the "HTTP Format Size".

Could someone give me some guidance, please?

[PhoneBoy]

You’ve enabled the debug flags (likely) but have not issued the command to see the messages.
It’s fw ctl kdebug with some options (depending on where you want the messages to go): https://support.checkpoint.com/results/sk/sk98799

[Matlu]

Hello.

Applying these debugs on the GW, can "Impact" on the resources of it (CPU, Memory)?

What I would like is a filter that allows me to see the behavior, for example from my IP 172.16.30.10 to the URL "outlook.office365.com".

Is there any recommendation, to apply the filter that allows me to see the size of the header of that URL?

Regards.

[PhoneBoy]

Depending on the debugs enabled, yes, there can be impacts to CPU and Memory.
We always recommend taking these debugs in maintenance windows where possible.
You can filter debug based on IP address: https://sc1.checkpoint.com/documents/R81.10/WebAdminGuides/EN/CP_R81.10_Quantum_SecurityGateway_Guid... 
I'm guessing, based on what you're asking for, you need to enable debugs in the WS module (cookie and parser seem most promising):
https://sc1.checkpoint.com/documents/R81.10/WebAdminGuides/EN/CP_R81.10_Quantum_SecurityGateway_Guid... 

Note the above are version specific, so you may need to consult the relevant guide for your version.
You should also confirm the correct debug flags with TAC.

[Matlu]

Hello @Timothy_Hall 

Is there a debug ‘focused’ on web traffic?

For example if I want to observe the behavior of internal IP 10.50.50.100 trying to access a URL like https://cisco.com and this permission is being worked by the URLF layer

Is there something at CLI level that ‘tracks’ this behavior for domain accesses?

Thanks

[Timothy_Hall]

This SK is focused on taking a packet capture of HTTPS traffic, then decrypting it, but you should be able to adapt some of the steps to your needs:

sk181440: How to collect decrypted packet capture of HTTPS traffic for analysis by Check Point Suppo...

[Matlu]

For the SK to make “sense” is it necessary/mandatory that my CP device has HTTPS Inspection enabled?

Currently I only work with the APPC+URLF blades but for the moment HTTPS Inspection is not available in our scenario, however we want to “track” the behavior of the users, when they try to consume certain URLs like https://cisco.com or https://checkpoint.com.

Thanks for your comments.

[Timothy_Hall]

It sounds like you want to enable the Track option Extended Logging in your Access Control policy, which logs every full URL that a user's browser pulls, and not just the main site name.  Be aware that this option can easily be the cause of an overloaded log server, so use it sparingly.  

[PhoneBoy]

And to log the full URL, you need HTTPS Inspection enabled.
Otherwise, it is not possible to see the full URL.

[Bryce_Myers]

I would recommend downloading the healthcheck script from sk121447 and looking at the section called "check_debugs()".

Or just run the healthcheck script on the gateway and read the output about the debug configurations.