Constant monitoring with 'zdebug drop'

[Matlu]

Hi, Mates.

A query, the command

#fw ctl zdebug drop | grep 'Destination IP'

can be left 'running' in the background in order to 'monitor' a particular IP and know if during the day, the command 'registers' something relevant?

We do this in order to capture traffic at the exact moment, because unfortunately when we apply the command in real time everything works fine, but just when we stop testing, the connection between 1 source and 1 destination begins to fail and we have no records of those precise moments.

Is this possible?

Greetings.

[the_rock]

I let it run in my lab before for a week, never had a problem.

Andy

[Matlu]

Hey, Buddy.

I think my query was not well explained.

I want to leave ‘running’ the zdebug drop for example x 1 week.
So, if for example I connect via ssh and applied the command, it is simply not to interrupt the command with Ctrl+C?

If I close the terminal (Putty, SecureCRT, etc), I can be sure that the command is still running?

[the_rock]

Thats exactly what I did.

[Matlu]

Is it possible to send the output of the command #fw ctl zdebug drop | grep -IP DESTINATION- to a particular ‘path’ of a vsenv (For example in /var/tmp/) and tell it to save the result in a file ending in .txt extension?

[the_rock]

Yup...example

fw ctl zdebug + drop | grep 1.1.1.1 > /var/log/zdebug.txt

[Matlu]

Hi,
The #fw ctl zdebug drop
is 100% reliable for troubleshooting?
The messages that appear here for a particular IP that you are having problems with, for example related to TCP RETRANSMISSIONS, is it reliable to rely on this command or are there other testing options?

[the_rock]

Personally, I would say its reliable, yes, but you would need to use it along with other things, such as tcpdump, logs, fw monitor, etc

Andy

[Matlu]

Hi,
2 inquiries, please.

- To capture traffic in Troubleshooting with the FW Monitor is it ‘indispensable’ to turn off the SXL?

- In a VSX environment, can I capture traffic with the FW Monitor and export the output to a pcap file, in a particular vsenv?

Thanks for the replies.

[the_rock]

1) You can, but most people dont, unless its really complicated issue or involves kernel debug (in my experience)

2) Its been some timeI dealt with VSX, but I recall you can do so in VS itself

example -> vsenv 1 -> fw monitor -e "accept host(1.2.3.4) and port(444);" -o /var/log/testfwmonitor.out

Andy

[Matlu]

I want to capture TCP traffic with #FW Monitor and #fw ctl zdebug....
In an environment of a TCP communication between 1 source and 1 destination on a particular port, as we are currently reporting a problem with TCP retransmissions.

So I wanted to be sure if it is necessary to turn off the SXL, because to be honest, many times one forgets to 'turn it on' again. 😏😅

[the_rock]

One time when I was helping a lady with some CP upgrades and they would always turn off sxl and she says "Dont worry Andy, I always remember to turn it back on after the upgrade", there comes 6 am, I say you forgot something, she goes "What, should we go for a breakfast?"...Im thinking, YES, PLUS turn back on sxl haha 🙂

No shame in writting it on big piece of paper or whiteboard, all good!

Andy

[Timothy_Hall]

There are very few situations where you must use fw monitor -e (which can only capture traffic in F2F/slowpath) on a modern Check Point gateway.  If all you need to see is whether traffic was received and then whether it was transmitted (basically just capture points i and O), use cppcap.  If you need to see all 4 capture points iIoO (say for detailed NAT troubleshooting, or to see if Gaia is dropping something instead of the Check Point code), use fw monitor -F.  Both these tools will give you a complete capture regardless of the state of SecureXL.  SecureXL should really not be disabled for any reason via fwaccel off on a modern Check Point gateway.

If you attempt to run fw ctl zdebug + drop and fw monitor -F simulteneously, be aware that you need to run them in the proper order to keep them from interfering with each other.  See here: Debug Filter Battle -- fw monitor -F vs. fw ctl zdebug + drop

[the_rock]

On another note @Matlu , what is the EXACT issue you have here? You said lots of tcp retransmissions?

Andy

[Timothy_Hall]

For long-term monitoring with fw ctl zdebug + drop (the "+" ensures you see drops by SecureXL which are rather rare) I wouldn't recommend piping the full output to a grep like that.  While the overwhelming majority of packets are accepted and don't show up in the zdebug output, the commencement of a DoS attack or even port scan will cause the amount of output to increase rapidly and potentially impact the performance (or worse) of the firewall.

What I would recommend is something like this in one window:

fw ctl zdebug + drop > output.txt 

Then after a few seconds (long enough for the zdebug to fully initialize) do this in another window:

fw ctl set str simple_debug_filter_daddr_1 DESTINATION_IP

Then after hitting CNTRL+C to stop the zdebug in the first window, now run this command in the second window just in case:

fw ctl debug 0 (to ensure your filter has been unset, the zdebug being stopped should do this automatically though)

The second command will filter the output inside the debug itself and substantially reduce the chance of a DoS flooding the output.  Unfortunately you can't run that fw ctl set command before starting the zdebug as it will just get reset by the zdebug when it starts (and usually when it ends).

If anyone knows how to add/set simple_debug_filter_daddr_1 to the zdebug as part of invoking the zdebug in a single command I'd be all ears.