Troubleshooting Azure ClusterXL

[Lloyd_Crosby]

Good day fellow Check Point engineers! Remember the day when troubleshooting ClusterXL was best done with cphaprob commands and standard fw ctl debug -m cluster debugs? Nay! I say, use the old IPSO shell ICLID. "show cluster state" will show you all of your clustering woes even when your cphaprob stat is showing Active/standby with no transitions.

The things you find when you've been working the same issue for ages and decided to poke around in old IPSO shells for giggles and grins.

[HeikoAnkenbrand]

Hi Lloyd,

I think that should be enough to cluster debug:

The good old ones:

cphaprob state

cpstat ha -f all

cphaprob –a if

fw ctl pstat

fw hastat

cphaprob syncstat

tcpdump –i ethx -vvv -e -nn port 8116

You can use the new R80.x commands:

clish -c \"show routed cluster-state detailed\"

cphaprob mmagic   

Or for more deep debuging CCP proto:

tcpdump -nepi ethX -x port -s 0 8116 2>/dev/null | ccp_analyzer -g -c

Or my favorite command , many don't know it (for more infos see "fw ctl zdebug" Helpful Command Combinations 😞

fw ctl zdebug + sync

I don't want IPSO shell anymore:-)

Regards,

Heiko

[Lloyd_Crosby]

You would think so though?

Been working on an issue that I identified related to Azure ClusterXL, No standard CP commands show the cluster transitions.

Only issues that standard files showed were CPU load and freeze state.

iclid actually showed the transitions.

We corrected the issue with the transitions:

fw ctl set int fwha_osu_enabled 1
fw ctl set int drop_fwhap_mac_non_ccp 0

And then we made it persistent in the fwkern.conf

The article related to this is:

sk123579: Azure ClusterXL fail over due to "READY" state

Corrected the issue with the transitions. I guess the tone was missed in the post needed a "/s"

[Lloyd_Crosby]

Also, we really shouldn't be having people using zdebug for standard troubleshooting. Zdebug should only be used in very rare situations or if you already know the issue and want to confirm what you suspect without using full kdebugs.