Cluster dropping packets in R80.30 unicast Load sharing Mode

[Sudip_Majee]

Cluster is dropping packets in unicast Load sharing Mode after upgrading from R80.10 to R80.30, while in HA mode it is working fine.

below are the output of "fw ctl zdebug + drop"

@;825535;[cpu_8];[SIM-207416775];pkt_handle_stateless_checks: Packet dropped (cluster decision). conn: <xxx.xxx.xxx.xxx,46667,xxx.xxx.xxx.xxx,443,6>;

@;825535;[cpu_8];[SIM-207416775];do_packet_finish: SIMPKT_IN_DROP vsid=0, conn:<xxx.xxx.xxx.xxx,46667,xxx.xxx.xxx.xxx,443,6>;

@;825535;[cpu_0];[SIM-207416775];sim_pkt_send_drop_notification: (0,0) received drop, reason: cluster error, conn: <1xxx.xxx.xxx.xxx,46667,xxx.xxx.xxx.xxx,443,6>;

@;825535;[cpu_0];[SIM-207416775];sim_pkt_send_drop_notification: no track is needed for this drop - not sending a notificaion, conn: <xxx.xxx.xxx.xxx,46667,xxx.xxx.xxx.xxx,443,6>;

[PhoneBoy]

Probably best to open a TAC case on this so we can have a closer look.

[_Val_]

To enable LS mode with R80.20 and R80.30, use the solution mentioned here: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...

[Sudip_Majee]

yes, already followed the SK162637 to enable load sharing on R80.30, but post that we are facing issue in load sharing while HA mode is working fine.

[_Val_]

Please open a case with TAC

[_Val_]

One more note. Did you make sure to get the required HFA, as the case states? Just checking.

[Sudip_Majee]

Yes, Take 111 is installed on GW.

[Yair_Shahar · Accepted solution]

Hi,

Those drops are known and are harmless, originated from debug being enabled while using +

these debug messages show cluster decision on load sharing while one member handles the connection and other drops it.

you can use "fw ctl zdebug drop" to avoid these debug messages and see FW drops only.

Yair

[Sudip_Majee]

it drops all the connection on all GW and unable to communicate through firewall.

The issue resolved post installation of Custom hotfix provided by TAC.

[Yair_Shahar]

Hi Sudip,

Can you share SR number so we can follow up on the issue observed by TAC?

Yair

[Pawel_Szetela]

Hi,

We observed the same behaviour: 3 gateways, clean install, JH111 applied, ClusterXL Load Sharing Unicast, open server.

Is there any chance to integrate fix with JH in near future? Or we have to open service request?

Regards,

[Yair_Shahar]

Hi, 

I understand you observe real drops and not just drop messages as suggested above.

Did you follow sk162637?

specifically this note:

• Starting from R80.20, the ccl_force_sticky kernel parameter must be set to 1 (in fwkern.conf) on all cluster members.

Yair

[Pawel_Szetela]

Hi,

Yes, we've followed instructions in sk162637 and we observe not only drop messages mentioned above but real drops in traffic as well.

Regards

[Yair_Shahar]

Thanks,

I'm not aware of any known drops in this configuration, if you can add more details on what kind of drops you observe, it can help pinpointing the root cause here.

Anyway, I would suggest to open Service Request to for proper handling and fixing  if needed.

Yair

[Pawel_Szetela]

Hi,

To be more precise: no traffic is going through the cluster, we can only connect (ssh) to cluster members from directly connected networks.

Yesterday we did some labs (in vmware) and we observed the same problem - no traffic in clusterxl load sharing unicast (vmac), only switching to HA mode resolves problem.

We'll check sugestion from Sudip_Majee (below) to disable vmac (probably next week in maintenance window).

Regards

[Sudip_Majee]

Yes.

you can disable vMAC and run the below command to make it working and later ask TAC to provide custom hotfix to resolve the issue.

fw ctl set int fwha_vmac_global_param_enabled 0

[Pawel_Szetela]

Hi,

Thanks a lot for Your advice. We'll try this next week.

Regards,

PS. Anyway, I don't like custom hotfixes because of future compatibility problems with JH accumulators.

[Yair_Shahar]

Can you share the drop log you observe when using fw ctl zdebug drop ?

Yair

[Pawel_Szetela]

Hi,

Sorry but we did "fw ctl zdebug + drop" and results were the same like in first post in this thread (we didn't save results). We reverted back to R80.10.

We're considering next try for next week.

Regards

[Yair_Shahar]

Hi,

We did additional lookup in regards to these drops.

Following drop message is as mentioned above and are harmless debug messages

@;825535;[cpu_8];[SIM-207416775];pkt_handle_stateless_checks: Packet dropped (cluster decision). conn: <xxx.xxx.xxx.xxx,46667,xxx.xxx.xxx.xxx,443,6>;

Following drop message is a real drop and is a bug reflected in Cluster Load Sharing mode - this has been addressed in R80.40 and is already planned to be addressed in R80.30 next Jumbo HF (above take 140)

@;825535;[cpu_0];[SIM-207416775];sim_pkt_send_drop_notification: (0,0) received drop, reason: cluster error, conn: <1xxx.xxx.xxx.xxx,46667,xxx.xxx.xxx.xxx,443,6>;

I hope this clear things out.

Yair

[Pawel_Szetela]

Hi,

Thank You for clarification. We're looking forward for next Jumbo HF.

Regards

[Pawel_Szetela]

Hi,

I forgot to mention that in our case we've seen both drops You wrote about.

Thanks again