cancel
Showing results for 
Search instead for 
Did you mean: 
Post a Question
Highlighted
Vladimir
Pearl

Problem accessing standby cluster member from non-local network

Log shows accepted traffic on SSH and 443, cluster members connected to number of Cisco switches with VLANs in L2 mode.

No problem accessing both members from connected network.

vMAC in the cluster object IS ENABLED.

Any suggestions will be appreciated.

Thank you.

Tags (2)
20 Replies

Re: Problem accessing standby cluster member from non-local network

How about fwha_forw_packet_to_not_active? Helps with similar situations with ping also.

Cluster debug shows "FW-1: fwha_forw_ssl_handler: Rejecting ssl packets to a non-active member" 

Try with just entering # fw ctl set int fwha_forw_packet_to_not_active 1, and if works, enable on permanent basis in fwkern.conf.

I hope there is still fwkern.conf on R80.10

Vladimir
Pearl

Re: Problem accessing standby cluster member from non-local network

I'll give it a shot shortly and let you know if it works.

Thank you.

0 Kudos
Vladimir
Pearl

Re: Problem accessing standby cluster member from non-local network

Kaspars,

The situation is as you described, i.e. accessing from the "other" side.

However, ssh and https traffic to the management interface of the standby member logged as "accepted".

That being said, I'll try adding the route and see if it does the trick.

Thanks!

0 Kudos

Re: Problem accessing standby cluster member from non-local network

This is what happens if you don't have the specific /32 to the standby. Packet arrives on active FW. It gets accepted and needs to be forwarded to standby box. It will do so based on topology, that is outside interface. But when this packet arrives to standby on outside interface with source IP from inside network, it will get dropped as spoofed. Unless spoofing is off Smiley Happy

Vladimir
Pearl

Re: Problem accessing standby cluster member from non-local network

Ah, but I've run the infamous fw ctl zdebug drop on the standby member (not yet in production), and have not seen the drops there.

0 Kudos

Re: Problem accessing standby cluster member from non-local network

Will try tomorrow in one of clusters. Interesting. I just recall seeing spoofing drops in logs. Let you know.

0 Kudos

Re: Problem accessing standby cluster member from non-local network

Here you go, fw monitor shows packet being accepted on incoming interface on active cluster member. But no outgoing packet there

fw ctl zdebug shows on active member

Note that this is without enabling fwha_forw_packet_to_not_active. After I enable it it started working immediately. Without static routes. Smiley Happy

But I have seen your case not that long ago and that's why I remembered about /32 option..

Sorry won't have much time to dig into my past notes what happened there.

Vladimir
Pearl

Re: Problem accessing standby cluster member from non-local network

Thank you again for looking into it: I have not seen the drops because I was looking for them on the standby member, thinking that if I see "accepts" in the log, the primary was not dropping them.

Apparently it drops them on egress.

Of two solutions available, which one do you like better?

0 Kudos

Re: Problem accessing standby cluster member from non-local network

Hehe, I think I found exactly your situation Smiley Happy

Connection from one side of the ClusterXL destined to the physical IP address of a non-Active cluster member on the other side of the ClusterXL fails - sk42733 

I just wanted to mention that this fix with fwha_forw_packet_to_not_active is also connected to many other possible issues:

Simultaneously pinging the cluster members and the VIP address...  

"Contract entitlement check failed" error on policy installation failure 

Cluster debug shows "FW-1: fwha_forw_ssl_handler: Rejecting ssl packets to a non-active member" 

"ERR_CONNECTION_REFUSED" error is displayed in web browser when connecting to Gaia Portal 

Updates For Anti-Virus/Anti-Bot/Application Control/URLF blades are not working on standby ClusterXL... 

So, it might be even a "best practice" Smiley Happy  But I have only experience with versions before R80, don't know how it is there. Also these "strange" routes might be a bit confusing for a new administrator for example.

Vladimir
Pearl

Re: Problem accessing standby cluster member from non-local network

I agree that given how many issues this setting resolves, it may make better sense to have it on by default and have an option of commenting it out.

I wander what is the reason for it not to be and what possible side effects of it being set are.

Vladimir Yakovlev

973.558.2738

vlad@eversecgroup.com

0 Kudos
Vladimir
Pearl

Re: Problem accessing standby cluster member from non-local network

In the process of deploying new cluster of 15400s and ended-up using this suggestion.

Just wanted to tip my hat to you.

Thanks,

Vladimir

Re: Problem accessing standby cluster member from non-local network

I had similar problem, solved after setting "fw ctl get int fwha_forw_packet_to_not_active" value to 1. sk42733 was helpful

Re: Problem accessing standby cluster member from non-local network

You probably trying to connect to IP that's on the "other" side of the firewall and not locally connected IP. If you have only one route to the firewall VIP then traffic will get dropped as spoofed between firewalls. You can create manual static route for IP on the other side to point to standby memebr's locally connected IP interface address

Hope it makes sense Smiley Happy

In other words if you have inside  member-act 1.1.1.1 and member-stb 1.1.1.2 with VIP 1.1.1.3. And outside has IPs 2.2.2.1 and .2 and .3 then to connect to 2.2.2.2 via inside interface you will need to add static /32 on the router: 2.2.2.2 next-hop 1.1.1.2

Or I misunderstood maybe the problem?

0 Kudos
Vladimir
Pearl

Re: Problem accessing standby cluster member from non-local network

Kaspars and Alexei, thank you guys!

Ended-up using /32 routes as per Kaspars suggestion since I've recalled being in similar situation before and this being a less intrusive solution.

I will keep Alexei's solution in my toolbox.

That said, I still am baffled as to why zdebug drop did not yield anything on standby member when it was failing.

Regards,

Vladimir 

Re: Problem accessing standby cluster member from non-local network

Jerry
Gold

Re: Problem accessing standby cluster member from non-local network

got similar story where

standby device in A/S HA is accessible by IP but GAIA Portal (http2 daemon) isn't working at all

I've regenerated self-signed certs

Still no go

httpd won't start

I can ssh to the standby device but one thing isn't accessible - https/http either on 443 or any other port simply HTTP daemon won't start on that device (there were no chances to that cluster since nearly 1y in terms of the PKI/FQDN etc.)

any ideas folks? alreago got a SR with TAC Smiley Happy just from yesterday.

Cheers

Jerry

Jerry
0 Kudos

Re: Problem accessing standby cluster member from non-local network

I would say it's a separate issue as you can SSH to standby cluster member. So L3 routing is working end to end. There are multiple SKs regarding dead httpd, really depends on SW version you are running and actual errors you see. You might want to try this

How to debug the Gaia Portal 

0 Kudos
Jerry
Gold

Re: Problem accessing standby cluster member from non-local network

I did debug even with Ottawa TAC man - still no joy Smiley Sad

Cheers Kaspars, I do appreciate what you've wrote just now but all this is already known, we know it isn't routing or access lists issue but httpd dead as you called it.

now clue what's the problem but that HA runs pretty much latest "takes" ...

Jerry
0 Kudos

Re: Problem accessing standby cluster member from non-local network

I just meant to say that this thread probably is irrelevant to your case so best would be to start new one  

Jerry
Gold

Re: Problem accessing standby cluster member from non-local network

I did not nobody replied yet ...

thanks. eot.

Jerry
0 Kudos