VSX Funny IP behavior with 81.20 ,they are used to do real traffic generated by VS

[AleLovaz82]

Hi

is it normal that all the traffic generated by a Virtual System appeaers to be generated by the funny IP of the outgoing interface of the virtual system and not with the VS ip ?

try to reach 10.0.0.240 from a vs with ip 10.0.0154 on wrp64 interface.

ip route get 10.0.0.240 -> 10.0.0.240 dev wrp64 src 192.168.196.145 

wrp64 ip is 10.0.0.154
Funny IS 
## \ifconfig wrp64
wrp64       Link encap:Ethernet  HWaddr 00:12:C1:21:E0:00
            inet addr:192.168.196.145  Bcast:192.168.196.159  Mask:255.255.255.240

I tried to do a telnet to 10.0.0.240 port 60001 but in log I can find it with source 192.168.196.145.
I also try to flag the option "Hide the internal network behind the GW ip" but nothing changed.

For specific destination I used a lot of nat as a workaround ,but I can't use this workaround if I don't know the destination

I had this behaviour with a lot of vsx installation with different version but they were "old" ,but also with all my new fresh install ( 29100 + 81.20 Take 89 ).

[emmap]

This is normal. You should see it NAT to the configured interface IP in the logs. If you don't see that NAT, you may have a 'NoNAT' rule that is too broad and is covering that internal network.

[AleLovaz82]

I don't have any NoNat rules, I also tried to apply a brand new empty rulebase but same results.

[Don_Paterson]

@Timothy_Hall 
Hey Tim,
Can you shed any light on this one?

I have been told in the past that the Internal Communications Network (the funny IPs) should not be visible externally, but I also know that that range is not to be used in the same customer network.

I used to see funny IPs pop up in in trace routes but have not seen that for many years/version now, so that was fixed.

As per https://support.checkpoint.com/results/sk/sk110345: 

"The "Internal Communication Network" is a virtual network that is required for Check Point ClusterXL environments in addition to the synchronization network. The internal communication network is invisible to external networks and enables cluster members to communicate and recognize the state of the environment."

When I run fw getifs I see the funny IPs on the wrp interface but when I run ifconfig I see the real IPs attached to the warp interface on the VS.

Regards,

Don

[Bob_Zimmerman]

---

@Don_Paterson wrote:

When I run fw getifs I see the funny IPs on the wrp interface but when I run ifconfig I see the real IPs attached to the warp interface on the VS.

---

It's actually the other way around. 'fw getifs' and '/usr/sbin/ifconfig' show the real IPs which the interfaces actually have (what you're calling the "funny IPs"). 'ifconfig' by itself is aliased to '/bin/cp-ifconfig.sh', which returns the cluster VIP defined in SmartConsole instead of the IP the interface actually has.

Internally, VSX clusters just use off-net member IPs from automatically-allocated blocks. They have cluster VIPs like normal. You only define the cluster VIP, and various commands return false information to make it look like the member actually has the VIP you defined, but it's really just a VIP like on any other cluster.

Which brings us to what @emmap said earlier. Traffic originating from a cluster member shows in the logs as having a source of that member's real IP which is then NATed to the cluster VIP. If the traffic isn't being NATed, then either a NAT rule is preventing the translation, or something similar (like setting fwha_cluster_hide_active_only to 0) is disabling the cluster NAT.

[Don_Paterson]

On an R81.10 VSX cluster member I get funny IPs in the fw getifs output, see attached.
ifconfig looks 'normal'.

That is from a lab with a R81.10 VSX cluster, freshly installed using only defaults.

I guess the deployment needs to be checked for defaults (OOB) settings and any customisations with consideration for cluster hide

Thanks,

Don

[Bob_Zimmerman]

Yes, like I said, 'fw getifs' returns the real IPs, and ifconfig lies to you.

[Oliver_Fink]

It is just a matter of perspective what you call the "real IPs".   Me, I tend talk of "real IPs" (externally used IPs) vs. "funny IPs" (ICN IPs).

Technically, you are right: The real IP of the interface IS the funny IP. That is the IP attached to the interface. Everything else is NAT magic. Maybe I should switch to terms like "internal IPs" and "external IPs" to be more precise.

[AleLovaz82]

i'll double check everything again ,but I have the same behavior with fresh installed 29100 + 81.20 and also with a lot of different hw / version not managed by me from the beginning ( from 80.10 to 81.10 ,from 12400 to 23000 )
Regarding fwha_cluster_hide_active_only  is set at 1

[JozkoMrkvicka]

one "special" situation where you will see funny IP as VIP is the moment when you configure new interface on VS within SmartConsole and click OK. Without policy push for affected VS, funny IPs will be seen as cluster IP. After policy installation, real cluster IP should be seen and used for any communication.

[JozkoMrkvicka]

You mentioned it "appears" to be using funny IPs. Do you have some proof like cppcap or fw monitor that the traffic is leaving VS with funny IP and not with VIP ?

Checking logs within VS is not user-friendly at all. You might be confused with that No-NAT rule translating funny IP to VIP. I dont see logic why confusing customers to have visibility "under the hood" what is happening. I dont care and dont want to be confused that there is internal NAT for my communication which is in fact no using real NAT at all.

Please paste here output from "fwaccel if" and "fwaccel6 if" where we can see all IPs together.