Why CCP packets in VSX are send to network address of internal network subnet?

[Petr_Hantak]

I'm trying to figure out a strange case when we are able to catch traffic towards VSX internal subnet in different part of network.

I have a VSX VSLS cluster. Multiple virtual systems are connected to the same virtual switch, which is connected to normal network terminated by router. Router has default route out and here we can see the bottleneck. I can see traffic following traffic 0.0.0.0 -> 192.168.196.96 (UDP) 8116 going out of my network via that router.

I started to search why. According ClusterXL Advanced Technical Reference Guide is the source IP 0.0.0.0 fine for CCP traffic because it does not care about it. However, I am confused from the destination. I use Internal VSX cluster network 192.168.196.0/22 which is default setup. If I check the interface configurations in CLISH  I can see that was divided to /28 networks for the interfaces and some internal IPs were assigned there (multiple times for same interfaces, but it is correct according sk110345 - Identical IP addresses from VSX "Internal Communication Network" are assigned to interfac...).

So I expected to see communication of CCP on broadcast or particular addresses but I see it towards 192.168.196.96 – which is /28 subnet IP and not assigned to particular interface. There are send FWHA_MY_STATE messages there for example. Funny thing is that this traffic blocking stealth rule in the policy.

I found the same results on multiple all my VSX clusters on R77.30 and on one running on R77.10. Therefore, it seems to be regular thing. All clusters are fully synchronized and fine.

Do you know why is it communicate this way? I was not able to find it anywhere. You can see FW monitor result from one of clusters in attachment.

P.S. – I’ll ask support of course as well.

[_Val_]

What is the confusion, exactly? VSX is using "funny IP network" for internal adressing. CCP is using physical IPs as source for multicast or broadcast, depending on the mode settings. It is not different from a physical cluster communications, when you are using feature mentioned here: https://community.checkpoint.com/community/secure-knowledge/blog/2018/11/26/secureknowledge-weekly-c... 

This is all by design and normal. If you have multiple clusters, physical or VSX, on the same broadcast domain, you need to change cluster ID (also known as magic_mac) from default on at least one of them.

[Petr_Hantak]

I agree VSX is using funny IP. If I take a look I can see "funny IP" assigned on VSX cluster in following way

[Expert@FW01A:0]# clish -c "show configuration" | grep 192.168.196.9
set interface bond1.2213 ipv4-address 192.168.196.97 mask-length 28
set interface bond1.2217 ipv4-address 192.168.196.97 mask-length 28
set interface bond2.2277 ipv4-address 192.168.196.97 mask-length 28


[Expert@FW01B:0]# clish -c "show configuration" | grep 192.168.196.9
set interface bond1.2213 ipv4-address 192.168.196.98 mask-length 28
set interface bond1.2217 ipv4-address 192.168.196.98 mask-length 28
set interface bond2.2277 ipv4-address 192.168.196.98 mask-length 28‍‍‍‍‍‍‍‍‍‍

But I don't understand why it is use in traffic 192.168.196.96 as destination which is not present on any my node and doesn't look like broadcast anyway. I have no other cluster in the same network, but I have a cluster ID set.

[_Val_]

Uh, that was not a joke, "Funny IP Network" is an internal term for something called "VSX Cluster Internal Communication Network". 🙂

It is internal address pool used to build "physical" interfaces attached to different virtual devices in a VSX cluster. 

You kind find some references to this concept in Changing the VSX Cluster Internal Communication Network  and  Check Point VSX R80.10 Administration Guide 

[Petr_Hantak]

Thank you Valeri. I know that was not a joke. Many SK articles contains "Funny IP" term already 

I understand that is pool to build "physical" interfaces attached to different virtual devices in a VSX cluster.

I'm trying to understand more how the concept of "Funny IP" address assigment works and what is the relation with CCP traffic. Maybe someone has more deep knowledge about it here. 

I was able to solve the issue with leeking CCP traffic addressed to "Funny IP addresses" out of my network by blackholing "Funny IP range" on the border router. 

[_Val_]

The concept is explained in an official VSX course. If you ever take it, there is about 20 minutes of explanation. Unfortunately, it is hard to do in a comment here, sorry.

[Petr_Hantak]

Ok thanks, I understand that is more complex topic.. I took this course about 4 years ago and unfortunatelly it was not so deep there. I'll try repeat it during in case I'll get an opportunity.

[Chris_Hoff]

I would like to revisit this topic. I think the OP was trying to get to the point of why is the funny IP being seen on the physical network. My understanding of the topic is the funny IP is for use when routing packets internal to VSX and kind of gets natted on the way out to the physical network. So if you were to simply ping from one of the VSes that has say a virtual switch, that vsw has a funny interface (wrp) that has a funny ip. On the way out, the packet traverses the physical interface and somewhere along the line gets translated to the physical IP address of the VS's interface. 

So back to the OP question, why would the network ever see the "internal" funny IP addresses on the wire? Even for CCP traffic, shouldn't this actually look like it is coming from the physical IP? 

[HeikoAnkenbrand]

Hi @Petr_Hantak 

More to VSX and CCP read here:

R80.x - cheat sheet - ClusterXL

[Petr_Hantak]

Thank you Heiko!