In which cases would you use VSX?

[AlekseiShelepov]

Hello everyone.
I would like to hear your opinion and thoughts on the following topic.

Under which conditions the use of VSX on a cluster would really improve things? When would you prefer to use VSX over a usual setup? In other words, where is the line after which you decide "ok, this must be a VSX setup"?

The reason why I am thinking about this is that I cannot really see a lot of options for myself to use a VSX. I believe that in many usual cases adding VSX would just complicate things. VSX has some limitations, there are some additional things to be taken care of during operations like upgrades or migration, it should require additional hardware resources, as well as additional training of administrators.

Of course, VSX can bring several positive effects, which could weight out everything else (cost saving, flexibility, ease of adding new firewalls). But in my opinion this would work for only very few specific cases. 

I can see two options when most probably I would use VSX:

1. One of the cases when VSX could be preferable is when your company is a service provider and needs to support similar services for many customers. It would mean the you need to have many similar firewalls in the same data center, but they also must be separated from each other - own policies and objects, administrators, logs, etc.

So, many similar firewalls for different customers, separated from each other. It will help to minimize cost and time for adding a new customer to the environment.

2. The second case is when you are working in a quite big company which has multiple appliances for different purposes - external/perimeter firewall, VPN and mobile access appliance, internal firewall, separate firewall for server networks, etc.

One company with multiple firewalls for different purposes. It most probably would save quite a lot of money on appliances and their support contracts and would add possibilities to create VS for new purposes without big changes.

But would it be better to use VSX for a new VPN-only gateway for example? Or when you have only external and internal firewalls in your network? What about when you replace your old almost end-of-life internal firewall to a new cluster and besides of that there is only a separate VPN gateway?

[Vladimir]

One case where I would consider using VSX is where your Perimeter Firewall (Virtual System) is configured in Bridge Mode and has no IPs on its External and Internal interfaces and your Internal Firewall is addressable.

Deploying systems in this configuration allows you to drop a lot of junk before it reaches system that needs to process viable traffic.

Additionally, you have the ability to split IPS policies between different OSes (more applicable to r77.XX), as well as dynamically adjust number of cores (instances) dedicated to protecting particular segments of your network.

With R80.XX concurrent administration, the case for VSX becoming less prevalent than before, but it is still a compelling option when you protecting different entities of parent organization that sharing same hardware.

I would not recommend using VSX on anything smaller than 15000 series appliances where core count is sufficient to warrant it.

The administration overhead is definitely noticeable as are the limitations and the elevated risk of SNAFUs affecting multiple gateway instances.

Option of utilizing Open Platform VMs (with pre-allocated CPUs, RAM, Storage and dedicated NICs) should be weighted against that of VSX when choosing solution suitable for your organization.

Regards,

Vladimir

[Kaspars_Zibarts]

It's been love and hate relationship with VSX for over 12 years now.. Stated on Nokia IP appliances, went though Crossbeam X series, then more modern days of CP high end appliances and finishing with 40k chassis. Two fairly  sized but completely different enterprise networks - so it's case 2 so to say.

Upgrades used to be a big hassle. Especially in early days. But last two releases been noticeable improvement. Especially 77.30 to 80.10. Yes you're absolutely correct, planning upgrades is tougher as it may affect the whole datacenter if not all business. But in last 5 years I cannot recall a single upgrade that would have caused a major disturbance because of VSX. So in some ways it has been a "risky" win situation as you only had to upgrade one or two platforms instead of multiple clusters.

What's really good about VSX is ability to deploy new firewalls literally in the blink of an eye. We actually went through a fairly big network update and this part helped us heaps to save time, effort and $$$.

Downside is always going to be certain limitations - not all bells and whistles that are available on the latest regular gateway will be able fully available on VSX.  But then again it is getting better and better with every release.

Yes, you will need powerful HW to run VSX. Lots of CPU cores and memory. 

Administration overhead.. I don't notice it that much after 12 years

If you have only two firewalls/clusters I would probably stay away from VSX as you would not get return for your money.

[Alex_Rozhko]

can you elaborate more on your statement : "If you have only two firewalls/clusters I would probably stay away from VSX as you would not get return for your money"

[Kaspars_Zibarts]

I meant to say if for example you had regular external and internal firewall clusters, then converting that to VSX and not adding more  virtual gateways would be wasted money and also loosing some functionality as not every feature that's available in regular gateway works in VSX. The only compelling reason I can see physical limitations of datacenter, when you cannot have 4 appliances but 2 still fit in the rackspace...

[Maarten_Sjouw]

As an MSP, we run several different setups with VSX, most are perimeter VS'es per customer. Next to that we have a few customer that run different VS'es for different functions, like Dev/test, acceptance and a Guest network. Another customer is a holding with 3 companies that use the same perimeter hardware but with a VS per company.

So you can really say we run all kinds of setups with VSX.

The biggest hassle is how you have a mix and match on the management side, when you need to move to a different version or another management server, all connected CMA's need to be moved in one go. You also need to be very aware of what you are doing here as there is a lot of different parameters that need to be adjusted during the move.

Regards, Maarten.

[Alex_Rozhko]

Maartin,

can you elaborate bit more on "run all kinds of setups with VSX"? Did you run everything on single VSX? Was it HA or LS configuration in any case? How many VSs did your run on VSX(s)? What hardware platform(s) did you use? In case of multiple VS(s) on HA VSX(s) did you use dynamic routing (OSPF)? Did you utilize sub-interfaces/vlans/OSPF in single setup for multiple VS(s) on HA VSX(s)? If not hard, can you drop simple visual diagram(s) for such cases?

[Maarten_Sjouw]

Alex,

We are an MSP, in this environment we run 4 clusters for shared environments, 1 in Hong Kong, 3 stretched in different DC's in NL.

they are all connected to different MPLS clouds and 1 to a Machine to Machine cloud, they all run a vs per customer and are all setup as VSLS.

Hardware for these are Open server, 12200 and 4600.

In Dedicated customer setups we have from a single 4200 with 1 VS; a 12200 with 4 VS's; 1 cluster of 12200 running 6 VS's in VSLS mode; 2 clusters of 12600's with a growing number of VS's, VSLS and at least 3 VS's running OSPF on each cluster; a pair of 13500's running 5 VS's in VSLS mode.

We use any combination of bonding (load-share and active/backup mode), VLAN trunking, direct interfaces and Virtual switches.

[Kaspars_Zibarts]

Can add just to confuse you more  We use HA instead of VSLS, static routing only and tagged interfaces on bonds. Mostly used as enterprise segregation platform between different business functions, units, geographical etc

[Alex_Rozhko]

What platform? How many VS(s)? Bonding on 1g or 10g ports? How big is your routing table? How many users/devices protected? What blades enabled? This is good conversation. I'm at the decision point Aleksei was : VSX or not VSX. My personal preference is HA. In VSLS still have to account for all VSs running on one VSX for maintenance with 7/24 uptime requirements

Sent from my iPhone

[Kaspars_Zibarts]

To give you some sample

13800 not hyperthreaded (20 cores)

2x10G bond plus couple 1G ports

couple hundred static routing table

7 VSes, HA not VSLS

one having 95% of all connections

altogether 500000 concurrent conns

total traffic 5Gbps,

CPU ave 30%, RAM 25%, HDD - 30GB used

Blades: FW+VPN, IPS (basic mode) IA

[Vladimir]

VSLS only makes sense if you have 3 or more VSX units.

[Alex_Rozhko]

Thank you, good stuff. When you say AI does it include both, url/app blades or just app? Also, no mention of TP (av/bot) blades? And lastly what code version?

Sent from my iPhone

[Kristof_Vermael]

I don't share your opinion, I always use VSLS when using VSX, as this allows you to fail-over 1 specific node to 1 specific physical host.

Without VSLS, it is all or nothing, 1 failure on 1 VS will failover all virtual systems.

[Vladimir]

Perhaps there is a case when using VSLS with two members is advantageous and I would like for you to describe scenarios when having an option to fail over a single VS to a different node would be helpful, while fail over of the entire node is undesirable.

In its present form, VSLS precludes the possibility of using virtual routers as well as, IMHO, in two member cluster creates a more complex troubleshooting environment.

But I'll always welcome a different take on implementation with explanation of benefits.

Regards,

Vladimir

[Kaspars_Zibarts]

What can I say - we like being able to swing one VS to a specific datacentre leaving rest in the other but the same time I had to bang my head against the wall when one of the new projects could have been implemented in much faster and nicer way using VRs..

Really depends on your requirements.

[Kristof_Vermael]

Hello Vladimir,

Failing over 1 virtual system is indeed not a common thing to do, but if you have for example 5 virtual systems, 1 interface failure on 1 virtual system will fail-over every virtual system and in many cases, this is not a desired result  when having an "active" and "passive" datacenter.

With VSLS, the VS failover don't impact other systems.

[Petr_Hantak]

Possibility to failover simple or some VS only are nice and could be useful. We used VSLS for this possibility on some VSX boxes as well. But in summary it shows up to be a problematic when we used it some "shared" box and we close down possibility to use features which wouldn't be a problem on standart HA solution.

At least for me it makes a sense to use it on certain environment when you know your requirements and known limitations are not an issue for you.

[AlekseiShelepov]

Why do you use a single 4200 with 1 VS? I think it gives 0 pluses over a usual setup of 4200 and complicates things a bit.

And just for understanding and possible future planning of VS setup, could you provide some average resource usage (CPU, RAM, HDD maybe) numbers from different types of your VSX setups?

[Maarten_Sjouw]

It's even worse, it's a IP297 and we had a need for this due to the connectivity on that location with a separate DMZ and internet leg (old 10Mb DSL) they could not yet decommission. That was 3 years ago. 

For resource usage I have collected some info.

Open server:

   8 cores, average 10% used with 15 active VS's (no blades) 160GB disk 30GB used; 32GB mem 19GB used

4600:

   2 cores average 5% used with 2 active VS's (no blades) 100GB disk; 20GB used; 4GB Mem 3GB used

12200:

   4 cores, average 20% used with 7 active VS's (IPS blade) 120GB disk 20GB used; 4GB mem 2GB used
12600:

   10 cores, average 5% used with 8 active VS's (IPS blade) 180GB disk 20GB used; 12GB mem 3GB used

[Philip_W1]

I used to work in a company that (like Maarten) hosted VSes per customer in a Datacenter. In another case, I deployed a VSX with 2 VSes because the customer had 2 separate environments (company network vs iot)

Another customer was hosting his own (electronics retail) website and once got DDOS'ed, which not only took down the website but the whole firewall and thus the company. He was considering migrating to VSX to be able to separate his website networks from the other company networks, so in case of another attack only the website's VS would go down. Never implemented due to 'other important projects' though.

[AlekseiShelepov]

That's an interesting idea - DDoS resiliency. Would it really work in that way though? Have you maybe tested it somehow?

I would expect that a DDoS attack still affects performance of the whole device. In theory, if you have an appliance with 2 VSX, it could be that only cores and interfaces assigned to only one VS are fully loaded with this garbage traffic and other ones function normally. Not sure how it would affect RAM and HDD. Looks quite plausible, but I still have doubts.

[Kaspars_Zibarts]

It would work as long as you set up CoreXL correctly and separate cores for each VS. By default in R77 all VSes would share all available cores. So it can be tricky depending on number of cores you have. Memory is not such a big issue in R80 as VSes run 64bit kernel each in own RAM space. So as long as you have enough RAM for all tenants on the box it should be ok. We've seen it in practice - not because we were ddosed but some other "unexpected" internal traffic

[Alex_Rozhko]

Hi Philip,

Sounds like your previous configuration was single VSX with multiple VS(s)? please correct me if I am wrong. What platform you run VSX on in your case? Did you use HA or LS in your case?

[Oliver_Fink]

For me VSX seems to be very useful when implementing microsegmentation. You gain flexibility in building new firewalls regarding "provisioning", "installation" and "network cabeling". Such, you are able to build new firewalls for different segments or security domains with little effort and on short notice.

You can stay with one hardware platform for all different performance requirements for the gateways and balance the load via VSLS. 

[AlekseiShelepov]

How "micro" is your microsegmentation?

Could you elaborate a little bit more on what segments could be in your case? Something like I mentioned in the second case for my initial message or like in this comment? I try to understand when you decide that one firewall could be connected to 10 networks with different purposes (users, servers, printing, VoIP, etc.) and when it must be two separate VS, for example.

What series of appliances you are using for VSX? Some very powerful ones, or 5000 series too?

[Oliver_Fink]

It is a first step from a flat firewall infrastructure with one gateway towards micro segmentation. We did it at a customer of us, a media company. One challenge was that endpoints – mainly this of journalists – have to be considered insecure for the future – even more if the access via wlan. Such we decided to build on "external" firewall to control outgoing traffic and to be a first "wave-breaker" for incoming connections. Behind that we built 3 main firewall gateways: one data center firewall to protect the crown jewels of the customer, one dmz firewall with several zones with different kinds of services of services and one client firewall to separate different security levels of client endpoints, to host the terminal servers and to connect branch offices. Furthermore we implemented a vpn and remote access firewall inside one of the dmzs.

As far as I know the customer implemented one more virtual system after the end of the implementation project I was involved in. That was the plan: to gain the flexibility to do that if the need for an additional firewall gateway rises. But at the moment, I do not know what this special firewall gateway is used for.

On network level we tried to suppress any routing between any network segments behind the firewalls, so we are leading all traffic through the firewalls. Traffic from the client networks has normally to pass 2 firewalls concentrating on different key aspects, first the client firewall and after that the dc, dmz or external firewall. This enables clean policy structures for the different purposes and such reduces errors on osi layer 8.

The customer insisted upon using open server. So we used 3 IBM/Lenovo x3650 with 16 processors and 32 GB of RAM and several 10 GE interfaces.

[AlekseiShelepov]

Many thanks for such detailed and clear response.

[Alex_Rozhko]

VSX very cool term, what is the difference between VSX and built in micro-segmentation using sub-interfaces?

[Vladimir]

VSX allows to to have logical segregation of the Virtual Gateways. Each one of the VS' can have its own policy package administered by different people,serving different group of users and protecting different networks. Multi tenant environments are the simplest example of such utilization.