VSX Cluster - CoreXL decision

[Arthur_DENIS1]

Hi all !

I need somes inputs to help me with performance optimization.

We are going to deploy a new VSX (to replace another vendor running virtual firewalls), but I cannot find a specific documentation/method of calcul to define how much CoreXL i need to grant for each VS...

In the Admin Guide - section optimization, I see this sentence:
"We recommend that you use the number of CoreXL Firewall instances for each Virtual System according to the expected network traffic on the Virtual System. Configuring unnecessary CoreXL Firewall instances can have a negative impact on performance."
So... not too much, clear... But which amount is optimized ?

What's your method to define this ? Based on the number of throughput ? Concurrent connection ?
Split the physical amount of core to share this inside all VS?

Thanks for your inputs !

[Alex-]

HCP will recommend you to configure 1:1 based on the number of FWK and will give you a warning under that, so a machine with 2 SND and 14 FWK would give you 14 cores to assign to your VS'es you could split according to the expected load on each.

Also to consider the performance of the competitor it is necessary to know which appliance/Open Server you will be using and check the relevant hardware specs.

The maximum ratio is 2:1 but I personally never used it and now there's Dynamic CoreXL balancing which might change the roles of cores dynamically too.

There's also the mix of blades you will be using per VS which come into play. 

I believe the drop in performance is due to the increased memory usage and relevant computations depending on blades and processes which ensue. Under 100% of FWK utilization, HCP will tell you your instances count is too low and you should consider increasing in case of performance issue. Should doesn't automatically mean must, I've seen VSX cluster performing with reduced cores <1:1 without any issues on all VS.

You could start halfway or 1:1 among all your VS knowing that changing cores allocation later on causes an outage on the VS.

[genisis__]

What appliance are you running on with how many cores/memory?

What version of Checkpoint are you going to run with what JHFA?
Are you going to be running any additionally blades other than Firewall, if so what blades?

As an example:
I have a VS running with 6 cores assigned to it, its processing about 500-600Mbps on average and the cores are running between 60-70%, concurrent connections are between 100 - 180K.  
I have firewall/IPS/AV/ABOT/Application Control/URL Filtering blades running.

I also have dynamic balancing enabled, but no hyperflow.

[Arthur_DENIS1]

Hi both !

Thanks for your answers!

I'm totally agree about HCP recommendations... However we are not in production yet, and as you says changing cores allocation later on causes an outage on the VS ...

What do you mean by the maximum ratio is 2:1 ? Ratio of coreXL for ipv4/ipv6 ?

We are running 16200 appliance, with 48 cores and 64GB of RAM, running R81.20 JHF8.

Blades enabled are currently FW, IPS and VPN, but we are going to activate Application Control/URL Filtering blades  in the futur.


So if I take your example - for VS where i configure 200k concurrent conns, I can assign 6 cores
But that mean for another VS having 1M concurrent conns, need to assign 30 cores ?
As far I know, is not recommended to assign more CoreXL that the amount of physical core ?

Thanks

[genisis__]

Suggest upgrading to JHFA10.

Now what are blades are you going to run, and how many VSs are you building? What are your throughput requirements per VS and how many concurrent connections do you believe will be needed?

Memory wise - I would always suggest getting the maximum memory into the appliance from day 1.  

[Arthur_DENIS1]

Noted for JHF10

Blades enabled are currently FW, IPS and VPN, but we are going to activate Application Control/URL Filtering blades  in the futur.

We have 9 VS in total - with this load repartition:
- 6 VS, 200k concurrent conns, 10GB throughput 
- 2 VS, 1M concurrent conns, 20GB throughput 
- 1 VS, 50k concurrent conns, 1GB throughput 

Noted for memory, but budget is driving this decision ...

[Alex-]

My experience with VSX on the 16200 and Dynamic CoreXL is that 8 cores will be set as SND very quickly but I've never seen  the firewall going above, also I haven't used yet VSX with R81.20.

R81.20 will also reserve a dedicated core for FWD so all of this considered it leaves you 39 FWK.

From there, you can split the VS cores according to their foreseen utilization ratio.

About changing the number of cores once in production, I did it already on both R81 and R81.10 with very little impact (a few pings at most) but it was always performed in the lowest utilization periods.

For that setup I would recommend at least the Plus over the Base model which gives you the 10G and 64GB RAM built-in.

[AmitShmuel]

Hi, FWD has a dedicated core only in USFW, not VSX.

[Arthur_DENIS1]

Hi,

So that's means we have our 48 cores to split according to their foreseen utilization ratio ?

We have indeed the PLUS model 🙂

[AmitShmuel]

Yes 🙂

[genisis__]

The way I read your reply is you have a total of three VS's:
VS ID 1
VS ID 2
VS ID 6

All three are running FW/IPS/VPN currently and there is a plan to enable further blades.

VS ID 1:
50K Concurrent connections
through requirement 1Gbps.
Suggested number of cores to test initially 4

VS ID 2:
1M Concurrent connections
Throughput requirement 20GB
Suggested number of cores to test initially 16

VS ID 6:
200K Concurrent connections
Throughput requirement 10GB
Suggested number of cores to test initially 16

If you have built from scratch then Dynamic Balancing and Hyperflow should be enabled (I can't remember if Hyperflow is supported on VSX, so something to check).

Dynamic Balancing should deal with SND cores as throughput demand changes.

Please keep in mind you need to really test this in your environment to determine what works best.

Memory - I still think 16GB is a little on the low side, if you thing say 4GB per VS + the memory to run the appliance you could be hitting more swap file then you like, so please take this into consideration.

[genisis__]

Sorry just read this again (was reading on a mobile earlier),  you have a total of 9 VS's.  Can we break down the throughput requirements per VS.

[Arthur_DENIS1]

Hi !

Indeed we have 9 VS in total.

Regarding the throughput, we want to allocate this capabilities (with FW, IPS, VPN):
- for 6 VS: up to 200k concurrent conns and up to 10GB throughput 
- for 2 VS: up to 1M concurrent conns and up to 20GB throughput 
- for1 VS: up to 50k concurrent conns and up to 1GB throughput 
=> meaning in total: 3,25 M concurrent conns and 101GB throughput 

Of course, the max throughput is not supposed to be reach in the same time for each VS. 16200 datasheet tell us that we can have up to 16M concurrent conns and up to 35GB throughput in IPS.

Regarding memory: we have 64Gb of RAM, not 16 🙂

[genisis__]

Memory - makes more sense now!

IPS I would take what it tells you on the spec sheet and half it.

I'm not convinced you're going to hit these throughput figures with this appliance, is sound like you need something like Maestro where you can grow your resources to meeting your demands.

You're going to seriously have to load test this.  Ideally you want to be in a position where your 16200 does not exceed 70% of its total CPU resource as an average with all your load (I generally use this as a rule of thumb).

[AmitShmuel]

HyperFlow is supported on VSX, yes.

[genisis__]

Awesome! thanks for confirming, so basically on the 16200 with R80.20 built clean as VSX we should have dynamic balancing and hyperflow enabled by default for all VS's.

I would be interested on Checkpoint's view on the core count allocations based on the load per VS indicated above on a 16200, and also if Checkpoint has any tools that could be used to give us a general guidance of how many cores would be needed based on different throughput levels, example 1GB, 4GB, 8GB, 10GB?  Perhaps the sizing tool can be improved?

One thing I wanted also also ask Arthur, when enabling the additionally blades do you also indent on using https inspection?  If not please keep in mind that majority of traffic will unlikely benefit from the other blades unless the payload can be inspected.

[Arthur_DENIS1]

Of course I'm going to test in real environment, and adapt if needed.
And deploying https inspection is indeed planned.

And good news for hyperflow 🙂

Thanks everyone !

[genisis__]

good luck, and please keep us posted, always interesting to hear about real world metrics.