topic Re: Quantum Force CPU cores <=> virtual in Firewall and Security Management

Quantum Force CPU cores <=> virtual

Wolfgang — Thu, 29 Feb 2024 06:23:58 GMT

For some of the new Quantum force appliances curious virtual cores are shown in the datasheet. 10 physical and only 16 virtual. Should be 20 or not ?

9200 => 1 CPU, 4 physical cores, 8 virtual

9300 => 1 CPU, 10 physical cores, 16 virtual

9400 => 1 CPU, 14 physical cores, 20 virtual

9700 => 1 CPU, 16 physical cores, 32 virtual

Re: Quantum Force CPU cores <=> virtual

Markus_Genser — Thu, 29 Feb 2024 07:25:14 GMT

I think they will use the Intel 12th or 13th gen CPUs with the P and E cores, where only the P cores are hyper threaded.

https://en.wikipedia.org/wiki/Alder_Lake

Re: Quantum Force CPU cores <=> virtual

danel — Thu, 29 Feb 2024 08:17:26 GMT

Hi Wolfgang

The new 9300 and 9400 appliances contain Intel's CPUs in hybrid architecture,
were some of the cores are power efficient (E cores) and some are performance cores (P cores).

Power efficient cores are designed to run non-critical tasks while the performance cores are designed for intensive workloads.
Also, each power efficient core support a single virtual core where each performance core support two.

So the number of virtual cores are based on the number of each power efficient cores and performance cores:

In 9300 appliances:
No. Performance cores: 6 (12 virtual)
No. Efficient cores: 4 (4 virtual)
Total cores: 10 (16 virtual)

In 9400 appliances:
No. Performance cores: 6 (12 virtual)
No. Efficient cores: 8 (8 virtual)
Total cores: 14 (20 virtual)

Hope it helps,
Dan.

Re: Quantum Force CPU cores <=> virtual

Timothy_Hall — Sat, 02 Mar 2024 03:25:46 GMT

Hi @danel,

Is CoreXL aware of P-cores vs. E-cores and treats them differently? Or does it not distinguish them and just rely on Multi-Queue and the Dynamic Dispatcher to not overload the E-cores, which overall look to be about 30% slower than P-cores?

As an example there is no way I'd want an SND instance running on an E-Core, I'd also not like to see an elephant flow get assigned to a Firewall Worker Instance on an E-core (although Hyperflow can help out to some degree).

This seems like a scheme to save power/battery which I suppose I'd expect on a laptop, but not a server. I've had some very bad experiences with server-based power saving schemes in the past (i.e. P-States, C-States, & HPC Optimizations, Dynamic Power Capping) in regards to performance. Thanks!

Re: Quantum Force CPU cores <=> virtual

danel — Sun, 03 Mar 2024 14:02:50 GMT

Hi Timothy,
An SK which explains the logic of which type of cores (E or P) is affined to SND or Firewall Worker in MultiQueue auto mode and in Dynamic Dispatcher will be published soon.
I'll post it here when it's ready.

Thanks,
Dan.

Re: Quantum Force CPU cores <=> virtual

genisis__ — Sun, 03 Mar 2024 19:54:19 GMT

I look forward to that.

Re: Quantum Force CPU cores <=> virtual

Bob_Zimmerman — Sun, 03 Mar 2024 19:58:33 GMT

Possibly worth noting that the data sheets have some obvious inaccuracies. I'm not sure how much I trust them at this point. For example, the 9100 through 9400 claim they have an SFP28 slot for sync, but the photos definitely don't have an SFP slot of any kind. The catalog also lists those models as having 8x1/10g copper interfaces built in.

Re: Quantum Force CPU cores <=> virtual

genisis__ — Mon, 04 Mar 2024 17:37:22 GMT

That's not a good start, I also think the price point is way too high, totally get the appliances are more powerful, but cost is and always has been a massive sticking point.

Re: Quantum Force CPU cores <=> virtual

danel — Thu, 07 Mar 2024 07:55:41 GMT

Hi @Timothy_Hall ,
The following SK explains the behavior of setting cores for SNDs and Firewall Workers
in hybrid architecture CPUs:
https://support.checkpoint.com/results/sk/sk182000

Hope it helps,
Dan.

cc: @genisis__

Re: Quantum Force CPU cores <=> virtual

Bob_Zimmerman — Thu, 07 Mar 2024 16:03:54 GMT

So instead of relying on the Linux scheduler, the worker threads are pinned to exact cores, and the CoreXL scheduler is asymmetric-core-aware? Interesting.

Which cores are removed from the worker pool in what order? If I have a 9300 (6p4e) and I decide to run two SNDs and 13 workers, am I left with an unloaded hyperthread, E-core, or P-core?

Re: Quantum Force CPU cores <=> virtual

AmitShmuel — Sun, 10 Mar 2024 11:13:06 GMT

Hi Bob,

The FW worker threads have been pinned to exact cores since day one of CoreXL.

What do you mean by asymmetric-core-aware? The CoreXL dispatcher is aware of queue utilization, and CPU utilization, hence, naturally, if E-cores are working harder, it will dispatch less connections to them.

Regarding SNDs/workers split, as always, you configure the number of workers via cpconfig, and those will be pinned to the cores in decreasing order, from the top, and the remaining cores will be assigned to SNDs.
Nowadays it is less common to manually configure CoreXL, as Dynamic Balancing does that for you out of the box.

Re: Quantum Force CPU cores <=> virtual

Timothy_Hall — Sun, 10 Mar 2024 18:15:50 GMT

Thanks the SK was very informative. After thinking about the differences between P-cores and E-cores this is a pretty big deal.

All cores have always been assumed to have equal capabilities, with the lone exception that a SMT/Hyperthreaded "core" is really two threads heading for the same physical core. In the past various server-based power-saving schemes have wreaked havoc, to the point that for open hardware servers on pages 83-84 of my last Max Power book I advised disabling all these schemes in the BIOS to help ensure that all cores are equal at all times, power consumption be damned. This is a pretty fundamental tenet of how CoreXL and SecureXL works, with various technologies such as Multi-Queue, the Dynamic Dispatcher and Hyperflow to keep the load between all the equal cores relatively balanced.

With some cores much faster than others (and possibly supporting different processor extensions and CPU cache sizes between them - see below), I can envision a number of scenarios where bottlenecks may occur that weren't really possible before. A few thoughts which are pure speculation:

1) Are E-Cores currently acting as Firewall Workers eligible to be reassigned as PPE-based cores for Hyperflow when an elephant is detected? I would assume not.

2) When detecting spikes via the Spike Detective on an E-Core, will the same thresholds apply as do for a P-core? Given the disparity the thresholds might need to be a little lower for an E-Core.

3) I'm a little concerned that a P-Core SND blasting a high rate of packets at an slower E-Core Firewall Worker may cause CoreXL queuing problems (and loss) for traffic trying to reach the E-Core Firewall Worker.

4) Same threshold for activation of Priority Queuing on an E-Core vs. P-Core? Might need to adjust it a little lower on an E-Core to try to keep it out of queuing trouble sooner?

5) It appears E-Cores will support fewer processor extensions than P-cores. My concern here is encryption-oriented operations such as VPNs and HTTPS Inspection. Extension AES-NI is an obvious example, I'd think we'd want to keep encryption-based operations away from an E-Core if advanced processor extensions are being relied upon for performance? AVX512?

6) E-Cores will have much less fast CPU cache than P-Cores. Would we want to keep operations away from E-Cores that rely heavily on fully-populated "hot" CPU fast caches for performance? Operations like rulebase lookups perhaps?

Re: Quantum Force CPU cores <=> virtual

_Val_ — Mon, 11 Mar 2024 11:38:05 GMT

@Bob_Zimmerman @genisis__

Why are we talking about price here? Let's stay on point. The appliance sheet has correct information, and the reason for the odd amount of cores is already established in this discussion.

Re: Quantum Force CPU cores <=> virtual

AmitShmuel — Mon, 11 Mar 2024 13:00:46 GMT

Hi Tim,

Thanks for the great questions and feedback 🙂

1) Your assumption is correct, both SND and PPE workers are excluded from E-Cores pool, I've added it to the SK.

2+4) The Dynamic Dispatcher should naturally send less connections to FW workers running on E-Cores, keeping the cores mostly balanced. So far, we haven't observed any significant difference in our testing or customers environments in regards to those features. Super instance is a good example of our ability to work with cores with varying capabilities.

3+6) Unless we're talking about Elephant Flows, the Dynamic Dispatcher should take care of that, sending less connection to that FW instance. As to Elephant Flows, in the upcoming JHF, Dynamic Balancing will be able to swap E-core FW instance handling an EF, with a P-core FW instance, making sure heavily loaded workers get the benefit of stronger cores.

5) Intel has made sure both cores have the same instruction set.

Hope that clears things up, feel free to contact me offline anytime.

Thanks!
Amit

Re: Quantum Force CPU cores <=> virtual

Bob_Zimmerman — Mon, 11 Mar 2024 13:32:44 GMT

It has some correct information, but also some obviously incorrect information. For example, this screenshot is from the fourth page of the 9300 data sheet as loaded under ten minutes before making this post:

The 9300 includes 1x 1/10 copper GbE management ports plus an additional 1x 1/10/25 SFP28 ports for synchronization when using the 9300 PLUS in a cluster. With one network I/O slot, modify the base or plus configuration to meet your networking requirements.

My company's sales team has confirmed for me the 9300 has no 1/10g copper interfaces, and no SFP28 slot.

Re: Quantum Force CPU cores <=> virtual

Timothy_Hall — Mon, 11 Mar 2024 13:33:45 GMT

Looks like you guys have thought of everything, as usual. Very interesting about the new "swapping" ability for a heavy flow off an E-Core to P-Core, that must have taken some serious work as normally something like that would not be possible. Could this eventually lead to the ability on all firewalls for existing "mice" connections trapped on a Firewall Worker with an big meanie elephant flow to "scurry away" to another worker core for handling, and thus cease getting stomped by the elephant on the overloaded core?

I'm still not real clear on why Intel is doing this kind of power-saving scheme at the server level (other than an attempt to appear to be keeping up with AMD), but that is a discussion for a completely different forum than CheckMates. 😀 Thanks!

Re: Quantum Force CPU cores <=> virtual

_Val_ — Mon, 11 Mar 2024 14:14:43 GMT

Thanks for pointing this out. However, I ask again, please stay on topic. The current post is about the number of cores. If you want to report errors in the datasheets, please use different means to convey it.

Appreciate your understanding.

Re: Quantum Force CPU cores <=> virtual

AmitShmuel — Mon, 11 Mar 2024 14:47:42 GMT

The "swapping" refers to the worker's affinity, so we only move the thread from one core to another, it will still handle the same connections.

Re: Quantum Force CPU cores <=> virtual

Timothy_Hall — Mon, 11 Mar 2024 18:10:50 GMT

So basically the firewall instance/worker on the E-Core (including all its elephants and mice) gets moved to a P-core, and the firewall instance/worker formerly on that selected P-Core moves to to the E-Core. Got it. Thought for a moment there you meant that a connection or series of connections could simply be moved to a new Firewall Worker instance without restarting them, but it looks like they all have to go together which makes sense. Thanks!

Re: Quantum Force CPU cores <=> virtual

Bob_Zimmerman — Wed, 13 Mar 2024 21:01:04 GMT

I handled a few tickets in the early days of CoreXL with workers which moved from core to core as the Linux scheduler wanted. Here's some output from one of my R81.20 jumbo 26 firewalls taken just a few minutes ago:

[Expert@MyFirewall:3]# fw ctl multik stat ID | Active | CPU | Connections | Peak ----------------------------------------------- 0 | Yes | 2-7 | 501 | 859 1 | Yes | 2-7 | 17 | 91 2 | Yes | 2-7 | 25 | 97 3 | Yes | 2-7 | 24 | 103

They're definitely not always pinned to specific cores.

As for "asymmetric-core-aware", I mean just that. Until fairly recently, all cores available to a kernel scheduler had the same capabilities and the same performance characteristics (outside the mainframe world, anyway). This is the symmetric in "symmetric multiprocessing". Relatively recently, asymmetric core complexes have become available outside of mainframes and other exotic environments. The first really common one I saw was ARM's big.LITTLE arrangement. The Linux scheduler at the time didn't know that the little cores couldn't handle the same amount of work in the time slice. It would schedule a relatively light process on a little core, which would max it out, so the process would get moved to a big core, leaving the little core idle, so a process would get moved back to it. This would thrash the cores' caches, seriously hurting performance.

The Linux kernel scheduler didn't become sufficiently aware of Intel's E-core differences to allow effective use of P-cores until 5.15.35 in early 2022.