Calculate MQ cores when upgrading

[Kaspars_Zibarts]

Here's "Tims's" question 🙂 @Timothy_Hall 

Trying to work out number of MQ cores for my upgrade from 41k to 26k (VSX but that's not the point)

41k processors are technically twice slower according to publicly available benchmarks (Xeon Gold 6254 CPU @ 3.10GHz vs Xeon CPU E5-2658 v2 @ 2.40GHz)

Current 41k chassis runs 36Gbps accross 4 SGMs, so 9Gbps per SGM. 

Each SGM is set up with 8 HT cores for MQ and it runs at 50% CPU / 9Gbps. FWK cores average 25% atm.

Allocating 4 x 8 = 32 cores on the new 26k seems excessive, especially considering that actual CPU HW is much faster.

Having 72 HT cores in total I thought doing 24 (1/3) for MQ / SND and rest 48 for CoreXL. I need to leave room for growth of course.

Note that most traffic currently is accelerated (>85%)

Any thoughts on this? 🙂

[Andrejs__Андрей]

hi Kaspars!

may be set SND=20 HTcore and CoreXL=40 HTcore?
12HTcore reserve for SND or CoreXL to distribute in next 1-2 week?

[Kaspars_Zibarts]

I'd rather over-dimension it initially in order not to suffer from surprises when the first peak hour comes and then reduce it if needed. Instead of leaving 12 cores just heating the room.. 🙂

[Timothy_Hall]

Yeah I think a 24/48 split is a reasonable starting point for your VSX 26k if SMT is enabled. However if more than 70% of your traffic is fully accelerated, you might consider disabling SMT and doing a 16/20 split instead. SMT does not help the SND cores out at all and actually hurts them a little bit when under load in conjunction with high levels of fully-accelerated traffic. Probably won't be a huge difference either way.

How many VS's do you have?  Also if your 26k model has more than one CPU socket (I think all 26XXX models do but can't remember), try to ensure that the SND cores are assigned from the same CPU socket that is directly attached to the NIC ports you are using (this will depend on what slot they are in).  If the SND cores are assigned from a CPU socket not directly servicing the NIC slots all that traffic has to cross the bus between sockets.

[Kaspars_Zibarts]

Funny enough that's what I did with our other VSX cluster on 23800 - disabled HT. But I didn't dare to turn it off with 41k - it was strongly recommend not to..

So now I'm having split brain situation as we might turn on more advanced blades in the future pushing more traffic to pxl, yet now it's mostly accelerated.

As for MQ allocation, it does pick first cores automatically on the first socket. 

That's a 18 VS system btw, one extremely big, couple medium and the rest very small

[Kaspars_Zibarts]

Turns out we cannot turn off SMT on 26000 appliances that come with R80.30 and 3.10 kernel - you can only turn it on/off in BIOS and that's password protected and you would need to contact CP to get password.. There is no option in cpconfig anymore

So I'm guessing it's "hidden" now for a reason.

[Timothy_Hall]

Hmm, interesting design decision.  On the majority of firewalls with the "typical" blades enabled that require extensive passive and/or active streaming SMT is a definite plus.  But on firewalls with at least 70% of traffic fully accelerated, SMT doesn't help and can actually hurt performance a bit under load as two SND/IRQ threads fight each other for the same physical core. 

Having that amount of traffic fully accelerated is admittedly not common (and usually means only the Firewall and IPSec VPN blades are enabled), but having to ask TAC to turn off SMT via the BIOS seems to be a bit cumbersome.  Can we please get a little more background from R&D on this decision to remove disabling SMT from cpconfig?  Tagging @PhoneBoy