AnsweredAssumed Answered

CoreXL: only one SND core is busy

Question asked by 1fb431d2-d804-449a-bccd-5f3f95de6259 on Nov 25, 2018
Latest reply on Nov 26, 2018 by 1fb431d2-d804-449a-bccd-5f3f95de6259
  • Comment • 6

I'll put pictures because the copy paste breaks the table formatting:

 

fw ctl affinity -l -r

cpstat os -f multi_cpu (it's Sunday morning so not much traffic)

 

As you can see only the first SND (on the first CPU core) seems to be doing anything. The other SND (CPU 2/3 depending on the command) is mostly idling.

 

I suspect this may be because there is only one LACP bond interface (of two physical interfaces) on the gateway (but with many vlan subinterfaces). Therefore all the queuing of this bonding interface would maybe be handled by the first SND only, does that make sense?

If this is correct, I guess I should reconfigure CoreXL with only one SND?

 

It is an open server R77.30 gateway.

Timothy Hall
Timothy Hall Champion Nov 26, 2018 8:35 AM
Correct Answer

The SND/IRQ core is assigned to the physical interfaces, eth1 is being handled by CPU 0 while eth0 is being handled by CPU 2.  95% of your traffic is fully accelerated in the SXL path (and a relatively high templating rate of 49%) so higher CPU load is to be expected on your SND/IRQ cores, a few points:

 

  • Is SMT/Hyperthreading enabled?  (/sbin/cpuinfo)  If so you need to disable it and then assign 4 kernel instances via cpconfig (which will assign 2 SND/IRQ cores).  Having SMT enabled with such a high percentage of accelerated traffic actually hurts performance.

 

  • Since it looks like all traffic is being handled by just two physical interfaces, I'd recommend enabling Multi-Queue on both of them even though the RX-DRP rate is well below 0.1%.  This will help to spread the load out a bit among the 2 SND/IRQ cores.

 

  • Your distribution of bonded traffic between the two physical interfaces looks pretty even, so no issues there.

 

  • You definitely do NOT want to drop from 2 SND/IRQ cores down to one.

 

  • I suppose you could configure 4 SND/IRQ cores w/ Multi-Queue enabled, not sure how much that would help since there are only two active physical NICs but it probably wouldn't hurt.

 

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

See the reply in context
No one else had this question

Outcomes