Unable to set MQ on more than 10 cores R80.30 T215 2.16 kernel

[Kaspars_Zibarts]

I was trying to add two more MQ cores to my 23800 VSX cluster (10>12) but after increasing ixgbe core count and reboot, one of 5 interfaces that had MQ configured before would remain on "Pending ON" state

cpmq set rx_num ixgbe 12

[Expert@vsx:0]# cpmq get -a

Active ixgbe interfaces:
eth2-01 [On]
eth2-02 [Off]
eth2-03 [On]
eth2-04 [Pending On]
eth3-01 [On]
eth3-02 [On]

Non-Active ixgbe interfaces:
eth4-01 [Off]
eth4-02 [Off]
eth4-03 [Off]
eth4-04 [Off]

even it says clearly that ixgbe should support 16 cores in the performance tuning admin guide

But then from command line it shows 10! 

[Expert@vsx:0]# cpmq get rx_num ixgbe
The rx_num for ixgbe is: 10

Ouch! Anyone knows if I can get more than 10 on 2.16 kernel R80.30? Is it a bug or wrong documentation

Just noticed that there is override option, has anyone tried it?

[Kaspars_Zibarts]

OK, -f did not help so I suspect since this is non HT system with total of 24 cores there must be some calculation in place that limits max cores for ixgbe MQ to 10..

[Timothy_Hall]

The driver-based max limit for ixgbe is 16 queues, but Multi-Queue is partially implemented by the underlying NIC hardware which can have lower limits.  As an example the igb driver theoretically supports up to 8 queues, but if the underlying NIC hardware is I211 (onboard interfaces for 3200,  5000, 6500, 15000, 23000) it can only have a maximum of two parallel queues per interface.  I suspect this is why you can't go beyond 10 queues in your case and attempting to go higher fails, which leads to the persistent "Pending On" state.

[PhoneBoy]

Believe the older Linux kernel supports a maximum of 5 interfaces with multiqueue.

[Kaspars_Zibarts]

Yap, that's correct, but I wanted to add a bit more horse power to SND on those five as we were suffering from voice quality in teams due to very bursty nature of traffic. CPU (60%) and interface utilisation (25%) all looked good but we still saw 0.18% RX-DRP on some of the interfaces. Increased RX ring buffers to MAX and it helped a little initially. But then as Tim has been suggesting back in March adding more SND cores may help even though they do not report 100% utilisation. So we went from 8 > 10 MQ cores first and that showed massive improvement in RX-DRP.  Since I was able to free up more cores in VSX, I wanted to go all the way to 12 but run to this limitation. So next step is 3.10 which was not available back in March for 23800 appliances 🙂

My old reference https://community.checkpoint.com/t5/VSX/RX-DRP-rx-missed-errors-on-VSX-23800-R80-30-with-mltiqueue/t...