Ah, now I understand! That was the missing piece: in order to change the default number of SNDs that are available, I need to use fw ctl affinity to reduce the total number of cores for VSes. Whatever is left is a potential SND candidate.
To make sure I do understand, here is an extreme example!
23500 with 48 cores. I want three VSes - VS0 with 4 coreXL; VS1 with 10 coreXL and VS2 with 10 coreXL. So, I could assign 24 cores to coreXL...
fw ctl affinity -s -d -vsid 0-2 -cpu 24-48
Leaving 24 CPUs 'free' and then use sim affinity commands to assign individual CPU cores to interfaces.
Yes, 24 cores for SND is pointless and even with multi-queue unlikely to ever be needed, but I could do it?
Oh, I suppose that there is one other question: You mentioned that the CoreXL cores are spread between the VSes. So in my example above each CPU core 24 to 48 would have an active instance running as the total number of coreXL instances is 4+10+10. But if I had VS1 only configured for 5 coreXL instances that would be 4+5+10, so there would be 24 possible coreXL instances, but only 16 used coreXL instances?
After reading Michael Endrizzi's tuning guide, I can see that it is possible to assign different VS coreXL instances to individual CPUs, but it seems like there is usually no point to do so? You'd likely end up breaking things more than tuning them,