ARP table size of 131072 entires?

[Carsten_R]

Hi,

sk43772 says, that with R80.30, the ARP table size has been extended to 131072 entires.

However, it's not working:

The SK says nothing about any HW or RAM requirements, so my test device is only a VM with R80.30, Take 111.

[Tommy_Forrest]

You need to step down to instruction number 4 - Instructions for Gaia OS.

You've never been able to set the value in the GUI above 16k.  The wording is really weird.

And I know for a fact that the ARP entries can be set to 131k entries in 80.10.  So I'm not sure where that's coming from.

[Carsten_R]

well, do I something wrong?

cp-fw01> set arp table cache-size 131072
CLINFR0409 Invalid number: 131072. Not in range 1024..16384.
set arp table cache-size 131072
--------------^^^^^^^^^^^^^^^^^
cp-fw01>

[Tommy_Forrest]

You know, I should pay closer attention.  Section 4 probably needs to be reviewed by someone.

We followed the steps in section 5 in 80.10 and were able to take the system to the full 131k.

Keep in mind, this can impact memory usage and performance.  Ideally, you should put a switch or router in front of your gateway and let that switch/router do the work if at all possible.

In our case, it was necessary because of an outdated design we were in.  After we got our ducks in a row and had proper switching/routing, we were able to go back to the normal values.  But this was on a 15600 appliance with a lot of memory.

[Carsten_R]

Thank you!
I think, there is a gap between "what is documented" and "what is possible".
sk153152 says, that take 107 will enhance the value to 131072 (PRJ-4156).
My gateway is on take 111.

[Timothy_Hall]

The ARP limit was increased to 131072 in R80.30 Jumbo HFA Take 107+.  In R80.30 GA "vanilla" the limit is still 16384.

The limit is 16384 in R80.10 & R80.20 GA "vanilla", but both R80.10 Jumbo Take 245 and R80.20 Jumbo Take 117 (both October 2019) say this:

PRJ-3795, PRHF-1778 Gaia OS Enhancement: The maximum size of the arp table was increased to 4096.

This doesn't make any sense, so pretty sure this is a typo and should say 131072.  When the older 16384 limit was in effect higher values could be "poked" directly into the running Linux kernel, but this was unsupported and could sometimes cause stability problems with the routed daemon.

In general this limit should not be arbitrarily increased unless needed, my book goes into how to make that determination.

[Tommy_Forrest]

Not sure how it could be unsupported when, in our case, TAC told us to do it.

[Timothy_Hall]

Increasing the value through the Gaia web interface and/or clish is supported, directly poking it into the Gaia kernel with something like this from expert mode:

sysctl -w net.ipv4.neigh.default.gc_thresh1=4096
sysctl -w net.ipv4.neigh.default.gc_thresh2=16384
sysctl -w net.ipv4.neigh.default.gc_thresh3=32768

or even

echo 4096 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
echo 16384 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
echo 32768 > /proc/sys/net/ipv4/neigh/default/gc_thresh3

allowed one to go far beyond the original 16384 limit and is what is not supported.

[Tommy_Forrest]

Not supported by whom?  We were explicitly told by TAC to modify the threshold values via sysctl.

[Timothy_Hall]

What I meant to say is that setting gc_thresh3 beyond the amount allowed by the clish set arp table cache-size command is not supported, regardless of how it is done.

[Carsten_R]

I've contacted meanwhile the author of sk43772 - he contacted RnD, and the increased value of 131072 entries will be pushed in the next Jumbo HF for the 3.10 kernel. It seems, that it's currently only supported in the 2.6 kernel. However, I have actual no chance to test a 2.6 kernel with R80.30....
Let's see what the future brings...