I have an R80.40 (JHF Take 48) HA pair of MDSes and a single MLS which takes all of the logs. About three weeks ago, the MLS started showing up as disconnected in SmartConsole. SOLR was crashing with "out of memory" error. The MLS had 64GB of RAM and was using a small amount of swap. After trying a number of things with TAC, none of which helped, I ended up adding another 64GB of RAM. After two days, it's started using swap.
The weird thing is, SOLR is showing as using a ridiculous amount of virtual memory. Here's a snapshot from "top"
top - 10:31:15 up 2 days, 20:46, 1 user, load average: 0.25, 0.25, 0.29
Tasks: 354 total, 1 running, 353 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.2 sy, 2.1 ni, 97.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 13151273+total, 5328456 free, 28719140 used, 97465136 buff/cache
KiB Swap: 66107468 total, 66095180 free, 12288 used. 99952048 avail Mem
PID PPID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ P COMMAND
10043 9755 admin 39 19 2.789t 0.027t 6.674g S 28.2 22.4 1323:47 8 /opt/CPshrd-R80.40/jre_64/bin/java -D_solr=TRUE -Xdump:director+
How can it purport to be using 2.8TB of VM?
# uname -a
Linux m-va-qld-ir-mdls01 3.10.0-957.21.3cpx86_64 #1 SMP Tue May 19 22:57:34 IDT 2020 x86_64 x86_64 x86_64 GNU/Linux
Any hints on what more information I can provide?