If I understand your question; you are showing full physical memory utilization AND you are starting to see swap usage (>0). My inclination is to reboot/restart the gateway during a maintenance window. Linux has a tendency to used free memory to cache frequently used files to speed up the performance of the device. Unfortunately, this also decreases the "free" memory. The solution is the reboot; normally once every 6 months with a failover, actually not a bad idea since you are confirming the boot sector.
HTH!