Memory usage on 9000 series appliances

[David_C1]

Hi Checkmates,

Over the course of 2025, we replaced 8 clusters of 6800/6900 appliances with 9400 appliances. We are running R81.20 and the latest recommended JHFA. Two things I've noticed: the 9400 appliances use significantly more memory than the 6800/6900 appliances did. Both the 6800/6900 and the 9400 appliances have 32G of memory. The 6800/6900 were consistently running 25-40% memory usage (as shown in SmartConsole), the 9400 appliances are running from 45-65% memory. Same blades, policy etc. through the change from the 6800/6900 to the 9400 appliances. I am guessing this is due to going from KPPAK to UPPAK? Can anyone confirm similar behavior?
The second thing I've noticed (and have tracked) is the slow, consistent increase in memory usage of the 9400s. Nothing outrageous, maybe a percentage point every 2-3 days or so. And it never decreases. There have been a number of memory related fixes in the latest Jumbo HFAs, seems there still may be a leak somewhere. Again, can anyone else confirm same behavior? I have not engaged TAC or gone through sk35496 (honestly as I customer I should not have to chase down memory leaks...). Before I go down either of those paths, just curious if others have observed the same.

Thanks,

Dave

[PhoneBoy]

How specifically are you tracking "free" memory?
If you are using free -m, the "available" column is actually what you need to track.
This includes the "free" memory (which hasn't been allocated) plus previously allocated memory that can be freed if needed.

[David_C1]

I am simply going by what is reported in the Gateway & Servers view in SmartConsole. I have not used anything more advanced (free -m, cpview) than that.

[PhoneBoy]

Not sure if SmartConsole is pulling the right information, but cpview is probably a better way to track this over time.

[David_C1]

Well...I appreciate that cpview might give more detailed information, but if you are telling me that the data provided by SmartConsole is not reliable, that's another problem (and not my problem, but Check Point's).

Dave

[Lesley]

smart view monitor is fist quick check if you see something you don’t trust you troubleshoot further with cpview, free -m fw ctl pstat etc.

So it is incorrect to say it is unreliable but sometimes you need to search deeper. Memory monitoring on Linux is always bit difficult because the nature of the product likes to use all the memory it can. That why you need to check the swap memory , aggressive aging activation. Cpview -t you can go back into the history to see any memory leaks. Note: sometimes it looks like memory leak, seeing that memory increases every day. But after 1 point systems cleans itself up and memory decreases. Just watch for any aggressive aging

[the_rock]

Agree. I would not say either its unreliable (its not fair to label it as such), but definitely better to explore other venues.

[David_C1]

Thanks everyone for your replies. I was using the memory reporting in SmartConsole because I'm tracking this over 10 or so gateways, so it gave me a quick way to look at all of them. I understand for more detailed information to use cpview or built in Linux tools.

Dave

[the_rock]

Hope you got some useful info from it!

[Ilya_Yusupov]

Hi @David_C1,

Yes in UPPAK comparing to kppak you will see memory grow, it is because in uppak mode we allocating memory during init stage as a pool of memory called mbuf pool for a traffic, while in kppak it was part of skb mechanism.

Hope this clarify the diff.

Thanks,

Ilya

[the_rock]

Apart from great point @Ilya_Yusupov  (AMAZING guy btw, had to say that), I would say since you mentioned they are on recommended jumbo fix, can you see what process in particular (if any) would be consuming most memory?

What does free -g show?

Example from my lab:

[Expert@CP-GW:0]# free -g
total used free shared buff/cache available
Mem: 15 6 4 0 3 7
Swap: 7 0 7
[Expert@CP-GW:0]#