Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Young_Wook_Choi
Contributor

SMEM's "Failed Allocation" value is very high.

Hello,

All traffic going through the firewall was cut off.
The firewall is using the IPS blade function with the FW blade.  (R80.10)
CPU usage is not very high. However, the memory usage seems very high.

The output of the fw ctl pstat command is shown below.

Young_Wook_Choi_0-1608028539040.jpeg

This firewall has an uptime of 3 weeks.
SMEM's "Failed Allocation" value is very high. (It is more than about 4,600,000)
This count was recorded for only 3 weeks.

This system has 16GB of memory (RAM).

When looking at the CPVIEW output, memory usage is always high as shown below.

1.jpg

2.jpg

3.jpg

Could a session be disconnected due to high memory usage?

I opened a Case with this problem, but TAC doesn't help. Rather, they say the system is normal.

Will upgrading this system's physical memory solve this problem?

Over the past few weeks, we've been experiencing a problem of disconnecting service sessions several times.

 

0 Kudos
Reply
6 Replies
Young_Wook_Choi
Contributor

Aggressive Aging feature is active. (Default)
Is there a possibility that the service session is disconnected by this feature? (If you guess from the above memory usage)

If the gateway has ever had this function active, where can I check it? (/var/log/messages? or fwd.elg?)

Could this issue be related to this feature?

0 Kudos
Reply
Timothy_Hall
Champion
Champion

Depending on the load, 12GB of memory being used by the kernel of of 16GB total may be high.  Could indicate a memory leak, would recommend ensuring you have the latest GA R80.10 Jumbo HFA loaded for starters as there are almost certainly fixes for memory leaks contained within it.  Please provide the full output of fw ctl pstat as that will provide connection statistics and sync info as well.  It doesn't look like your firewall is dipping very far into swap space, but please provide output of free -m to see more detailed memory usage and help determine if a RAM upgrade is needed.

Once a connection is up and running, I think all necessary memory allocations for state table space have been performed so I don't believe an existing connection could be killed by a memory shortage, but new ones could certainly be prevented from starting which depending on how an application might be utilizing multiple connections could look like an existing connection getting killed.

"Max Capture: Know Your Packets" Video Series
now available at http://www.maxpowerfirewalls.com
0 Kudos
Reply
Young_Wook_Choi
Contributor

Thanks for your reply.
TAC provided us with a script that could detect memory leaks. And they said this issue was not a memory leak.

free_m_t.PNG

fw ctl pstat.PNG

 

0 Kudos
Reply
Timothy_Hall
Champion
Champion

You have a pretty large number of concurrent connections for a system with only 16GB of RAM (176k current/313k peak); it looks like the kernel is using almost all the RAM (and trying to get more) and the firewall's processes are having to dip into swap space to the tune of 1.3GB. 

Assuming a memory leak is not present it does look like a RAM upgrade to 32GB will help here.  Perhaps a later version of code will utilize kernel memory a bit more efficiently, but it looks like you just have a very busy firewall there that could use more memory.

"Max Capture: Know Your Packets" Video Series
now available at http://www.maxpowerfirewalls.com
0 Kudos
Reply
Young_Wook_Choi
Contributor

Thank you for your advice. We are planning an upgrade to 64gb of RAM. Also, an upgrade to R80.40 is planned. We hope this plan helps us.

0 Kudos
Reply
Benedikt_Weissl
Advisor

https://community.checkpoint.com/t5/General-Topics/R80-x-Performance-Tuning-Tip-Connection-Table/td-... See Tip 4 and Tip 7 for advice on how to lower session lifetime, this could also reduce the size of the connection table. Please use caution when adjusting session lifetime.

0 Kudos
Reply