topic Re: How to know the amount of memory allocated to the logs on the management server in Firewall and Security Management

How to know the amount of memory allocated to the logs on the management server

Oliver_222 — Fri, 05 Aug 2022 08:04:15 GMT

I need to understand how much logs are stored per week on the management server to further allocate a new amount of memory to the management server. At the moment we only see logs for the week in SmartConsole, we suspect that the problem is a lack of memory and we need to add memory to the management server so that the logs are displayed for at least 2-3 weeks.
Can you tell me how to get the amount of logs for a certain period of time from the management server (we would like to understand how many logs the management server gets in terms of memory capacity for the week)? We don't need any reports, we just need the amount of memory occupied.
1) Is it possible to do this through the command line?
2) What commands can I use to do this?
3) Can I see the files where the logs are stored on the management server and how much space they take up?
4) Is it possible to see how much log memory each Security Gateway sends per week?

Re: How to know the amount of memory allocated to the logs on the management server

Chris_Atkinson — Fri, 05 Aug 2022 08:34:41 GMT

Can you provide the output of "df -h" on the Mgmt.

From there we need to understand the log retention/rotation settings that you have configured?

Re: How to know the amount of memory allocated to the logs on the management server

G_W_Albrecht — Fri, 05 Aug 2022 08:43:47 GMT

You mean disk space, not memory (RAM) ! Yes to all your questions - we use standard linux bash commands to list directory and file size, and logs are stored in /var/log/opt/CPsuite-R8x.x0/fw1/log/ (sk66003).

Re: How to know the amount of memory allocated to the logs on the management server

Oliver_222 — Fri, 05 Aug 2022 09:00:24 GMT

Disk space is full, but the logs are stored for 3 days at 40GB per day
Maybe I should delete something from /var/log/ directory?
What settings are recommended to do?

Re: How to know the amount of memory allocated to the logs on the management server

Chris_Atkinson — Fri, 05 Aug 2022 09:23:46 GMT

Why are there parallel threads with the same screenshots for this issue?
https://community.checkpoint.com/t5/Management/Optimizing-HD-space-usage-on-the-management-server/td-p/154593/jump-to/first-unread-message

Is the Management a Virtual Machine / Open Server / Smart-1 ?

Choose one of the options below:

Free the disk space in the /var/log/ partition by deleting unnecessary files.
Follow the sk60080 - Disk space tips and tricks for SecurePlatform / Gaia / IPSO / Linux OS
Extend the /var/log/ partition e.g.

sk94671: How to add hardware resources, such as log storage, to a Virtual Machine running Gaia OS
sk95566 - Managing partition sizes via LVM manager on Gaia OS.
License & Deploy dedicated log server

Re: How to know the amount of memory allocated to the logs on the management server

Hllrdm — Fri, 05 Aug 2022 09:25:10 GMT

Hello!

We are working on this problem as a team.
In the parallel topic, we found that we have not configured Daily logs retention configuration.
As I understood, if this setting is not made, the logs are stored as the disk space is full.
We are using a virtual MDS and so far we haven't been able to add resources to the /var/log/ directory.
In the MDS settings it says Cleanup when free disk space is below 200 Gb. Am I correct in assuming that old logs are deleted when memory is less than 200 Gb?
To keep the logs longer than 3 days, we need to free /var/log/ directory. I did an analysis with the command find /var/log/ -size +1000000k -type f. There are a lot of files in the output, but we don't know if they can be deleted. What is the right thing to do to free /var/log/ directory?

sk60080 - Disk space tips and tricks for SecurePlatform / Gaia / IPSO / Linux OS Presented for versions r77.x. Can we use it for R81.10?

Re: How to know the amount of memory allocated to the logs on the management server

Hllrdm — Fri, 05 Aug 2022 09:39:04 GMT

I have an idea to free up disk space for /var/log/. I would like your advice.
If we set Cleanup when free disk space is below to 500 GB (currently 200 GB), we will free up 300 GB of memory. (If I am not mistaken that will be cleared /var/log/ directory with this setting or all disk space on the management server.)
Parallel to this configuration we will make a Daily logs retention configuration of 7 days to keep the logs and 7 days to index them. When we delete 300 GB of memory, will the management server not delete the current logs because of the Daily logs retention configuration or will it also delete the current logs? What is the best way to optimize the /var/log/ load and remove really unnecessary files from there?

Re: How to know the amount of memory allocated to the logs on the management server

G_W_Albrecht — Fri, 05 Aug 2022 09:33:13 GMT

Then please also post together as a team - makes no sense to have the same issue posted two times...

Re: How to know the amount of memory allocated to the logs on the management server

Chris_Atkinson — Mon, 08 Aug 2022 04:38:54 GMT

One member of the team is probably better spent communicating with TAC support about the issue.

You need to get an understanding of what is using the space is it only logs/indexes and how old are the files?

If it's not logs where did it come from and how important/critical is it or can it be removed and prevented from reoccuring?

Re: How to know the amount of memory allocated to the logs on the management server

Hllrdm — Fri, 05 Aug 2022 10:30:41 GMT

I output with the command find /var/log/ -size +1000000k -type f the contents of the directory /var/log/ files greater than 1 Gb. As far as I understand, these are log and index files.

I also output information on the /var/log/opt/* directory
Do we need directories with older versions? We are using R81.10.

Re: How to know the amount of memory allocated to the logs on the management server

Chris_Atkinson — Fri, 05 Aug 2022 11:16:29 GMT

If facing the behaviour described in sk114114 this won't necessarily have the desired result.

Re: How to know the amount of memory allocated to the logs on the management server

Chris_Atkinson — Fri, 05 Aug 2022 11:28:09 GMT

It's not wise to blindly remove whole locations/paths but archiving the contents of specific sub-folders including logs/index's from old versions would be my focus, if unsure please consult TAC.