/opt partition filling up

[ED]

Hi,

The /opt partition is getting filled up. What could be eating up the space? Anything that can be deleted?

[Maarten_Sjouw]

On management, on a gateway?
Version?

[HeikoAnkenbrand · Accepted solution]

See SK:
"Partition /opt has: XXXX of free space and it's lower than required: 2000 MB" error in SmartConsole...

[HeikoAnkenbrand]

Solution
If possible, add additional disk space to the root partition or clear up space.
Note: The disk space needed is different for each customer. The Release Notes specify the minimum requirements.

For a Virtual Machine:
Refer to the instructions in sk114115 - How to extend partition on Gaia.

For an Appliance:
Refer to the instructions in sk60080 - Disk space tips and tricks for SecurePlatform / Gaia / IPSO / Linux OS.

[ED]

I know about those sk. The problem is that I don't have access to sk114115 (it's a virtual machine). Just curious what is slowly eating up space. 

[Maarten_Sjouw]

You can always create and use treesize:
vi /bin/treesize:
du -k --max-depth=1 | sort -nr | awk '
BEGIN {
split("KB,MB,GB,TB", Units, ",");
}
{
u = 1;
while ($1 >= 1024) {
$1 = $1 / 1024;
u += 1
}
$1 = sprintf("%.1f %s", $1, Units[u]);
print $0;
}
'

from the OPT directory drill down in the bigger sub directories which are eating up the space.
When it is a R77.30, first check the Database revisions that you keep, more then 10 does not make much sense, but only eats up a lot of space.

[Jerry]

what about the CPUSE downloads which sometimes split over from /var/log
to /opt/CPsuite-R80/ ... check the dir, there would be lots of crap too 🙂

[HeikoAnkenbrand]

nice command:-)

chmod 770 /bin/treesize

[Maarten_Sjouw]

Been using this for a long time now, especially on MDS servers this is a very valuable command. Which customer is taking up all that space....

[ED]

Hi Maarten,

Silly question, but is it safe to run that script as it is written and displayed on this forum? Don't have time to test it in lab. 

Any big difference from this command?  du -h --max-depth=1 /opt/ | sort -n -r

It's R80.10 

[ED]

Here is what it looks like:

[Maarten_Sjouw]

It is just a simple script that I picked up somewhere a number of years ago and have been using it ever since.
treesize
20.6 GB .
12.5 GB ./CPshrd-R80
5.0 GB ./CPsuite-R80
1023.7 MB ./CPrt-R80
936.5 MB ./CPda
283.0 MB ./CPInstLog
194.8 MB ./KAV
95.1 MB ./CPmds-R80
93.8 MB ./CPinfo-10
63.2 MB ./OSHF
60.2 MB ./CPSFWR77CMP-R80
56.4 MB ./DDR
40.6 MB ./CPUserCheckPortal
38.9 MB ./CPSG80R75CMP-R80
38.4 MB ./CPdiag-R80
32.3 MB ./CPNacPortal
29.7 MB ./CPSG80CMP-R80
25.5 MB ./CPR76CMP-R80
24.9 MB ./CPR7540VSCMP-R80
24.8 MB ./CPR7540CMP-R80
23.2 MB ./aspam_engine
23.0 MB ./CPEdgecmp-R80
20.2 MB ./CPSmartLog-R80
18.6 MB ./CPR77CMP-R80
18.1 MB ./CPR7520CMP-R80
17.7 MB ./CPR75CMP-R80
16.3 MB ./CPR71CMP-R80
15.0 MB ./CPNGXCMP-R80
11.5 MB ./postfix
7.5 MB ./CPcdt
5.0 MB ./CPvsec-R80
4.4 MB ./MegaRAID
124.0 KB ./SecurePlatform
84.0 KB ./CPwrp
8.0 KB ./CPshared

The difference is in the sorting, this has the biggest in the top, your command does not.

[Colin_Campbell1]

Hi,

I have two R80 management servers that continually suffer this problem. Fortunately they're in AWs so I can just keep adding disk. In my case the culprit is Postgres. It lives under CPshrd-R80 using 28GB out of 37GB. More specifically:

/opt/CPshrd-R80/database/postgresql/data/base contains these directories (du -sh *)

6.0M 1
26G 1051516
21M 1051517
5.9M 12000
6.0M 12005
4.0K pgsql_tmp

There are approximately 2700 files in 1051516 with enough of them around 1GB in size.

Does anyone know how to reduce this space? Removing revisions doesn't do it. 

Colin

[Prashant_YADAV1]

Hi Maarten,

Thanks for the script , i have no knowlege on the scipt part , can you please help me how to create this script and run it ..

Thanks ..

[Maarten_Sjouw]

You log in to the server in expert mode, then type:
vi /bin/treesize:
Now you first press "i" and then copy and paste the text below into the screen (not the lines)
-------------------------------------------
du -k --max-depth=1 | sort -nr | awk '
BEGIN {
split("KB,MB,GB,TB", Units, ",");
}
{
u = 1;
while ($1 >= 1024) {
$1 = $1 / 1024;
u += 1
}
$1 = sprintf("%.1f %s", $1, Units[u]);
print $0;
}
'
----------------------------------
now press the <Esc> key then ":wq <Enter>"

[Rashid_Anwar]

Why does the /opt shows different space for the free disk in dashboard Monitor and on the CLI. "/" partition shows different data for available space.

[Tomer_Noy]

If it's the Postgres DB that is growing in size, then I can suggest two actions:

1) Run "Purge Revisions" to free up space of old DB revisions

2) If #1 doesn't solve the problem, you should open a TAC ticket to reduce your DB size. There was an issue that caused some unnecessary growth and we have a utility to resolve that. In later jumbo versions, the growth shouldn't happen anymore. 

[Colin_Campbell1]

Hi,

I'm not sure if this was directed at my problem or not but I'll answer anyway. I'm running R80 (not for much longer)

1) Deleting old revisions did not free up any significant space

2) I opened a TAC case but the engineer was unable to free up any space. The tools available in R80 aren't anywhere near as useful or powerful as those in later revisions.

In the end I just added another 100GB. It's easy in AWS.

Colin

[Tomer_Noy]

R80 is a pretty old version by now. Definitely worth moving to at least R80.10, or preferably R80.20/R80.30.

Indeed some of our tools are only applicable for R80.10 and up, as is the case with the tool to free up the DB space.

[emreturkmenler]

Hi,

We have the same issue as /opt/CPshrd-R80.30/database/postgresql/data/base/ folders keep on growing as we have upgraded from R80.30 Take 140 and growing speed increased in this version.
Are there any SK's or tool names you can give to ask for the cp support as this is an old issue and they couldn't solve it with any tools before.

Thank you

[Eran_Habad]

Hi @emreturkmenler,

My name is Eran and I'm a manager in the R&D of Check Point, responsible for the core of the Management server and its health, among other things. @Tomer_Noy mentioned in one of his previous comments an issue we had few months ago, causing inflation to the DB - but it was fixed in JHF of R80.10 and R80.20 and the fix is also part of the GA version of R80.30, so you have the fix for that issue. As you still experience noticeable growth which is even faster now, we need to get to the bottom of it. I suggest you will open a ticket to TAC and add my name to the ticket, so it will reach my personal attention. We need to understand what is growing and why, and decide if it's reasonable or not. As you can imagine, we still have more things to do in order to reduce the size of the DB with tools and SKs (some of them are exposed only internally to TAC and are not public).

Eran

[emreturkmenler]

Thank you for your involvement, I will add your name to to SR.
As with the faster growth of the DB we have started have issues with our management server. I am not sure as CP support is working on our case but we cannot push policies to some of our devices as the policy installation to the other gateways take much more time than before. I am not sure if you had any complaints with the latest take but it would be great if you could inform as I am planning to downgrade to our earlier version if the support cannot solve the issue.

[Eran_Habad]

@emreturkmenler thanks for elaborating on the policy installation issue, I'm sorry for your experience. I will contact you offline to have the SR number and I guarantee me and my team will closely monitor the progress of your case. Hoping for a fast resolution.

Eran

[TA_05]

Hey @Eran_Habad , I was running into this same issue since our upgrade to R80.20 (Database just keeps growing) and have been waiting a resolution on our already opened SR, any chance you would allowed me to tag you in our SR for assistance?

Thanks in advance, 

Tyler

[Eran_Habad]

Hi @TA_05, In R80.20 JHF 103 we added a fix which prevents unnecessary inflation of the DB. You can see it listed in the JHF SK under take 103 with PRJ-1403: "The Multi-Domain Server database size grows significantly causing operations like 'mds restore' and HA full sync to take a long time." What JHF are you using?

While the fix prevents further rapid growth, if the DB has grown already TAC have a procedure to shrink it down using a procedure they are qualified to perform. I'm sure TAC are on top of your SR and working to resolve it. If you feel further assistance is needed reach me out privately with the SR number and I'll have a look.

[TA_05]

Thanks again for your response @Eran_Habad , we have now worked this to resolution and were able to get the old DB files cleaned up using the vacuum tool and it is now working as expected.

[emreturkmenler]

Thank you for your help and involving Eran.

[LucasCosta]

Hi,

You can also to take a look at the sk114879 (https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...)