Maestro Orchestrator 140 - disk space

[Lesley]

Hi everyone,

Anyone has any good idea / tips regarding disk space of a MHO unit?

The disk inside is quite small and every upgrade is an issue. I think soon enough we are forced to perform a clean install to have enough space for an upgrade. Especially if you update frequently from Jumbo to Jumbo the disk usages piles up. The older the system is the more space is in use, old files / logs etc etc.

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_splat-lv_current 20G 9.3G 11G 47% /
/dev/sda1 290M 107M 169M 39% /boot
tmpfs 7.7G 51M 7.7G 1% /dev/shm
/dev/mapper/vg_splat-lv_log 20G 17G 3.6G 83% /var/log

du -a /var/log | sort -n -r | head -n 100
17052796 /var/log
9773440 /var/log/CPda
9752136 /var/log/CPda/repository
3951044 /var/log/AutoUpdater
2928604 /var/log/CPda/repository/CheckPoint#Major#All#6.0#5#4#R81.20_ivory_main_T631/Check_Point_R81.20_T631_ScalablePlatform_Upgrade.tgz
2928604 /var/log/CPda/repository/CheckPoint#Major#All#6.0#5#4#R81.20_ivory_main_T631
2597784 /var/log/CPda/repository/CheckPoint#CPUpdates#All#6.0#5#4#BUNDLE_R81_20_JUMBO_HF_MAIN#53/Check_Point_R81_20_JUMBO_HF_MAIN_Bundle_T53_FULL.tgz
2597784 /var/log/CPda/repository/CheckPoint#CPUpdates#All#6.0#5#4#BUNDLE_R81_20_JUMBO_HF_MAIN#53
2263056 /var/log/opt
2214708 /var/log/CPda/repository/CheckPoint#CPUpdates#All#6.0#5#4#BUNDLE_R81_20_JUMBO_HF_MAIN#41/Check_Point_R81_20_JUMBO_HF_MAIN_Bundle_T41_FULL.tgz
2214708 /var/log/CPda/repository/CheckPoint#CPUpdates#All#6.0#5#4#BUNDLE_R81_20_JUMBO_HF_MAIN#41
2048268 /var/log/AutoUpdater/repository
2011040 /var/log/CPda/repository/CheckPoint#CPUpdates#All#6.0#5#3#BUNDLE_R81_10_JUMBO_HF_MAIN#110/Check_Point_R81_10_JUMBO_HF_MAIN_Bundle_T110_FULL.tgz
2011040 /var/log/CPda/repository/CheckPoint#CPUpdates#All#6.0#5#3#BUNDLE_R81_10_JUMBO_HF_MAIN#110

cd CheckPoint#CPUpdates#All#6.0#5#3#BUNDLE_R81_10_JUMBO_HF_MAIN#110
ls -lah
total 2.0G
drwx------ 2 admin root 67 Jan 3 2009 .
drwxr-xr-x 7 admin root 330 Apr 22 2009 ..
-rw-r--r-- 1 105 80 2.0G Jul 24 2023 Check_Point_R81_10_JUMBO_HF_MAIN_Bundle_T110_FULL.tgz

cd CheckPoint#Major#All#6.0#5#4#R81.20_ivory_main_T631
ls -lah
MHO01total 2.8G
MHO01drwx------ 2 admin root 66 Jan 3 2009 .
MHO01drwxr-xr-x 6 admin root 258 Apr 2 12:07 ..
MHO01-rw-r--r-- 1 105 80 2.8G Jan 12 2023 Check_Point_R81.20_T631_ScalablePlatform_Upgrade.tgz

rm -r CheckPoint#CPUpdates#All#6.0#5#3#BUNDLE_R81_10_JUMBO_HF_MAIN#110
rm -r CheckPoint#CPUpdates#All#6.0#5#4#BUNDLE_R81_20_JUMBO_HF_MAIN#41
rm -r CheckPoint#Major#All#6.0#5#4#R81.20_ivory_main_T631

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_splat-lv_current 20G 10G 11G 50% /
/dev/sda1 290M 68M 208M 25% /boot
tmpfs 7.7G 51M 7.7G 1% /dev/shm
/dev/mapper/vg_splat-lv_log 20G 9.7G 11G 49% /var/log

[JaAnd]

Hi,

have you tried this SK?

https://support.checkpoint.com/results/sk/sk182325

For my MHOs this was very helpful, nevertheless it is sad, that CP has decided to develop universal JHT packages, their size became an issue on those biffed up switches...

[Wolfgang]

@Lesley  I’m always cleaning the repository. That solves the disk space issue but always it‘s a bad feeling doing this.

[emmap]

As of R82 the regular DA client is supported on MHOs, which will benefit from better automatic disk cleanup routines. I can't say whether it will actually solve the issue, but it should help at least.

[the_rock]

Not sure how effective commands like below would be on maestro, but its what I always do to check what files can be deleted.

Example:

find /opt -size +500M (check for files bigger than 500Mbs in root dir)

or

find /var/log -size +400M (size can be anything)

Andy

[Jones]

When I clean up the MHO before an upgrade, I use the following command to find files that are over 50M:

find / -type f -size +50M -exec ls -lh {} \; 2> /dev/null | awk '{ print $5": " $NF }' | sort -nk 2,2

To only find files in the /var/log and look for +100M use:

find /var/log -type f -size +100M -exec ls -lh {} \; 2> /dev/null | awk '{ print $5": " $NF }' | sort -nk 2,2

I mostly find the /var/log/opt/CPshrd-R81.20/cpview_services/cpview_services.dat file to be to large so I remove this file for sure. Occording to sk101878 - CPView Utility this is the history database.

For sk182325, I asked Check Point to update the SK to include cleaning up the cpview_services.dat file.

[yukaia · Accepted solution]

I just went through this with my MHO-175s. The solution that my CSE and I came up with was to clean up the /var/log/AutoUpdater/offlineRepository as it still contained all of the old JHF Takes that we had installed, made sure to leave the last two which were the only ones that showed up in CPUSE.

Then we expanded the LV by 5GB using the following command.

lvresize -L25G vg_splat/lv_log

and then expanded the fs partition using this command.

xfs_growfs /var/log

Doing all of that gave us just enough space to upload the R82 upgrade package while still leaving us with 32GB of unused space for roughly 2 snapshots.

[the_rock]

Good to know, thank you!

Andy

[the_rock]

@yukaia This is GOLD!! Cant believe I never knew of this before...just tested it in eve-ng lab, worked like a charm, no reboot needed either, amazing.

Andy

[yukaia]

@the_rock Big shout out to my CSE on this, he reached out to his contacts and figured that process out. When she showed me the commands I was like doh, I've done this exact thing on other CentOS/Alma Linux boxes that I have.

[the_rock]

Like my good friend always says when someone shows you something so genius "Andy, did you ever realize that 99% of people on this planet peel the banana the WRONG way?"...me "What do you mean?"...him "Well, youtube "How monkeys open the banana" and you will see"

Go figure LOL

Andy

[the_rock]

I hope you dont mind I made a post about it, tagged your username, as its well deserved!

Andy

https://community.checkpoint.com/t5/Security-Gateways/Tip-for-extending-disk-space-without-entering-...

[yukaia]

Not at all, it's fantastic information to have, best to shout it from the rooftops.

[the_rock]

Agree 100%, thanks again, its genius!

Andy

[dj0Nz]

Thanks a lot! This one and sk182325 made my day! 😅

One thing: Before expanding /var/log, always check free PEs using pvdisplay and make sure you leave enough free space for one (IMHO you won't get more on MHOs) snapshot... 

[JH_Ranger]

Hmm. I have struck the same issue, even after following sk182325 and deleting the offlineRepository files, I'm coming up 2GB short. TAC said that the only way is to resize the partition using lvm_manager. As I'm managing the system remotely, I'd rather not boot into maintenance mode and play around with partitions.

Does anyone expect CP to release a smaller R82 package just for Scalable Platforms in the near future? Other than that, the only other option is to delete the currently installed JHF from the /var/log/CPda/repository, which may break things....

[Alex-]

Yeah, same issue. Bunch of MHO-175 with not even enough space for an hotfix, so an upgrade is impossible.

With the ever-increasing size of images, this limited disk space is an issue.

[Alex-]

We worked with TAC and it looks like the use of the lvm tool is the approach.

We will probably wait that the next JHF's to become recommended before going to R82, to limit the cadence of installations on these systems.

[AkosBakos]

What does #lvm_manager say? Maybe there is some free space?