High disk consumption of /dev/mapper/vg_splat lv-log

[Vincent_SF11]

Hello,

I have the following problem: disk usage is reaching 99% of its capacity according to the path /dev/mapper/vg_splat lv-log.

Although I have deleted *.log files to free up space, after a short period of time the disk fills up again.

For R81.10

1. df -h = Display disk space statistics (df)
2. cd /var/log/opt/CPsuite-R80.10/fw1/log/ = Enter the log folder
3. # rm 2019-01* = Here you can delete old logs from SmartLog after this date

After applying the commands mentioned above, the disk is freed up; however, as I mentioned, it fills up again after a while.

Is there a way to identify which .log file is causing the disk to fill up?

I hope someone can help me with this situation, thank you in advance.

Regards!

[PhoneBoy]

First of all, R80.10 is well past End of Support.
However, an expert command like the following will help you identify large files:

find /var/log -type f -size +10000 -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | sort -nk 2,2 

[Vincent_SF11]

Hello @PhoneBoy 

My mistake, the version is 81.10.

Using the command `cd /var/log/opt/CPsuite-R81.10/fw1/log/`, I discovered that the `fw.log` file was the largest, so I backed it up and deleted it.

This action significantly reduced the disk space used. However, I noticed that the "Logs and Monitors" section of SmartConsole isn't displaying any information; it only shows the message "Error reading log number...". I restored the deleted file, but the error persists. Do you have any idea what might be happening?

Regards!

[Don_Paterson]

The fw.log file is the active log file and should always be switched before doing anything with it. 

The command fw logswitch does that. 

The resultant files can then be moved. 

For your problem you could try a log switch but then also evstop and evstart to restart the whole log server element. 

Have you reviewed the settings of the log server in SmartConsole to check the disk space management and log rotation settings?

https://sc1.checkpoint.com/documents/R81.10/WebAdminGuides/EN/CP_R81.10_LoggingAndMonitoring_AdminGu...

[the_rock]

Brother, R80.10 version? Feels like windows Vista lol. Anyway, just do what @PhoneBoy said. Say you wish to check files bigger than 500Mbs in /var/log dir, you would run below from expert mode:

find /var/log -size +500M 

[Vincent_SF11]

Hello @the_rock 

My mistake, the version is 81.10

Using the command `cd /var/log/opt/CPsuite-R81.10/fw1/log/`, I discovered that the `fw.log` file was the largest, so I backed it up and deleted it.

This action significantly reduced the disk space used. However, I noticed that the "Logs and Monitors" section of SmartConsole isn't displaying any information; it only shows the message "Error reading log number...". I restored the deleted file, but the error persists. Do you have any idea what might be happening?

Regards!

[the_rock]

I really apologize, you did say R81.10 in the post, I just assumed reading Phoneboy's response it was R80.10. Anywho, regardless, same commands would apply, so I would certainly run those to see if there are files that can be safely deleted.

[PhoneBoy]

fw.log is the current, active log file.
This should get rotated every day at midnight or when it reaches 2GB (whichever comes first).
You can also rotate it manually with the expert command fw logswitch.
If this isn't happening, I suggest engaging TAC.

[Lesley]

fw.log is the current active log file that you should not delete. You can try to force log switch: fw logswitch

You need to configure automatic clean-up for this dir. 

Open Smart Console -> open the system that is logging (fwmgt , log srv etc) and tweak these values.

1 / 2 % is maybe to little on 93gb system, consider higher value. After this is done, install database on relevant mgmt system

[the_rock]

Hey mate,

Were you able to sort this out?

[MarcuzShinz]

# Find file > 1000MB
find /var/log/ -size +1000M -type f | grep -v -E "/var/log/AutoUpdater|/var/log/CPDepInst|/var/log/CPda/repository|/conf/log_indexes/"

# Find file > 500MB
find /var/log/ -size +500M -type f | grep -v -E "/var/log/AutoUpdater|/var/log/CPDepInst|/var/log/CPda/repository|/conf/log_indexes/"

If you see file in repository you can run rm -i /path of folder/ and remove it.

[Amir_Senn]

Best way to approach this is to create a good log retention policy.

Deleting indexes and logs from SSH is not the best way to delete those.

Also they will just fill up again and you'll need to delete again.

Either set minimum desired space or maximum time to preserve logs and indexes.

[the_rock]

Hey Amir,

I assume this is what you meant?

 

[Amir_Senn]

Exactly.

Will properly delete older logs when conditions are met.

By log/index retention it will clean nightly.

By free space - will delete the oldest when reaching threshold.

[the_rock]

By log/index retention it will clean nightly, by that is it automatic?

[Amir_Senn]

Yes. Since this is also set by number of days.

[the_rock]

Thank you!