The log repository quota has been exceeded.

[Vladimir]

Just run into it and the only references available are not really applicable ("log_sys_message: The log repository quota has been exceeded. No file could be deleted" alert in log... ).

SMS is R80.20 on ESXi VM.

Any ideas on how to remedy this issue?

[Expert@SMS8020:0]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_splat-lv_current 10G 6.0G 4.1G 60% /
/dev/sda1 291M 25M 251M 9% /boot
tmpfs 3.9G 4.0K 3.9G 1% /dev/shm
/dev/mapper/vg_splat-lv_log 25G 920M 25G 4% /var/log
[Expert@SMS8020:0]# ls -l /var/log/messages*
-rw------- 1 admin root 281752 Nov 8 13:36 /var/log/messages
-rw-rw---- 1 admin root 1060593 Nov 8 10:15 /var/log/messages.1
[Expert@SMS8020:0]# /bin/log_start list
Index File Max-Size Back-logs
0) messages 2097152 10
1) maillog 65536 4
2) send_alert 65536 10
3) blade_config 65536 15
4) pingable_hosts 65536 10
5) log_servers 65536 3
6) routing_messages 64536 4
7) wtmp 65536 4
8) lastlog 262400 4
9) secure 64536 4
10) auditlog 65536 4
11) asgaudit.log 102400 4
12) smd_smo.log 524288000 1
13) smd.log 1048576 4
14) cpstart.log 1048576 4
15) CPbackup.elg 64536 4
16) backup_logs.elg 64536 4
17) snapshot.log 64536 4
18) fwd.elg 1048576 4
19) dtlsd.elg 1048576 4
20) dtpsd.elg 1048576 4
21) sdsd.elg 1048576 4
22) aftpd.elg 1048576 4
23) atelnetd.elg 1048576 4
24) ahttpd.elg 1048576 4
25) unifiedd.elg 1048576 4
26) aclientd.elg 1048576 4
27) ahclientd.elg 1048576 4
28) arlogind.elg 1048576 4
29) asessiond.elg 1048576 4
30) asmtpd.elg 1048576 4
31) aufpd.elg 1048576 4
32) genericd.elg 1048576 4
33) lhttpd.elg 1048576 4
34) pingd.elg 1048576 4
35) mdq.elg 1048576 4
36) snauth.elg 1048576 4
37) cp_http_server.elg 1048576 4
38) cpwd.elg 1048576 4
39) cpwd.elg 1048576 4
40) stormc.elg 1048576 4
41) cprid.elg 1048576 4
42) rtmd.elg 1048576 4
43) fwm.elg 1048576 4
44) fgd50.elg 1048576 4
45) boot.log 1048576 4
46) server.log 10485760 10
47) access.log 1048576 10
[Expert@SMS8020:0]#
[Expert@SMS8020:0]#
[Expert@SMS8020:0]# cat /etc/cpshell/log_rotation.conf
/var/log/messages 2097152 10
/var/log/maillog 65536 4
/var/log/send_alert 65536 10
/var/log/blade_config 65536 15
/var/log/pingable_hosts 65536 10
/var/log/log_servers 65536 3
/var/log/routing_messages 64536 4
/var/log/wtmp 65536 4
/var/log/lastlog 262400 4
/var/log/secure 64536 4
/var/log/auditlog 65536 4
/var/log/asgaudit.log 102400 4
/var/log/smd_smo.log 524288000 1
/var/log/smd.log 1048576 4
$CPDIR/log/cpstart.log 1048576 4
/var/log/CPbackup.elg 64536 4
/var/CPbackup/log/backup_logs.elg 64536 4
/var/log/image/snapshot.log 64536 4
$FWDIR/log/fwd.elg 1048576 4
$FWDIR/log/dtlsd.elg 1048576 4
$FWDIR/log/dtpsd.elg 1048576 4
$FWDIR/log/sdsd.elg 1048576 4
$FWDIR/log/aftpd.elg 1048576 4
$FWDIR/log/atelnetd.elg 1048576 4
$FWDIR/log/ahttpd.elg 1048576 4
$FWDIR/log/unifiedd.elg 1048576 4
$FWDIR/log/aclientd.elg 1048576 4
$FWDIR/log/ahclientd.elg 1048576 4
$FWDIR/log/arlogind.elg 1048576 4
$FWDIR/log/asessiond.elg 1048576 4
$FWDIR/log/asmtpd.elg 1048576 4
$FWDIR/log/aufpd.elg 1048576 4
$FWDIR/log/genericd.elg 1048576 4
$FWDIR/log/lhttpd.elg 1048576 4
$FWDIR/log/pingd.elg 1048576 4
$FWDIR/log/mdq.elg 1048576 4
$FWDIR/log/snauth.elg 1048576 4
$CPDIR/log/cp_http_server.elg 1048576 4
$CPDIR/log/cpwd.elg 1048576 4
$CPDIR/log/cpwd.elg 1048576 4
$FWDIR/log/stormc.elg 1048576 4
$CPDIR/log/cprid.elg 1048576 4
$FWDIR/log/rtmd.elg 1048576 4
$FWDIR/log/fwm.elg 1048576 4
$FGDIR/log/fgd50.elg 1048576 4
/var/log/boot.log 1048576 4
$FWDIR/log/c-icap/server.log 10485760 10
$FWDIR/log/c-icap/access.log 1048576 10
[Expert@SMS8020:0]#
[Expert@SMS8020:0]#
[Expert@SMS8020:0]#
[Expert@SMS8020:0]# cpd_sched_config print
Task: "RotateLogs"
Command: /sbin/cp_logrotate
Arguments:
Interval: 100
Active: true
RunAtStart: true
Task: "CPDiag"
Command: /opt/CPdiag-R80.20/bin/cpdiag
Arguments: --periodic
Interval: 55125
Active: true
RunAtStart: true
Task: "CPDiagVarLogScript"
Command: /opt/CPdiag-R80.20/tools/cpdiag_facilitator
Arguments: -s /opt/CPdiag-R80.20/scripts/var_log_collector -p 1048576 10485760 1048576 /opt/CPdiag-R80.20/signatures/mkeys.dat /opt/CPdiag-R80.20/signatures/st1.dat 10485760 -d /opt/CPdiag-R80.20/signatures/mkeys.dat /opt/CPdiag-R80.20/signatures/st1.dat
Interval: 60
Active: true
RunAtStart: false
[Expert@SMS8020:0]#

[AlekseiShelepov]

Hmm... 

My colleague told me today that the same thing happen on our server, but it is a totally different setup - MDS, R77.30, Smart-1 50.

There is the same error message in logs on one of CMAs (or DMS), there are no logs from firewalls, /var/log has 10% of free space, logging is properly configured and automatically deleted. On the secondary CMA everything is fine, logs are visible. We tried to manually cleanup some more space and switch the log file, didn't help. The next thing we would like to try is just to restart this CMA. We didn't spend much time on the issue today and decided to continue tomorrow.

[Vladimir]

Well, in your case with 10% free space remaining, the issue described here may be the reason:

"log_sys_message: The log repository quota has been exceeded. No file could be deleted" alert in log... 

In my case, it is a brand new lab setup with barely any logs in it.

But please do let me know if and when you'll get to the bottom of your issue. Who knows, perhaps they are related or I could infer something from your resolution.

Meanwhile, I'll try to migrate export-import in a new management VM and hope that'll do the trick.

[Vladimir · Accepted solution]

It actually was the "log_sys_message: The log repository quota has been exceeded. No file could be deleted" alert in log... but with a caveat: the log indicating this issue does not show where it has arrived from:

Since I was working in VSX environment, EACH cluster member as well as EVERY VS' property had to be adjusted to the recommended values to solve the problem:

P.S. the R80.20 screenshot in sk is incorrect: it is from a management server, not the gateway.