Management logs

[ibrown]

Hello All,

I've noticed that since I've upgraded to an R81.20 smartcenter server, we are only keeping 8 days of the consolidated logs available for querying. The log server, which is also the smartcenter server,  has the local logs policy set on it for 60 days or purge when less than x space. The log partition has more than x and the logs show no triggering of the policy, so I don't know why it's rolling at such a short period.

Anyone else seen this and fixed it ?

Many thanks

Ian

[the_rock]

Hey Ian,

I had been using R81.20 more than a year now and had never seen this issue. You noticed it after the upgrade you said?

Andy

[ibrown]

Hello,

well I've noticed it in the last month, but it might have been since the upgrade. I tend not to need to look at logs longer than today/yesterday very often luckily.

[the_rock]

I saw answer @Amir_Senn just gave and I would also verify the same.

Best,

Andy

[PhoneBoy]

What version did you upgrade from?
We changed the log indexing in one of the R81.x versions, which requires triggering a manual reindex after doing an upgrade.
Otherwise, only logs from the day you performed the upgrade forward will be indexed.

[ibrown]

Hello,

it was R80.40. it's not stuck on that date though, it's a rolling 8 days. Which I cannot find anywhere.

[Amir_Senn]

2 things to check:

a. How much free storage you have on logs partition.

b. Try to open manually a log file. It is possible you still have the logs but no indexes.

[ibrown]

Hello,

We are currently higher than the policy (not that it appears to be paying any attention to it) with just under 11gb free and the daily consolidated log is around 1gb. I checked if I could open any other files and the number are the same, so it's not that it isn't indexing them. It really does seem to be pruning them.

thanks

[Amir_Senn · Accepted solution]

It's possible to see if they were deleted in $FWDIR/log/fwd.elg

If this is cleaning due to disk space you should see messages similar to the following:

CCyclicLogging::_moveFile: moving file: YYYY-MM-DD_HHMMSS.log, for deletion....
FWLOG_MAINTENANCE - RemoveFilesFromCLDir: removing file: YYYY-MM-DD_HHMMSS.log from dir....

Other things you can do is use log forwarding to another server with more storage.

One last thing, upgrade packages and JHF are uploaded to same partition as the logs so if you have a lot of old packages you can consider deleting them.

[ibrown]

Hello Amir,

you are correct it is deleting them. Hmm, now I need to work out why.

[Chris_Atkinson]

Did you reindex old log files post upgrade or remove them?

PMTR-60610: R81 includes the new mechanism for log indexing.

Before you upgrade a Management Server or Log Server that uses an external storage device to keep the log data, you must follow the instructions in sk66003 to change the location of the existing log indexes.

This applies only if it is necessary to keep the existing log indexes and use them after the upgrade.

This applies to Security Management Server, Log Server, Multi-Domain Security Management Server, Multi-Domain Log Server, SmartEvent Server, StandAlone Server. R81

GNG-1259,PMTR-52941 R81 includes new logs indexing mechanism, so when upgrading Management server/Log Server/Multi-Domain Server/Multi-Domain Log Server/SmartEvent from R80.x, old log indexes are not upgraded.

The indexing mechanism will re-index the last 24 hours automatically. To increase the period of offline indexing (how far in the past to re-index the logs), see sk111766.

[ibrown]

Hi Chris,

during the upgrade, I wasn't too fussed, so I let it do it's own thing. It's since then it's been purging the logs.

[ibrown]

Thank to all. I've expanded /var/log and we are good. I think my problem was that I was looking at the free space in /var/log during the day after it has purged and of course it was dropping below the limit by midnight and the purge was kicking off.

So many thanks for the help all !

[the_rock]

Thanks for sharing mate, glad its sorted out!

Andy