index log files after moving logs from one log server to another log server

[checkmate-Go]

We recently replaced our open server hardware that was running the dedicated log server.  I moved only the .log files (not other log files) from /var/log/opt/CPsuite-R81.20/fw1/log/2024-11-*.log to /var/log/opt/CPsuite-R81.20/fw1/log/ on another server ranging from 2024-11-1 to 2024-11-19.

I’m wondering how indexing works after this transfer. Does it happen automatically, or do I need to manually re-index the logs? I only need the last 14 days of indexed logs.

It seems like the article I found is relevant, but I wanted to confirm: is moving just the .log files sufficient, or should I have moved the other log files as well?

https://support.checkpoint.com/results/sk/sk111766

[PhoneBoy]

While I believe the log files alone are sufficient, the other files have to be rebuilt if they are not transferred.
It's better to move them all.

And yes, you will have to manually reindex the logs after moving files into the directory.

[checkmate-Go]

what does "rebuilt if they are not transferred mean"?

I just copied all the .log files and followed instruction from this sk artice:

https://support.checkpoint.com/results/sk/sk111766

I am not sure if it indexed logs or not. How can I verify that? How long does it generally take to re-index logs?

[AkosBakos]

Don't forget, indexing the logs takes a while.

And don't forget evstop, and evstart. Ususallly thats why we don't apply this after upgrades. Is not worth the time.

Akos

[PhoneBoy · Accepted solution]

There are "pointer" files that are necessary for working with the logs (thus why the instructions state to copy $FWDIR/log/*.log* instead of $FWDIR/log/*.log.
I was mistaken that they are rebuilt automatically, you'd have to use fw repairlog (from the CLI) to do that.

As for whether the logs will get imported/indexed automatically, I would assume this would not be the case if you simply copied the files over.
Starting the reindexing process (as described in the SK) ensures this will be done.
The time will depend on the amount of logs, overall management/log server load, etc.
You will notice some increased CPU during this time, which will "back off" when other management processes need to use the CPU.
This is normal, expected behavior.

[checkmate-Go · Accepted solution]

steps that I followed to resolve this issue:

Running sk111766 and then performing the below:

• • This is required because we are removing the indexes that already exists so we don't create duplicates.
• This also removes the FetchedFiles, which tells the server if files are already indexed.
• So we remove the FetchedFiles and the indexes then when we restart services it will index xxx days' worth of logs.
• If you do not run the commands listed and only run sk111766 then it will not index the log files from before sk111766 was ran.

1. # cpstop
2. b. # rm -r $RTDIR/log_indexes/other*
3. c. # rm -r $RTDIR/log_indexes/audit*
4. d. # rm -r $RTDIR/log_indexes/firewallandvpn*
5. e. # rm -r $RTDIR/log_indexes/smartevent*
6. f. # rm $INDEXERDIR/data/FetchedFiles
7. g. # rm -r $INDEXERDIR/data/CpmiLocalCopy
8. h. # cpstart

(I went through these steps but I am not sure if it fixed the issue. I was not seeing any indexed logs even after going through them)

The following steps actually started showing indexed logs in smart console.

Go to expert mode:   fw repairlog -u 2024-11-15_032113_2226.log          (you have to repair all the logs file that you want to repair)

After running fw repair log, I am seeing indexed logs. Thanks!