Reports data loss

[Bernardes]

Hello Mates!

I have a distributed environment (SMS + Gateway Cluster + Smart Event and Logs) and did an upgrade on SMS and Smart Event from 80.40 to 81.10. The SMS upgrade was by migrate process(create a new VM) and the Smart Event was by CPUSE.

After that, I can't see the reports about Application and URL Filtering or any other blade, look at this example bellow:

When I filter by "All Time" it just shows me this month which is the date I did the upgrade until now.

As I said, the Smart Event doesn't change, the upgrade was by CPUSE, but to the SMS was created a new VM and imported the migrate file.

Is there anything missing to bring from the old SMS to the new SMS so that I can see the old report information from other months?

Thank you!

[Chris_Atkinson · Accepted solution]

Did you move and reindex the old log data?

R81

GNG-1259,

PMTR-52941 R81 includes new logs indexing mechanism, so when upgrading Management server/Log Server/Multi-Domain Server/Multi-Domain Log Server/SmartEvent from R80.x, old log indexes are not upgraded.

The indexing mechanism will re-index the last 24 hours automatically. To increase the period of offline indexing (how far in the past to re-index the logs), see sk111766.

[PhoneBoy · Accepted solution]

When upgrading to an R81.x release from an R80.x release, you will need to manually reindex the logs after upgrading.
This is because we’ve changed the index format in R81.
See: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...

[the_rock]

You got 100% correct answers from Chris and Dameon, as thats exactly what you need to do. I been through this 3 times before and that was sk I had to follow. 

[Bernardes]

Hello @Chris_Atkinson @PhoneBoy @the_rock ! First of all, thank you very much!

I did the process of index old logs and configured for 60 days both on "./log_indexer -days_to_index 60" and in the storage config.

After that, I can see logs just 14 days since yesterday to back.

I saw the secondary process of the sk164553, but I didn't understand very well.

Is this second process from sk164553 is mandatory or it already would works with just the process of the index (./log_indexer -days_to_index 60) ?

Thank you

[PhoneBoy]

The procedure in sk164553 is if you need to index a specific log file.
If you’re indexing the last 60 days of logs per your CLI command this shouldn’t be necessary.

Note that it can take some time to index the log files since it is done at a lower priority.
You will see the CPU utilization on the management server higher than normal as a result, but the indexing process backs off when the CPU is needed for other tasks.

[Bernardes]

@PhoneBoy thank you for the explanation!

I can see a high-load CPU on Smart Event appliance indeed. And slowly I saw the old log being shown on the smart console.

[the_rock]

Keep in mind, it may take 24 hours for all to show up, I seen that before myself.

[Bernardes]

@the_rock thank you for the advice!

It has taken more than 3 days and just now I can see de CPU in normal use.

Is there any way to monitor when the index process is running or is finished?