Migrating MDM/MLM from R80.30 to R81.10 with all logs

[cem82]

Hi

We are looking to migrate our current R80.30 MDM +MLM to R81.10 on new servers.  I have tested with migrate_server export -v R81.10 -x <filename> on the MDM as a starting point but in R81.10 SmartConsole we see what appears to only go back to midnight of the previous day from when the migrate_server was run.  How can we extend that back to include all traffic/audit logs?

Thanks

[Ido_Shoshana]

Hi,

Important note: You won’t be able to preserve log-Indexes (the actual SME log-DB), if you’re upgrading to R81.x from R80.x, as the Solr I/S had been upgraded.

so this procedure should only be done on R80.x to R80.x or R81.x to R81.x (like R80.20 to R80.40 or R81 to R81.10).

If upgrading to R81, then you can enlarge the indexing backwards time limit to index older logs/events - more than the default 1 day back (assuming the actual log-files from its Log-Servers still exist).
See sk111766.

[cem82]

If it's not possible for the log indexes, how about the raw log files after doing a logswitch and copy those over or is that not an option?

[Ido_Shoshana]

yes, this is an option since the log files are kept.

[cem82]

That's great!  So all we'd need to do is copy over the old raw log files to the new server?  Are they in the same format and don't need any sort of script to be run to be able to be read by R81.10?  If we do that could we also then also follow the SK you mentioned to index the additional days as well?

[Ido_Shoshana]

No need to copy. The log files should have already been imported as part of the migrate server as you have shown above (-x / -l).

Yes, same format - all good. Then you can follow the sk111766 to re-index back as many log files days of data as you wish.

[SunilShivnani1]

For MLM migration on new hardware running same version (R81.10), if we are not worried about log files migration, migrate_server export -v R81.10 -x <filename> should be enough. Is that correct? 

[Maja_B]

And what about Mgmt upgrade from R80.40 to R81.20? We tried exporting 30 days of logs, but it took a lot of time. We decided to archive the logs, delete them and do migrate export with 5 days of logs. Can we just copy the archived logs back to R81.20 Mgmt and index them?

[emmap]

Yes you can manually copy log files over (make sure you have all the pointer files as well) and (optionally) re-index them.

[Maja_B]

Thanks!

[Amir_Senn]

From R80.xx to R81.xx there's a different version of SOLR (newer version has newer SOLR that works better) so everything will need to be re-indexed so don't bother exporting/importing your indexes. So instead of using -x (export logs and indexes) you can use -l (export logs only) and that will be faster.

[Maja_B]

Thanks. We already did it with -l flag.

[Daniel_Fidlin]

Hi Maja_B,

As my colleagues said, the SOLR version was upgraded in Version R81 and above, I would recommend for an upgrade from R80.xx versions to use (migrate_server export -v R81.xx -l <file name>) command to export only the log files not including the indexes.

this will transfer all logs under $FWDIR/log to the new version, than you may increase the days to index backwards as much days as you desire, you can track the index process with the following command (watch -d 'cat $INDEXERDIR/data/FetchedFiles').

to increase the days to index, just follow this simple steps:
1) cp $INDEXERDIR/log_indexer_custom_settings.conf $INDEXERDIR/log_indexer_custom_settings.conf.ORIGINAL

2) vi $INDEXERDIR/log_indexer_custom_settings.conf

3) add under ":max_disk_space_usage (0)" the following ":days_to_index (<number of days>)" then save changes

4) run 'stopIndexer' & 'startIndexer'

Kind regards, Daniel Fidlin

[Maja_B]

Thanks, Daniel!