Migrate export file size

[Amir_Arama]

Hi,

my migrate export file is about 5.1GB size, is it normal?

i have about 60 GWs, 4 policy packages, total of about 1000 rules & most of blades enabled.

i run r80.10 gaia, but it was around this size also on r77.30

Thanks

[PhoneBoy]

To me, that does not seem to be an unreasonable size for a migrate export, especially given the number of rules and objects involved.

[Chen_Fenghua]

You need check are there any Revision Control packages?

Otherwise it is an unreasonable size for a migrate export.

[Stefano_Bucci]

Hi, coming back here, is it possible to you that an exported db from an R80 smart-1 is about 20GB???

I deleted the revisions older than 2019. What do you suggest?

Thanks

[G_W_Albrecht]

Could well be the issue from sk164270: Migrate export file is abnormally large

[Stefano_Bucci]

I'll try this even if I'm running an R80. Thanks

[Maarten_Sjouw]

Before Migrating from R77.30 to R80 you should have removed all revisons. Are you including indexes or logs?

[Stefano_Bucci]

Customer wants to keep the last 6 months of revisions.
I'm excluding that. thanks

[Maarten_Sjouw]

In R80 you cannot use them anymore, so what is the purpose?

[Stefano_Bucci]

you right... try to convince the customer and go head deleting them

[Maarten_Sjouw]

Create a full backup/snapshot of the SMS before deleting them and then create the export file. Possibly restore the backup/snapshot.

[Timothy_Hall]

Having an excessive number of old IPS updates hanging around can significantly drive up the size of a migrate export in R80+ into the multi-GB range. Under Threat Prevention...Threat Tools...Updates...IPS...Update Now...Switch to Version you can see how many old IPS updates are present.  If there are a lot, see here for the cleanup procedure:

sk120573: All data from all IPS updates performed by user on R80.10 being saved in database and not ...

To reduce the export size you can also try purging old published sessions on the Manage & Settings...Revisions screen, just right click the various sessions and select Purge.

[Stefano_Bucci]

Hi Timothy, thanks for the answer. I deleted all the old log files.

Then I tried to used the $MDS_FWDIR/scripts/run_groovy_script.sh $MDS_FWDIR/scripts/IpsDomainFilesCleanup.groovy command but I got:

groovy.util.ResourceException: Cannot open URL: file:/opt/CPsuite-R80/fw1/cpm-server/dummy/opt/CPsuite-R80/fw1/scripts/IpsDomainFilesCleanup.groovy
at groovy.util.GroovyScriptEngine.getResourceConnection(GroovyScriptEngine.java:371)
at groovy.util.GroovyScriptEngine.loadScriptByName(GroovyScriptEngine.java:504)
at groovy.util.GroovyScriptEngine.createScript(GroovyScriptEngine.java:564)
at groovy.util.GroovyScriptEngine.run(GroovyScriptEngine.java:551)
at com.checkpoint.management.groovy_client.ManagementGroovyClient.runScript(ManagementGroovyClient.java:11)
at com.checkpoint.management.cpm.commands.GroovyClient.run(GroovyClient.java:14)
at com.checkpoint.management.cpm.Cpm.main(Cpm.java:70)

Indeed I checked the IPS update and the oldest one is dated 2017, but if this command does not work I don't know how to remove the old unnecessary files.

I also removed the published sessions correctly. I guess removing the IPS signatures will reduce the size of the db finally.

[Timothy_Hall]

If that script doesn't work with the latest SMS versions you'll probably need to engage with TAC, hopefully they have a more recent copy you can use.  About how many IPS update packages are you seeing?  If it is more than 10 I'd definitely try to clean it up before the migrate export as I've seen lots of old IPS packages spike a migrate export past 20GB.

[Stefano_Bucci]

raising an SR to TAC. I'll let you know.

thanks

[Stefano_Bucci]

Hi,

TAC from CheckPoint jsut told me that IPS signatures are cumulative. If I'd delete one signature I'll loose some specific configuration. 

So I have signature of 2017 that are still being used??

[Timothy_Hall]

I'm pretty sure that is not correct unless something radically changed in R80.20 or later, as far as I know an IPS Update is a standalone file that contains all IPS ThreatCloud Protections; you don't need to have the older IPS Update files for the newer ones to work.  

But perhaps only the deltas are being sent in the latest IPS updates?  We will probably need an answer from R&D on this one, paging @PhoneBoy ...

[PhoneBoy]

I agree, that doesn't sound...correct.

[Stefano_Bucci]

ok, thanks, so how can I delete from an R80 the old IPS packages?

[PhoneBoy]

Recommend working with the TAC here.

[archie]

TAC told me, that in R81.10 version old IPS updates working fully automatically, and you do not able to setup how many old IPS updates would you like to store. In R80.10 this solution worked better I thought.