topic Re: Migrate export file size in Firewall and Security Management

Migrate export file size

Amir_Arama — Tue, 21 Aug 2018 09:27:53 GMT

Hi,

my migrate export file is about 5.1GB size, is it normal?

i have about 60 GWs, 4 policy packages, total of about 1000 rules & most of blades enabled.

i run r80.10 gaia, but it was around this size also on r77.30

Thanks

Re: Migrate export file size

PhoneBoy — Tue, 21 Aug 2018 14:43:28 GMT

To me, that does not seem to be an unreasonable size for a migrate export, especially given the number of rules and objects involved.

Re: Migrate export file size

Chen_Fenghua — Wed, 05 Sep 2018 04:33:23 GMT

You need check are there any Revision Control packages?

Otherwise it is an unreasonable size for a migrate export.

Re: Migrate export file size

Stefano_Bucci — Wed, 08 Apr 2020 07:23:57 GMT

Hi, coming back here, is it possible to you that an exported db from an R80 smart-1 is about 20GB???

I deleted the revisions older than 2019. What do you suggest?

Thanks

Re: Migrate export file size

G_W_Albrecht — Wed, 08 Apr 2020 07:34:21 GMT

Could well be the issue from sk164270: Migrate export file is abnormally large

Re: Migrate export file size

Stefano_Bucci — Wed, 08 Apr 2020 07:37:15 GMT

I'll try this even if I'm running an R80. Thanks

Re: Migrate export file size

Maarten_Sjouw — Wed, 08 Apr 2020 09:06:18 GMT

Before Migrating from R77.30 to R80 you should have removed all revisons. Are you including indexes or logs?

Re: Migrate export file size

Stefano_Bucci — Wed, 08 Apr 2020 09:10:54 GMT

Customer wants to keep the last 6 months of revisions.
I'm excluding that. thanks

Re: Migrate export file size

Maarten_Sjouw — Wed, 08 Apr 2020 09:12:58 GMT

In R80 you cannot use them anymore, so what is the purpose?

Re: Migrate export file size

Stefano_Bucci — Wed, 08 Apr 2020 09:15:43 GMT

you right... try to convince the customer and go head deleting them

Re: Migrate export file size

Maarten_Sjouw — Wed, 08 Apr 2020 09:19:29 GMT

Create a full backup/snapshot of the SMS before deleting them and then create the export file. Possibly restore the backup/snapshot.

Re: Migrate export file size

Timothy_Hall — Wed, 08 Apr 2020 12:11:12 GMT

Having an excessive number of old IPS updates hanging around can significantly drive up the size of a migrate export in R80+ into the multi-GB range. Under Threat Prevention...Threat Tools...Updates...IPS...Update Now...Switch to Version you can see how many old IPS updates are present. If there are a lot, see here for the cleanup procedure:

sk120573: All data from all IPS updates performed by user on R80.10 being saved in database and not removed

To reduce the export size you can also try purging old published sessions on the Manage & Settings...Revisions screen, just right click the various sessions and select Purge.

Re: Migrate export file size

Stefano_Bucci — Thu, 09 Apr 2020 12:23:52 GMT

Hi Timothy, thanks for the answer. I deleted all the old log files.

Then I tried to used the $MDS_FWDIR/scripts/run_groovy_script.sh $MDS_FWDIR/scripts/IpsDomainFilesCleanup.groovy command but I got:

groovy.util.ResourceException: Cannot open URL: file:/opt/CPsuite-R80/fw1/cpm-server/dummy/opt/CPsuite-R80/fw1/scripts/IpsDomainFilesCleanup.groovy
at groovy.util.GroovyScriptEngine.getResourceConnection(GroovyScriptEngine.java:371)
at groovy.util.GroovyScriptEngine.loadScriptByName(GroovyScriptEngine.java:504)
at groovy.util.GroovyScriptEngine.createScript(GroovyScriptEngine.java:564)
at groovy.util.GroovyScriptEngine.run(GroovyScriptEngine.java:551)
at com.checkpoint.management.groovy_client.ManagementGroovyClient.runScript(ManagementGroovyClient.java:11)
at com.checkpoint.management.cpm.commands.GroovyClient.run(GroovyClient.java:14)
at com.checkpoint.management.cpm.Cpm.main(Cpm.java:70)

Indeed I checked the IPS update and the oldest one is dated 2017, but if this command does not work I don't know how to remove the old unnecessary files.

I also removed the published sessions correctly. I guess removing the IPS signatures will reduce the size of the db finally.

Re: Migrate export file size

Timothy_Hall — Thu, 09 Apr 2020 20:59:47 GMT

If that script doesn't work with the latest SMS versions you'll probably need to engage with TAC, hopefully they have a more recent copy you can use. About how many IPS update packages are you seeing? If it is more than 10 I'd definitely try to clean it up before the migrate export as I've seen lots of old IPS packages spike a migrate export past 20GB.

Re: Migrate export file size

Stefano_Bucci — Fri, 10 Apr 2020 07:28:11 GMT

raising an SR to TAC. I'll let you know.

thanks

Re: Migrate export file size

Stefano_Bucci — Thu, 16 Apr 2020 10:48:59 GMT

Hi,

TAC from CheckPoint jsut told me that IPS signatures are cumulative. If I'd delete one signature I'll loose some specific configuration.

So I have signature of 2017 that are still being used??

Re: Migrate export file size

Timothy_Hall — Thu, 16 Apr 2020 13:50:01 GMT

I'm pretty sure that is not correct unless something radically changed in R80.20 or later, as far as I know an IPS Update is a standalone file that contains all IPS ThreatCloud Protections; you don't need to have the older IPS Update files for the newer ones to work.

But perhaps only the deltas are being sent in the latest IPS updates? We will probably need an answer from R&D on this one, paging @PhoneBoy ...

Re: Migrate export file size

PhoneBoy — Fri, 17 Apr 2020 03:15:35 GMT

I agree, that doesn't sound...correct.

Re: Migrate export file size

Stefano_Bucci — Fri, 17 Apr 2020 07:18:58 GMT

ok, thanks, so how can I delete from an R80 the old IPS packages?

Re: Migrate export file size

PhoneBoy — Fri, 17 Apr 2020 22:25:18 GMT

Recommend working with the TAC here.