Python tool for exporting/importing a policy package or parts of it

[Inbar_Moskovich]

Overview

ExportImportPolicyPackage tool enables you to export a policy package from a R80.x management database to a .tar.gz file, which can then be imported into any other R80.x management database.

This tool can be used for backups, database transfers, testing, and more.

In the case you are exporting a policy package from a CMA, please verify that a global policy was NOT assigned to that CMA.
The tool doesn't support exporting a policy with global policy assigned!

Description

This tool enables you to export a policy package (Access Policy, Threat Policy or both) from a management server into a .tar.gz file.

Notice

There are some types of objects that the script might not be able to export.
In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the log files to notify you of this.
In the Check Point SmartConsole you can easily replace each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

Instructions

Download the latest version from our GitHub repository: https://github.com/CheckPointSW/ExportImportPolicyPackage 
First, make sure you have [2.7.9 <= Python <= 2.7.14] installed on the machine running the script.
To export a package, run the import_export_package.py script. An interactive menu will guide you the rest of the way.
Command line flags may also be set in order to skip some or all of the menu.
A lot more details can of course be accessed with the [-h] option. This option also prints the current version of the tool.

Current tool version is V3.0.

Limitations

This export/import script does not gather all data from a given management server/CMA.
In general, it is limited by the R80.x Management APIs.
Specifically, this means:

• CMAs with a Global Policy assigned cannot be exported
  • Workaround: unassign the Global Policy prior to export
• Gateway/Cluster objects have to be recreated
  • Placeholder objects will be created
• UserCheck messages have to be recreated
  • Placeholder objects will be created
• The Internal Certificate Authority will not be copied. This means:
  • Re-establishing SIC with the appropriate gateways
  • Re-generating VPN certificates
  • Manually recreating HTTPS Inspection and DLP Rules
• Other objects not currently readable/writable via the R80.x API will not be copied

Tested on version

R80.x
Releases earlier than R80 lack the necessary API support and are not supported.

Source Code Availability

The source code is available through GitHub: https://github.com/CheckPointSW/ExportImportPolicyPackage 

FAQ

Replies to this thread have locked.
Please refer to the FAQ below before you create a new post with your question.

When I run this tool, I get the message: APIResponse received a response which is not a valid JSON.

This most likely means you haven't enabled the API server yet.
See: https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Enabling-web-api/m-p/32641

I get an error message related to server fingerprint

Use the --unsafe option to ignore this error.

Can this tool export more than one policy package at a time?

Not currently, but you could call the tool in a script multiple times.

NOTICE: By using this sample code you agree to terms and conditions in this Terms and Conditions

[Robert_Decker]

Hi Bill,

If you are exporting several policies from the same database and then importing to another same database, they may have several objects in common.

In such case, the tool should report errors, there is no way to avoid this.

But, eventually, the tool should succeed to import the policy. It should be tolerant of object duplications.

What is that certain point that it fails? Can you print here the logs arount that point? How many duplications does it report?

Robert.

[Bill_Ng]

Hi Robert,

That is correct.  I am exporting several policies from the same db and then importing to another same db.  The 1st policy export went in fine and the next errors out during the import.  Look like it failed during the group import part according to the import_export txt file.  I'll add more details here shortly.   Been tied up with other things.  I notice the script hangs on importing screen then kicks out.  It does appear to be reimporting everything from the 1st  import ( groups, hosts, etc) but never makes it to the policy which is what I really want.

***Update***

Below is what files are left after the import stops.  Looks like it stops at the add-group.json file.

Bill

[Robert_Decker]

ok, but what is the problem that you are faceing? does it hang and exit? does it start importing the group and then aborts? is there any log output? i need more input to assist.

robert.

[Bill_Ng]

Hi Robert,

The script hangs then exits out of the python menu screen.  It looks like it's in middle of processing the groups which are duplicates from the 1st import.  The very last line of the import_export.txt file says 'Failed to import group with name [FW1-server1-1435-acceess] Error: More than one object named 'FW1-serve'.  The import_export.txt then just ends.  Let me know if more input is needed.  Thanks for your assistance.

[Robert_Decker]

Hi Bill,

I'd like to understand - is the name of the group that fails to import "FW1-server1-1435-acceess" or "FW1-serve"?

Can you also paste here the last lines from file "import_error_log.elg"?

If the tool just hangs without raising any exception, I suspect that it failed not due to duplicated object import, but due to something else.

Let's try a little test - I assume that you run it in your lab, so revert to a baseline snapshot and try to export-import directly the package that fails - make it the first one.

It should fail without objects duplication at all. This is my assumption, I think that there is a problem with an imported object.

Robert.

[Vanesa_Benito_O]

Hi,

I am trying to use this tool in order to realize a merge of two Management consoles in R80.10. I have done this procedure before with the cp_merge tool. But in this case i cant be able to complete the process:

I try to extract the package Standard from one Management:

import_export_package.py -op export -m 172.18.1.1 --debug on -->This process end with some warnings, the script is no able to export the firewall object, instead of it, exported a renamed object like partialexport_nameoffirewall.

The partial exported object is a dummy firewall,, therefore i have trying to import the exported file (with the Standard package) in the new management console. But in this case, the process stop with the following messages:

import_export_package.py -op import -m 172.18.2.2 --debug on

Failed to import group with name [export_error_CpmiHostCkp_000000b2-0097-0043-ab82-5c9d452c7cba_DGOJ_AHJC_Server]. Error: More than one object named 'export_error_CpmiHostCkp_000000b2-0097-0043-ab82-5c9d452c7cba_DGOJ_AHJC_Server' exists.

Failed to import group with name [IPsMaliciosasUpatre]. Error: More than one object named 'IPsMaliciosasUpatre' exists.

Importing Access_Layer [IMPORTED LAYER 1 Network]

Adding access-rules

Failed to import access-rule with name [Acceso_upadte_Ips]. Error: Requested object [tcp_12345] not found
Also failed to generate placeholder object: Validation failed with 1 error

Failed to import access-rule. Error: Requested object [TCP-40815] not found
Also failed to generate placeholder object: Validation failed with 1 error

Not unique name problem "iMesh" - changing payload to use UID instead.

Failed to import access-rule. Error: Requested object [Port_3105] not found
Also failed to generate placeholder object: Validation failed with 1 error

Imported 24 out of 24 access-rules (100%)

Adding access-sections

Imported 4 out of 4 access-sections (100%)

Importing Access_Layer [IMPORTED LAYER 1 Application]

Adding access-rules

Imported 1 out of 1 access-rules (100%)

Importing Nat_Layer [Standard]

Nothing to import...

Error analyzing package details! Aborting import.

Failed to attach layers to package! Error: Requested object [Standard] not found. Import operation aborted.

Can anybody tell me if I need to change something before export the package? How can i know what is wrong? The debug information dont tell me anything specific

Thank you in advance,

Regards

[Robert_Decker]

during the import process, the interactive menu asked you to provide the name of the file with the package data (a tar archive).

can you please paste here the name of that file?

Robert.

[Demith_Samaraw2]

Hi 

Thanks all for all the contributions, I am trying export few policies and import them to a new CMA, most of the process runs without any issue, but I have some errors that I cant find answers in the thread, if @ Robert  Decker can explain them that would be great.

Some of the policies failed to export Firewall/NAT rules because it says dummy object (export_error) cannot be created, this only happens to all the firewalls on one VSX gateway, all the VSs on the other VSX gateway are created as export_error fine, when this happens it does not create the rule and skip any rule with these objects

Failed to import nat-rule. Error: Requested object [export_error_CpmiVsNetobj_c89344b-7932-4e7e-9f2c-80b12b99ff94-xcfwext] not found

Also following error when trying to import Access Rules

Non unique name problem "PostgreSQL" - changing payload to use UID instead.

Thanks all

[Vanesa_Benito_O]

Of course , I wrote the complete route to the file.

C:\Users\Vanesa\Desktop\CheckPoint\R80.10\3. Paquetes\ExportImportPolicyPackage-master\exported__package__Standard__2018_04_17_12_10.tar.gz

Thank you

[Robert_Decker]

Hi Vanesa,

I need to analyze your log files to better understand the root cause of your problem.

I need the import_export.elg file from the tool's working directory and api.elg file from "$MDS_FWDIR/logs" folder on management server.

https://community.checkpoint.com/people/dwelccfe6e688-522c-305c-adaa-194bd7a7becc‌, please instruct Vanesa how and where to upload her files.

Robert.

[PhoneBoy]

Credentials sent to https://community.checkpoint.com/people/vanes6d3e7955-b261-41c0-9e50-fd5db4aeeaed‌

I will make sure you get the files.

[Robert_Decker]

Hi Demith,

As it is explained in the description section of the tool on the top of this page, not all object types can be fully exported (these are mainly the non R80.x native objects, such as VSX/VS/Clusters/Users etc.).

For such objects, a dummy object is created as a place-holder, so that you can later find it easily and manually replace it in the imported database.

In your case, an object of type CpmiVsNetobj with uid c89344b-7932-4e7e-9f2c-80b12b99ff94 and name xcfwext is not exported, instead a place-holder named export_error_CpmiVsNetobj_c89344b-7932-4e7e-9f2c-80b12b99ff94-xcfwext is created and exported.

This object is used in one of your NAT rules. For some reason, when importing this NAT rule, this object can not be found during the process, meaning that the tool failed to create it in the imported database.

In order to find out why, I need that you search for this object in the "export_import.elg" file that was created in your working folder during the import process. Please copy and paste all the info related to this object here.

In addition, please double-click on the tar archive file that you are importing - you should see a files/folders structure inside. Pleass copy and pase the full list here.

I'd like to ensure that this dummy object was really created and is not a fake one.

After this preliminary info, I'll ask for some additional info for further investigation. But let's start with this.

The second error - Non unique name problem "PostgreSQL" is ok, you probably already have an object with such name in your target database. To avoid duplication, it will be imported using its UID.

Robert.

[Robert_Decker]

Thank you.

I got the logs. Will analyze and update shortly.

Robert.

[Robert_Decker]

Hi Vanesa,

After an intensive research into your provided log files to find out what went wrong during the import, it seems that the management server into which you are importing, doesn't have a policy package Standard.

This means that it was manually removed from that server, because it is a default policy package and is created automatically. This is ok, nothing is wrong with this.

But the question is how it was removed. Maybe that manual action was done incorrectly and caused to database instability.

In your log files I saw that the tool detected that there is no policy named Standard - correct, and therefore it tries to create it - fine.

Then the tool tries to publish this change - creation of a new policy - the publish operation fails due to already existing object named 'Network', which is the name of a default access rulebase.

From this point, the whole process is into a mess and inconsistency.

I cannot say what is wrong with your database, maybe consult with your SE/Partner for deeper investigation.

Robert.

[Vanesa_Benito_O]

Hi Robert,

Thank you for your answer! It helped me to resolve the error

I have tryed to recreate again the second management concole, in order to resolve the database error when I deleted the Standad policy but I get the same error althoug i dont delete the Standard package.

In order to avoid this issue I have duplicated the policy package named Standard in the origin and renamed it! Then I have exported the new package and imported it in the other Checkpoint management and it works correctly!

Thank you!

[saravanakumar_J]

what is the way to export if you have global policy assigned to the cma? 

[PhoneBoy]

Unassign the global policy and perform the export?

[Robert_Decker]

correct!

[Dor_Marcovitch]

please note that you need to download the "cp_mgmt_api_python_sdk" directory from GIT separately because for some reason the clone for the package does not contains this folder (it downloaded empty) and its only mentioned on the README file

[Joshua_Hatter]

You need to clone the repository with the --recursive flag to get the dependency.

git clone --recursive https://github.com/CheckPoint-APIs-Team/ExportImportPolicyPackage

[Dor_Marcovitch]

does anyone have a good export-import with this tool on R80.10 DMS ?

we used this tool and it looks like it messed the policy and objects

missing rules, missign object, wrong object on the rules.

[Peter_Lyndley]

Hi All,

Thanks for all the development on this script - very useful.

I'd like to be able to cron multiple policy exports, which is fine, however i cannot find enough switches to make this script fully non-interactive. 

i.e it always asks 1, change settings, 2, run, 99, exit

Is there any way to make this script fully non-interactive so it can be run as a scheduled task in Gaia ?

thanks

Peter

[Robert_Decker]

Hi Peter,

Currently, this tool is an interactive Python application.

Since it is an open source on GitHub, anyone with Python knowledge can modify it as needed.

Robert.

[Usman_Shaikh]

I tried to import the policy package using the Python Script..

The policy to be imported is a layered policy with an ordered layer... This was applied to an R77.30 gateway on the source management server.. The import went OK apart form a few errors about duplicate objects and some rules not getting imported

However all the policies (apart from one) on the target management servers have now been changed to ordered layers “Security” and “Application”. All these policies are applied to R80.10 gateways on the target server

 

 Security layer now has all the firewall rules and application layers having a just the below..

 

 What I would like to understand is

1. Is there an impact to the firewall operation by the introduction of the ordered layers ?
2. Why have these policies been changed to ordered layers on the target management server ?
3. Why did one of the policy not get changed on the target server ?
4. How do I go back to using non-layered policy for the existing policies on the target management server (i.e. rollback) ?
5. What’s the best possible way to import a layered policy from source to target without impacting the existing policies on the target

Thanks

[Robert_Decker]

please refer to this thread about layers in R80, may be helpful - 

https://community.checkpoint.com/message/1139

regarding the source and destination policies and ordered layers, you say that the policies on the source were ordered as well, so where is the problem?

that one policy with the exception, can you paste its structure on the source and on the target?

robert.

[Usman_Shaikh]

Thanks Robert for coming back on this.. The imported policy was indeed ordered with a single layer; however as a result of the import all (but one) of the existing policies on the target were changed to have ordered layers of Security and Application. Is this becuase some of the objects used in the imported policy are also referenced in the existing policies on the target (that were changed to ordered layers) which would explain my point #3 about the only policy that did not get changed is the one that does not use any common objects ?

Can I simply remove the applicaiton layer from these existing policies on the target so they will only be left with a single layer as before without any imapct to the traffic? Also notice that the firewalls to which these new ordered policies are applied do not even have Application Control blade enabled so I think it should be OK to do so ?

Also with regards to the rules that failed to import.. I get the below error in the logs

"Failed to import access-rule with name [Rule-Name]. Error: Requested object name [tcp5666] is not unique"

Looking at the source Mgmt server, I have two objects defined.. one as tcp5666 and the other as TCP5666. Could this be the reason for the above failure message ?

[Robert_Decker]

Hi Usman,

You may remove the application layer, it is created by default.

Regarding the import error - there is an ambiguity due to two objects with the same name defined as a service and as an application. The tool doesn't have any context to distinguish, and therefore the error. This can be fixed manually.

Robert.

[Usman_Shaikh]

Thanks Robert.. i will tidy it up on the source Mgmt server and reattempt..

I also get the follwoing errors/partial errors

Imported simple-gateway with name [partial_export_error_simple-gateway_d8372965-ece3-42a6-8c97-50ce2a3d5a32_<gateway1-name>]

Imported com.checkpoint.objects.classes.dummy.CpmiGatewayCluster with name [export_error_CpmiGatewayCluster_cb284c2e-d24a-4941-9f92-8e69597e3fb3_<gateway2_name>]

Any ideas what could this be ?

[Robert_Decker]

Hi,

If you've read the introductory remarks on this tool, there are some types of objects that are partially exported or not exported at all.

In such case a manual work is required to complete the import operation.

Robert.

[Usman_Shaikh]

Many thanks I will look into these