Python tool for exporting/importing a policy packa...

Inbar_Moskovich · ‎2016-10-13

Overview

ExportImportPolicyPackage tool enables you to export a policy package from a R80.x management database to a .tar.gz file, which can then be imported into any other R8x management database.

This tool can be used for backups, database transfers, testing, and more.

In the case you are exporting a policy package from a CMA, please verify that a global policy was NOT assigned to that CMA.
The tool doesn't support exporting a policy with global policy assigned!

The tool is referenced in https://support.checkpoint.com/results/sk/sk180923

Description

This tool enables you to export a policy package (Access Policy, Threat Policy or both) from a management server into a .tar.gz file.

Notice

There are some types of objects that the script might not be able to export.
In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the log files to notify you of this.
In the Check Point SmartConsole you can easily replace each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

Instructions

Download the latest version from our GitHub repository: https://github.com/CheckPointSW/ExportImportPolicyPackage
First, make sure you have [2.7.9 <= Python <= 2.7.14] installed on the machine running the script.
To export a package, run the import_export_package.py script. An interactive menu will guide you the rest of the way.
Command line flags may also be set in order to skip some or all of the menu.
A lot more details can of course be accessed with the [-h] option. This option also prints the current version of the tool.

Current tool version is V3.0.

Limitations

This export/import script does not gather all data from a given management server/CMA.
In general, it is limited by the R80.x Management APIs.
Specifically, this means:

CMAs with a Global Policy assigned cannot be exported
- Workaround: unassign the Global Policy prior to export
Gateway/Cluster objects have to be recreated
- Placeholder objects will be created
UserCheck messages have to be recreated
- Placeholder objects will be created
The Internal Certificate Authority will not be copied. This means:
- Re-establishing SIC with the appropriate gateways
- Re-generating VPN certificates
- Manually recreating HTTPS Inspection and DLP Rules
Other objects not currently readable/writable via the R80.x API will not be copied

Tested on version

R8x
Releases earlier than R80 lack the necessary API support and are not supported.

Source Code Availability

The source code is available through GitHub: https://github.com/CheckPointSW/ExportImportPolicyPackage

FAQ

Replies to this thread have locked.
Please refer to the FAQ below before you create a new post with your question.

When I run this tool, I get the message: APIResponse received a response which is not a valid JSON.

This most likely means you haven't enabled the API server yet.
See: https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Enabling-web-api/m-p/32641

I get an error message related to server fingerprint

Use the --unsafe option to ignore this error.

Can this tool export more than one policy package at a time?

Not currently, but you could call the tool in a script multiple times.

Robert_Decker · ‎2017-08-27

This source code is now an open source on GitHub repository:

GitHub - CheckPoint-APIs-Team/ExportImportPolicyPackage

Julien_Tissot · ‎2017-08-29

Hello,

Is there a way to export all packages from 1 domain instead of exporting each package 1 by 1?

Thank you,

Julien

Lari_Luoma · ‎2019-08-30

@Julien_Tissot

With the Python-tool you will always have to define the policy name to export/import. However, you can use it in non-interactive mode and embed import_export_package into a script. This way you could get several policies imported at once.

Adam_Galmor · ‎2017-08-29

At the moment no, but it might be added in the near future.

Julien_Tissot · ‎2017-08-30

Hello!

I have an error when there is a special character (in my case 'é' or 'è' or others) in the rule name. Is there a way to just ignore this character but to let the export continue? The logs are attached

Thanks you

Julien

----------------------------------------------------------------------------

Retrieved 384 out of 384 rules (100%)

Traceback (most recent call last):
File "import_export_package.py", line 44, in <module>
    export_package(client, args)
File "/import_export_V2.0/exporting/export_package.py", line 38, in export_package
    = export_access_rulebase(access_layer["name"], client, timestamp, tar_file)
File "/import_export_V2.0/exporting/export_access_rulebase.py", line 16, in export_access_rulebase
    get_query_rulebase_data(client, "access-rulebase", {"name": layer})
File "/import_export_V2.0/exporting/export_objects.py", line 122, in get_query_rulebase_data
    rulebase_item else "???", rule["name"] if "name" in rule else "")
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 7: ordinal not in range(128)

OliviaMocellini · ‎2017-09-11

Hi guys,

I'm trying to make an export by running this utility but don't get what should I put as the package name, because from what I'm seeing it's not the the name that the policy has on the SMS server.

Thanks in advance!

Robert_Decker · ‎2017-09-12

Hi Julien,

Our team will check this problem and update the community once resolved.

Robert.

Robert_Decker · ‎2017-09-12

Hi Olivia,

Our team will check this problem and update the community once resolved.

Robert.

Shoham_Halevi · ‎2017-09-12

Hi Olivia,

You're supposed to put the name of the policy package as seen here:

If that doesn't help, please explain further what you are trying to do and how, and what is the output of the tool?

Shoham

Shoham_Halevi · ‎2017-09-13

Hi Julien,

The issue should be fixed now, reclone the repo and try again.

If there are any further issues, please tell us.

Shoham

Ivo_Hrbacek · ‎2017-09-15

Hello Adam and guys,

did some testing and here are my findings:

case 1:

not possible to import objects with tags, looks its a problem with exported data and parameters tags.0.X

debug msg:
Failed to import network with name [net_SIP-provider-broadsoft_XXXXX_21]. Error: Invalid parameter for [tags]. Invalid value

json data:
{
"subnet4": "XXX",
"nat-settings.auto-rule": false,
"color": "black",
"comments": "",
"name": "net_SIP-provider-broadsoft_XXX",
"broadcast": "allow",
"tags.0.name": "PUBLIC-INTERNET-LAB",
"mask-length4": 21,
"tags.0.comments": "",
"tags.0.color": "black",
"tags.0.type": "tag"
}

csv data:
broadcast,color,comments,mask-length4,mask-length6,name,nat-settings.auto-rule,nat-settings.hide-behind,nat-settings.install-on,nat-settings.ipv4-address,nat-settings.ipv6-address,nat-settings.method,subnet4,subnet6,tags.0.color,tags.0.comments,tags.0.name,tags.0.type,tags.1.color,tags.1.comments,tags.1.name,tags.1.type
allow,black,,21,,net_SIP-provider-broadsoft_x.x.x.x_21,false,,,,,,x.x.x.X,,black,,PUBLIC-INTERNET-LAB,tag,,,,

####

case2:

having two ordered layers Network and App on source mgmt - in Network layer I have rules where are defined inline layers - looks code does not count with this and all inline layers are loaded as ordered layers

debug msg:

IN ad OUT rules are defined with action inline layer
Failed to import access-rule with name [INT]. Error: Request for Apply Layer may not change Action Settings or User Check parameters
Failed to import access-rule with name [OUT]. Error: Request for Apply Layer may not change Action Settings or User Check parameters
Failed to import access-rule with name [INT]. Error: Request for Apply Layer may not change Action Settings or User Check parameters
Failed to import access-rule with name [OUT]. Error: Request for Apply Layer may not change Action Settings or User Check parameters

###

case3:

data export/import for non r80.10 gw is bad, I have few 1450 boxes -77.20 emdededGaia and look for definition below in json ->

{
"bladesInfo.1.sortOrder": 3,
"versionName": "R77.20",
"bladesInfo.4.comments": null,
"ipv4-address": "81.91.220.11",
"bladesInfo.5.sortOrder": 14,
"bladesInfo.6.sortOrder": 15,
"bladesInfo.5.display-name": "IPS",
"bladesInfo.3.comments": null,
"bladesInfo.6.comments": null,
"bladesInfo.1.customFields": null,
"bladesInfo.5.bladeCategory": "threat blades",
"bladesInfo.6.bladeState.changeable": false,
"bladesInfo.7.comments": null,
"bladesInfo.3.bladeCategory": "access blades",
"bladesInfo.5.bladeState.activationState": "on",
"bladesInfo.1.bladeState.valid": false,
"bladesInfo.7.bladeCategory": "threat blades",
"bladesInfo.4.display-name": "QoS",
"name": "partial_export_error_simple-gateway_86fc2467-e0a1-4db3-8efc-c50996c4ccf4_XXX",
"bladesInfo.0.bladeState.changeable": false,
"bladesInfo.5.bladeState.valid": false,
"bladesInfo.0.customFields": null,
"bladesInfo.4.name": null,
"bladesInfo.3.bladeState.changeable": false,
"macAddress": "",
"sicExists": true,
"bladesInfo.4.sortOrder": 9,
"bladesInfo.7.display-name": "Anti-Virus",
"bladesInfo.0.sortOrder": 0,
"bladesInfo.2.bladeState.changeable": false,
"accessLicense": false,
"bladesInfo.3.bladeState.activationState": "on",
"comments": "",
"bladesInfo.5.comments": null,
"bladesInfo.7.customFields": null,
"bladesInfo.2.comments": null,
"bladesInfo.6.name": null,
"bladesInfo.1.name": null,
"bladesInfo.5.customFields": null,
"bladesInfo.7.bladeState.activationState": "on",
"bladesInfo.6.customFields": null,
"osName": "Gaia Embedded",
"bladesInfo.0.bladeState.valid": false,
"bladesInfo.3.customFields": null,
"bladesInfo.7.bladeState.changeable": false,
"bladesInfo.7.sortOrder": 16,
"bladesInfo.4.customFields": null,
"color": "black",
"bladesInfo.6.bladeState.valid": false,
"bladesInfo.1.bladeState.activationState": "on",
"bladesInfo.4.bladeState.changeable": false,
"connectionState": "communicating",
"primaryManagement": false,
"bladesInfo.4.bladeState.valid": false,
"bladesInfo.4.bladeState.activationState": "on",
"bladesInfo.7.name": null,
"bladesInfo.3.name": null,
"bladesInfo.1.bladeState.changeable": false,
"bladesInfo.2.sortOrder": 4,
"natSummaryText": "None",
"customFields": null,
"bladesInfo.2.display-name": "URL Filtering",
"bladesInfo.1.display-name": "Application Control",
"proxyAddress": "Default Proxy Settings",
"bladesInfo.3.sortOrder": 7,
"bladesInfo.2.customFields": null,
"bladesInfo.3.bladeState.valid": false,
"bladesInfo.2.name": null,
"bladesInfo.6.display-name": "Anti-Bot",
"bladesInfo.6.bladeCategory": "threat blades",
"overallStatus": true,
"bladesInfo.0.display-name": "Firewall",
"bladesInfo.0.name": null,
"bladesInfo.5.bladeState.changeable": false,
"bladesInfo.2.bladeState.valid": false,
"licenseSKU": "CPSG-EVAL-P1207-30/1",
"bladesInfo.6.bladeState.activationState": "on",
"bladesInfo.2.bladeCategory": "access blades",
"bladesInfo.4.bladeCategory": "access blades",
"bladesInfo.1.bladeCategory": "access blades",
"bladeMgmtWorkflowOn": false,
"bladesInfo.2.bladeState.activationState": "on",
"bladesInfo.5.name": null,
"bladesInfo.0.bladeCategory": "access blades",
"display-name": "",
"bladesInfo.3.display-name": "Identity Awareness",
"bladesInfo.0.comments": null,
"hwName": "1100 Appliances",
"bladesInfo.0.bladeState.activationState": "on",
"bladesInfo.7.bladeState.valid": false,
"ipv6-address": "",
"bladesInfo.1.comments": null,
"threatLicense": false

for example "hwName": "1100 Appliances" does not seems to be correct
1450 are not imported and I have these logs:
Failed to import simple-gateway with name [partial_export_error_simple-gateway_d992da76-783b-4f50-ae1f-327701158bf0_XXX]. Error: Unrecognized parameter [natSummaryText]

thx

ivo

Shoham_Halevi · ‎2017-09-18

Hi Ivo,

I tried reproducing your first and second problems, but couldn't. Make sure you are on the latest version of the tool (clone it from the GitHub link above). If the problem persists, please reply with more details so I can reproduce the problem on my side.

This tool works for exporting/importing from R80.10 machines to R80.10 machines, nothing before, as there is no API support. So you can't export from or import to R77.20 machines.

Shoham

Ivo_Hrbacek · ‎2017-09-18

Hi there,

I have latest version.. is there a way how I can share with you exported data?

I understand its for exporting from/to R80.10 machine, but event R80.10 mgmt can orchestrate r77.x gateways.. specially 1400 boxes could be there since its just 77.20.x.x or something like that and those machines are normally in R80.10 database so why they should not be exported to another R80.10 machine?

ivo

OliviaMocellini · ‎2017-09-25

Thanks! it seems I had wrong typed the policy name many times!

OliviaMocellini · ‎2017-09-25

Thanks, resolved, package name was a typo.

Borut · ‎2017-10-25

Hi

I'm not successful with the tool. Trying to export the configuration from R80.10 mgmt.

python import_export_package.py -m x.x.x.x -op export -n Lab -o conf.out -u xxxxyyyyyy -p xxxxyyyyy --all

Traceback (most recent call last):
File "import_export_package.py", line 30, in <module>
payload={"read-only": "true" if args.operation == "export" else "false"})
File "/cygdrive/d/Python scripts/ExportImportPolicyPackage-master/cp_mgmt_api_python_sdk/lib/mgmt_api.py", line 154, in login
login_res = self.api_call("login", credentials)
File "/cygdrive/d/Python scripts/ExportImportPolicyPackage-master/cp_mgmt_api_python_sdk/lib/mgmt_api.py", line 225, in api_call
self.check_fingerprint()
File "/cygdrive/d/Python scripts/ExportImportPolicyPackage-master/cp_mgmt_api_python_sdk/lib/mgmt_api.py", line 522, in check_fingerprint
server_fingerprint = self.get_server_fingerprint()
File "/cygdrive/d/Python scripts/ExportImportPolicyPackage-master/cp_mgmt_api_python_sdk/lib/mgmt_api.py", line 414, in get_server_fingerprint
context = ssl.create_default_context()
AttributeError: 'module' object has no attribute 'create_default_context'

What am I doing wrong?

Best regards

Robert_Decker · ‎2017-11-02

Which version of python are you using?

It should be at least 2.7.9...

Borut · ‎2017-11-06

2.7.14.

BR

Borut · ‎2017-11-07

I managed to resolve the issue. First I tried with python 2.7.9. and didn't get the error message anymore, but got this one:

Login to management server failed. lib::APIResponse
{
"data": null,
"error_message": "APIResponse received a response which is not a valid JSON.",
"res_obj": {},
"status_code": 403,
"success": false
}

The solution was to enable API access from all IP's.

Borut · ‎2017-11-07

Now that the export is done, does anyone have a suggestion for a bulk rename of the objects/networks before importing?

Also, I can't seem to find NAT rules in the export files.

Julien_Tissot · ‎2017-11-15

Hello,

If there was 2 object with the same IP when exporting, we got errors when importing. Is there a way to force the creation of the second object with the same IP? In the API you can add "ignore-warnings true".

Thank you for your help,

Julien

Robert_Decker · ‎2017-11-16

Hi Julien,

We will check this and fix if needed.

I'll keep you informed.

Robert.

Robert_Decker · ‎2017-11-16

Hi,

The library code is fixed - "ignore-warnings" flag is set to "True" automatically for every command.

This should be the default behaviour. Thanks for pointing this out.

Please download it from GitHub repo.

Enjoy.

Robert_Decker · ‎2017-11-16

Only Access and Threat layers are exported.

NAT rulebase is a legacy database object and it is not trivial to manipulate it.

Robert.

Robert_Decker · ‎2017-11-16

After in-depth investigation of the code, NAT rulebase may also be exported/imported.
I'll add this task to our future improvements list. It is indeed a must-do task.
I'll inform this forum on progress.
Robert.

Marco_Valenti · ‎2017-11-17

hey there , just went trough some testing with the scripts , I'll try to recap a bit

I have made an export from an mds 77.30 with the tool provided with r80.10 iso

Following installation guide I have succesfully imported the tgz into a new r80.10 mds

Now I want to try to use this script to export the policy package from a migrated cma to a newly created cma but at the moment I encountered some issue like

-gateway object are not migrate with the following error :

Failed to import access-rule. Error: Requested object [partial_export_error_simp
le-gateway_cee6e5b1-8587-45a2-f62c-bd0e2ccd7146_fw.xxxxxxx] not found
Also failed to generate placeholder object: Validation failed with 1 warning

Failed to import access-rule. Error: Requested object [import_error_due_to_missi
ng_fields_partial_export_error_simple-gateway_cee6e5b1-8587-45a2-f62c-bd0e2ccd71
46_fwxxxxxxx.it] not found
Also failed to generate placeholder object: Validation failed with 1 warning and
1 error

And the final result is that I have a layer named with the oldest policy set name and the current Policy set standard have only the default rule , lthe created layer does not have nat rule.

Failed to import access-rule. Error: Requested object [partial_export_error_simple-gateway_cee6e5b1-8587-45a2-f62c-bd0e2ccd7146_fw.xxxxxx] not found

Also failed to generate placeholder object: Validation failed with 1 warning

Error analyzing package details! Aborting import.

Failed to attach layers to package! Error: Requested object [] not found. Import operation aborted.

Am I missing something here?

Thanks in advance

Robert_Decker · ‎2017-11-19

Hi,

It looks like an error related to a simple gateway object with UID "cee6e5b1-8587-45a2-f62c-bd0e2ccd7146".

Do you have such object in your DB?

Please run the tool again with a flag "--debug on", and it will produce a log file named "import_export.log".

Please send this file for analysis.

Robert.

Marco_Valenti · ‎2017-11-20

hey there

The object is the actual security gateway , there is anyway to send you the import_export file?

At the moment it does not seems to contain different output than the previous pasted one

regards

Robert_Decker · ‎2017-11-20

Hi,
We found the problem and fixed the code.
Please refer to the top of the page, go to the GitHub repo and download the updated code.
Please let me know if this solved the problem.
Robert.