Python tool for exporting/importing a policy package or parts of it

[Inbar_Moskovich]

Overview

ExportImportPolicyPackage tool enables you to export a policy package from a R80.x management database to a .tar.gz file, which can then be imported into any other R80.x management database.

This tool can be used for backups, database transfers, testing, and more.

In the case you are exporting a policy package from a CMA, please verify that a global policy was NOT assigned to that CMA.
The tool doesn't support exporting a policy with global policy assigned!

Description

This tool enables you to export a policy package (Access Policy, Threat Policy or both) from a management server into a .tar.gz file.

Notice

There are some types of objects that the script might not be able to export.
In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the log files to notify you of this.
In the Check Point SmartConsole you can easily replace each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

Instructions

Download the latest version from our GitHub repository: https://github.com/CheckPointSW/ExportImportPolicyPackage 
First, make sure you have [2.7.9 <= Python <= 2.7.14] installed on the machine running the script.
To export a package, run the import_export_package.py script. An interactive menu will guide you the rest of the way.
Command line flags may also be set in order to skip some or all of the menu.
A lot more details can of course be accessed with the [-h] option. This option also prints the current version of the tool.

Current tool version is V3.0.

Limitations

This export/import script does not gather all data from a given management server/CMA.
In general, it is limited by the R80.x Management APIs.
Specifically, this means:

• CMAs with a Global Policy assigned cannot be exported
  • Workaround: unassign the Global Policy prior to export
• Gateway/Cluster objects have to be recreated
  • Placeholder objects will be created
• UserCheck messages have to be recreated
  • Placeholder objects will be created
• The Internal Certificate Authority will not be copied. This means:
  • Re-establishing SIC with the appropriate gateways
  • Re-generating VPN certificates
  • Manually recreating HTTPS Inspection and DLP Rules
• Other objects not currently readable/writable via the R80.x API will not be copied

Tested on version

R80.x
Releases earlier than R80 lack the necessary API support and are not supported.

Source Code Availability

The source code is available through GitHub: https://github.com/CheckPointSW/ExportImportPolicyPackage 

FAQ

Replies to this thread have locked.
Please refer to the FAQ below before you create a new post with your question.

When I run this tool, I get the message: APIResponse received a response which is not a valid JSON.

This most likely means you haven't enabled the API server yet.
See: https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Enabling-web-api/m-p/32641

I get an error message related to server fingerprint

Use the --unsafe option to ignore this error.

Can this tool export more than one policy package at a time?

Not currently, but you could call the tool in a script multiple times.

NOTICE: By using this sample code you agree to terms and conditions in this Terms and Conditions

[Marco_Valenti]

hey Robert

Thanks for the reply , i'll paste the new output

Creating a Policy Package named [test_import]

Importing general objects

Traceback (most recent call last):
  File "C:\R80\ExportImportPolicyPackage-master\import_export_package.py", line
47, in <module>
    import_package(client, args)
  File "C:\R80\ExportImportPolicyPackage-master\importing\import_package.py", li
ne 48, in import_package
    layers_to_attach = import_objects(args.file, client, {})
  File "C:\R80\ExportImportPolicyPackage-master\importing\import_objects.py", li
ne 16, in import_objects
    export_tar = tarfile.open(file_name, "r:gz")
  File "C:\Python27\lib\tarfile.py", line 1693, in open
    return func(name, filemode, fileobj, **kwargs)
  File "C:\Python27\lib\tarfile.py", line 1747, in gzopen
    t = cls.taropen(name, mode, fileobj, **kwargs)
  File "C:\Python27\lib\tarfile.py", line 1723, in taropen
    return cls(name, mode, fileobj, **kwargs)
  File "C:\Python27\lib\tarfile.py", line 1587, in __init__
    self.firstmember = self.next()
  File "C:\Python27\lib\tarfile.py", line 2373, in next
    raise ReadError("empty file")
tarfile.ReadError: empty file

[Robert_Decker]

Marco,

Are you using the updated source?

The error you are facing is due to an invalid .tar.gz file...

Can you confirm that the export utility created a valid output file?

Robert.

[Marco_Valenti]

Thanks for the fast reply , and yes I was pointing to an empy tgz , now with the new version everything seems to working fine the gateway object at the moment is getting imported with this name

partial_export_error_simple-gateway_cee6e5b1-8587-45a2-f62c-bd0e2ccd7146_fw.xxxxxx

Nat rule does not seem to getting imported is that by design?

Thanks again

[Robert_Decker]

Excellent!

Only automatic (object) NAT rules are exported/imported at the moment.

Manual NAT rules support will be added soon.

Robert.

[Borut]

I'm trying to export with the latest version of the script. It ends with an error:

Processing rules and exceptions

Exporting Exception-Rulebase from Threat-Rule #1 in Threat-Layer[Lab Threat Prevention]

Traceback (most recent call last):
File "import_export_package.py", line 45, in <module>
export_package(client, args)
File "C:\Python279\exporting\export_package.py", line 54, in export_package
= export_threat_rulebase(show_package.data["name"], threat_layer["name"], client)
File "C:\Python279\exporting\export_threat_rulebase.py", line 32, in export_threat_rulebase
export_threat_exception_rulebase(package, layer, rulebase_rule, exception_groups, client)
File "C:\Python279\exporting\export_threat_exception_rulebase.py", line 16, in export_threat_exception_rulebase
{"name": layer, "rule-uid": threat_rule["uid"]})
File "C:\Python279\exporting\export_objects.py", line 57, in get_query_rulebase_data
payload={"name": payload["name"], "package": payload["package"]}):
KeyError: 'package'

[Robert_Decker]

Hi,

Please edit the file "export_objects.py" inside folder "exporting" as follows - 

goto line 56 - 

for rulebase_reply in client.gen_api_query("show-" + api_type, details_level="full", container_keys=["rulebase"],

and replace the word "full" with the word "standard".

Run the tool again and see if this helps.

PLEASE inform my with the results.

Robert.

[Nicholas_Sherid]

Got to ask - I have been trying to reliably parse the policy xml export from using a python, and I have had to write so many exceptions in my code its driving me mad.   Ideally, I'd like to get a policy imported into a dictionary list so I can query it (and avoid xml like the plague), or maybe even pickle it or just read it easily using a python scripts.  Is there a library for this or has anyone tried this before?  This would help me enormously for analyzing policy elements for consistency for instance.

Thanks!

[Borut]

This is the modified line in export_objects.py

for rulebase_reply in client.gen_api_query("show-" + api_type, details_level="standard", container_keys=["rulebase"],

The export still fails.

Exporting Exception-Rulebase from Threat-Rule #1 in Threat-Layer[IPS]

Traceback (most recent call last):
File "import_export_package.py", line 45, in <module>
export_package(client, args)
File "C:\Python279\exporting\export_package.py", line 54, in export_package
= export_threat_rulebase(show_package.data["name"], threat_layer["name"], client)
File "C:\Python279\exporting\export_threat_rulebase.py", line 32, in export_threat_rulebase
export_threat_exception_rulebase(package, layer, rulebase_rule, exception_groups, client)
File "C:\Python279\exporting\export_threat_exception_rulebase.py", line 16, in export_threat_exception_rulebase
{"name": layer, "rule-uid": threat_rule["uid"]})
File "C:\Python279\exporting\export_objects.py", line 57, in get_query_rulebase_data
payload={"name": payload["name"], "package": payload["package"]}):
KeyError: 'package'

[Robert_Decker]

Hi,

I'm very sorry for this inconvenience. There is a bug in our code that causes this exception.

We will fix it on sunday next week and inform you.

Robert.

[Robert_Decker]

Hi, we fixed the bug and uploaded a new version to GitHub repo (the link is on top).

Please inform if this works for you.

Thanks, Robert.

[Borut]

Hi, it seems to be working now.

How far along are you with exporting manual NAT rules?

[Robert_Decker]

Glad to hear.

Manual NAT export is on our roadmap during coming weeks.

We will update this forum when done.

Robert.

[SD_Networking]

Hello,

When will you be able to include Manual NAT ? we have big migration and it can help alot.

Thanks

[Robert_Decker]

Hi,

I've already answered this question here several times.

This task is on our roadmap for upcoming weeks, I cannot tell exactly when.

What are you migrating? Another vendor's configuration to Check Point?

Robert.

[SD_Networking]

Hello Sorry for repeating it again, No, from one SMS to another SMS. same R80.10

[Nader_Assi__Old]

Hi,

I've used this script to export 2 policy package from a SMS running R80.10. One policy is a regular/basic Access Control and the other one is an Inline Layer Access Control.

I can successfully export the regular one but not the Inline one. The size of the regular one is 63KB and the Inline is 1KB (and if I try to open the Archive it gives me an error). The export of the Inline seems to stop at one time but I don't get any error message.

Is there a way to debug the export process?

Any assistance would be appreciated.

Thanks,

Nader

[Robert_Decker]

Run the tool with a flag "--debug on", and it will produce a log file named "import_export.log".

[Nader_Assi__Old]

I ran the following command and it did not generate any output file neither update the "import_export.log"

C:\Python27>python.exe "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\import_export_package.py" --op export -n NV_AA_Prod_QA_Inline_Policy -u admin -p ********* n -m ******** --debug on

I ran the same command with 'debug off' and got the same result.

In both cases it shows during the execution the following messages:

Traceback (most recent call last):
File "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\import_export_package.py", line 45, in <module>
export_package(client, args)
File "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\exporting\export_package.py", line 38, in export_package
= export_access_rulebase(show_package.data["name"], access_layer["name"], client, timestamp, tar_file)
File "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\exporting\export_access_rulebase.py", line 47, in export_access_rulebase
timestamp, ["access-rule", "access-section"], client.api_version)
File "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\utils.py", line 208, in create_tar_file
with tarfile.open(layer_tar_name, "w:gz") as tar:
File "C:\Python27\lib\tarfile.py", line 1693, in open
return func(name, filemode, fileobj, **kwargs)
File "C:\Python27\lib\tarfile.py", line 1740, in gzopen
fileobj = gzip.GzipFile(name, mode, compresslevel, fileobj)
File "C:\Python27\lib\gzip.py", line 94, in __init__
fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: u'exported__access_layer__AA Prod to AA QA/PDS/OTS__2017_12_14_12_52.tar.gz'

Thought?

[Robert_Decker]

The output states that there was an error to create an archive file.

The name of the archive file is very strange - "exported__access_layer__AA Prod to AA QA/PDS/OTS__2017_12_14_12_52", where "AA Prod to AA QA/PDS/OTS" is a name of an inline layer.

Do you have an inline layer with such name?

If yes, try to change it to something simple and see what happens.

[Nader_Assi__Old]

It worked after replacing the '/' with '-'. Now I need to test the import.

The other thing I noticed is that it wasn't able to export a cluster object, for instance I got the error message "Object of type CpmiGatewayCluster with uid 3855de8e-dfc3-4b29-8088-aed930789947 named ANNTMSCXL01 is not exportable. Its name was changed to export_error_CpmiGatewayCluster_3855de8e-dfc3-4b29-8088-aed930789947_ANNTMSCXL01"

Thanks for your help!

[Nader_Assi__Old]

Robert Decker‌ I tested the import and it worked for both Regular and Inline policy packages. However for the Inline one I noticed that it created multiple policies associated with the Inline package. Each policy is basically a copy of an Inline layer sub-rules. I've attached a screenshot below that should help understand what I'm seeing:

And I've also pasted below a copy of the Command line output during the import:

Importing Access_Layer [AA ICCP to AA Prod]

Adding access-rules

Imported 13 out of 13 access-rules (100%)

Importing Access_Layer [AA ICCP to NV ICCP]

Adding access-rules

Imported 7 out of 7 access-rules (100%)

Importing Access_Layer [AA ICCP to NV Prod]

Adding access-rules

Imported 5 out of 5 access-rules (100%)

Importing Access_Layer [AA Prod to AA ICCP]

Adding access-rules

Imported 15 out of 15 access-rules (100%)

Importing Access_Layer [AA Prod to AA QA-PDS-OTS]

Adding access-rules

Imported 6 out of 6 access-rules (100%)

Importing Access_Layer [AA Prod to NV ICCP]

Adding access-rules

Imported 6 out of 6 access-rules (100%)

Importing Access_Layer [AA Prod to NV Prod]

Adding access-rules

Imported 20 out of 34 access-rules (58%)

Imported 34 out of 34 access-rules (100%)

Importing Access_Layer [AA Prod to NV QA-PDS]

Adding access-rules

Imported 6 out of 6 access-rules (100%)

Importing Access_Layer [AA QA-PDS-OTS to NV QA-PDS]

Adding access-rules

Imported 20 out of 39 access-rules (51%)

Imported 39 out of 39 access-rules (100%)

Importing Access_Layer [NV ICCP to AA ICCP]

Adding access-rules

Imported 9 out of 9 access-rules (100%)

Importing Access_Layer [NV ICCP to AA Prod]

Adding access-rules

Imported 5 out of 5 access-rules (100%)

Importing Access_Layer [NV ICCP to NV Prod]

Adding access-rules

Imported 13 out of 13 access-rules (100%)

Importing Access_Layer [NV Prod to AA ICCP]

Adding access-rules

Imported 8 out of 8 access-rules (100%)

Importing Access_Layer [NV Prod to AA Prod]

Adding access-rules

Imported 20 out of 40 access-rules (50%)

Imported 40 out of 40 access-rules (100%)

Importing Access_Layer [NV Prod to AA QA]

Adding access-rules

Imported 5 out of 5 access-rules (100%)

Importing Access_Layer [NV Prod to NV ICCP]

Adding access-rules

Imported 18 out of 18 access-rules (100%)

Importing Access_Layer [NV Prod to NV QA-PDS]

Adding access-rules

Imported 9 out of 9 access-rules (100%)

Importing Access_Layer [NV QA-PDS to AA QA-PDS-OTS]

Adding access-rules

Imported 20 out of 41 access-rules (48%)

Imported 40 out of 41 access-rules (97%)

Imported 41 out of 41 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER NV_AA_Prod_QA_Inline_Policy Network]

Adding access-rules

Imported 20 out of 90 access-rules (22%)

Imported 40 out of 90 access-rules (44%)

Imported 60 out of 90 access-rules (66%)

Imported 80 out of 90 access-rules (88%)

Imported 90 out of 90 access-rules (100%)

Adding access-sections

Imported 20 out of 30 access-sections (66%)

Imported 30 out of 30 access-sections (100%)

Is it by design?

Thanks,

Nader

[Robert_Decker]

I'll check this and get back with answers ASAP.

Robert.

[Borut]

Would just like to add a note, that we experienced similar behavior. Had a couple of gateway objects in the database (admin error), and all were exported as export_error.... objects.

[Robert_Decker]

Gateways/Clusters/Virtual Systems are legacy database objects (R77.30), and therefore cannot be exported/imported directly.

In the case of Simple Gateway object, it has a wrapper in R80.X database, hence it can be manipulated.

Robert.

[Robert_Decker]

Hi Nader,

You are correct, but this doesn't look right for me.

I'll check with the tool's developer to verify if this behaviour is intended.

Robert.

[Robert_Decker]

This is a bug, not a desired behaviour.

We will fix it ASAP.

Robert.

[Nader_Assi__Old]

Hi Robert,

Any luck on updating the script?

Thanks and happy new year!

Nader

[Robert_Decker]

Sure, we are working on a solution. Not an easy one...

Robert.

[Robert_Decker]

Problem fixed!

Please go to the GitHub repo link on top of this post and download the updated source.

Robert.

[Robert_Decker]

Hi all,

There is a major fix for this tool, if you are exporting/importing a policy with inline layers.

Please go to the Github repo link (at the top of this post) and download the updated source.

Thanks to Nader for reporting this problem.

Robert.