- CheckMates
- :
- Products
- :
- Developers
- :
- API / CLI Discussion
- :
- Re: Python tool for exporting/importing a policy p...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Are you a member of CheckMates?
×- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Python tool for exporting/importing a policy package or parts of it
Overview
ExportImportPolicyPackage tool enables you to export a policy package from a R80.x management database to a .tar.gz file, which can then be imported into any other R8x management database.
This tool can be used for backups, database transfers, testing, and more.
In the case you are exporting a policy package from a CMA, please verify that a global policy was NOT assigned to that CMA.
The tool doesn't support exporting a policy with global policy assigned!
The tool is referenced in https://support.checkpoint.com/results/sk/sk180923
Description
This tool enables you to export a policy package (Access Policy, Threat Policy or both) from a management server into a .tar.gz file.
Notice
There are some types of objects that the script might not be able to export.
In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the log files to notify you of this.
In the Check Point SmartConsole you can easily replace each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.
Instructions
Download the latest version from our GitHub repository: https://github.com/CheckPointSW/ExportImportPolicyPackage
First, make sure you have [2.7.9 <= Python <= 2.7.14] installed on the machine running the script.
To export a package, run the import_export_package.py script. An interactive menu will guide you the rest of the way.
Command line flags may also be set in order to skip some or all of the menu.
A lot more details can of course be accessed with the [-h] option. This option also prints the current version of the tool.
Current tool version is V3.0.
Limitations
This export/import script does not gather all data from a given management server/CMA.
In general, it is limited by the R80.x Management APIs.
Specifically, this means:
- CMAs with a Global Policy assigned cannot be exported
- Workaround: unassign the Global Policy prior to export
- Gateway/Cluster objects have to be recreated
- Placeholder objects will be created
- UserCheck messages have to be recreated
- Placeholder objects will be created
- The Internal Certificate Authority will not be copied. This means:
- Re-establishing SIC with the appropriate gateways
- Re-generating VPN certificates
- Manually recreating HTTPS Inspection and DLP Rules
- Other objects not currently readable/writable via the R80.x API will not be copied
Tested on version
R8x
Releases earlier than R80 lack the necessary API support and are not supported.
Source Code Availability
The source code is available through GitHub: https://github.com/CheckPointSW/ExportImportPolicyPackage
FAQ
Replies to this thread have locked.
Please refer to the FAQ below before you create a new post with your question.
When I run this tool, I get the message: APIResponse received a response which is not a valid JSON.
This most likely means you haven't enabled the API server yet.
See: https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Enabling-web-api/m-p/32641
I get an error message related to server fingerprint
Use the --unsafe option to ignore this error.
Can this tool export more than one policy package at a time?
Not currently, but you could call the tool in a script multiple times.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hey Robert
Thanks for the reply , i'll paste the new output
Creating a Policy Package named [test_import]
Importing general objects
Traceback (most recent call last):
File "C:\R80\ExportImportPolicyPackage-master\import_export_package.py", line
47, in <module>
import_package(client, args)
File "C:\R80\ExportImportPolicyPackage-master\importing\import_package.py", li
ne 48, in import_package
layers_to_attach = import_objects(args.file, client, {})
File "C:\R80\ExportImportPolicyPackage-master\importing\import_objects.py", li
ne 16, in import_objects
export_tar = tarfile.open(file_name, "r:gz")
File "C:\Python27\lib\tarfile.py", line 1693, in open
return func(name, filemode, fileobj, **kwargs)
File "C:\Python27\lib\tarfile.py", line 1747, in gzopen
t = cls.taropen(name, mode, fileobj, **kwargs)
File "C:\Python27\lib\tarfile.py", line 1723, in taropen
return cls(name, mode, fileobj, **kwargs)
File "C:\Python27\lib\tarfile.py", line 1587, in __init__
self.firstmember = self.next()
File "C:\Python27\lib\tarfile.py", line 2373, in next
raise ReadError("empty file")
tarfile.ReadError: empty file
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Marco,
Are you using the updated source?
The error you are facing is due to an invalid .tar.gz file...
Can you confirm that the export utility created a valid output file?
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for the fast reply , and yes I was pointing to an empy tgz , now with the new version everything seems to working fine the gateway object at the moment is getting imported with this name
partial_export_error_simple-gateway_cee6e5b1-8587-45a2-f62c-bd0e2ccd7146_fw.xxxxxx
Nat rule does not seem to getting imported is that by design?
Thanks again
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Excellent!
Only automatic (object) NAT rules are exported/imported at the moment.
Manual NAT rules support will be added soon.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm trying to export with the latest version of the script. It ends with an error:
Processing rules and exceptions
Exporting Exception-Rulebase from Threat-Rule #1 in Threat-Layer[Lab Threat Prevention]
Traceback (most recent call last):
File "import_export_package.py", line 45, in <module>
export_package(client, args)
File "C:\Python279\exporting\export_package.py", line 54, in export_package
= export_threat_rulebase(show_package.data["name"], threat_layer["name"], client)
File "C:\Python279\exporting\export_threat_rulebase.py", line 32, in export_threat_rulebase
export_threat_exception_rulebase(package, layer, rulebase_rule, exception_groups, client)
File "C:\Python279\exporting\export_threat_exception_rulebase.py", line 16, in export_threat_exception_rulebase
{"name": layer, "rule-uid": threat_rule["uid"]})
File "C:\Python279\exporting\export_objects.py", line 57, in get_query_rulebase_data
payload={"name": payload["name"], "package": payload["package"]}):
KeyError: 'package'
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Please edit the file "export_objects.py" inside folder "exporting" as follows -
goto line 56 -
for rulebase_reply in client.gen_api_query("show-" + api_type, details_level="full", container_keys=["rulebase"],
and replace the word "full" with the word "standard".
Run the tool again and see if this helps.
PLEASE inform my with the results.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Got to ask - I have been trying to reliably parse the policy xml export from using a python, and I have had to write so many exceptions in my code its driving me mad. Ideally, I'd like to get a policy imported into a dictionary list so I can query it (and avoid xml like the plague), or maybe even pickle it or just read it easily using a python scripts. Is there a library for this or has anyone tried this before? This would help me enormously for analyzing policy elements for consistency for instance.
Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is the modified line in export_objects.py
for rulebase_reply in client.gen_api_query("show-" + api_type, details_level="standard", container_keys=["rulebase"],
The export still fails.
Exporting Exception-Rulebase from Threat-Rule #1 in Threat-Layer[IPS]
Traceback (most recent call last):
File "import_export_package.py", line 45, in <module>
export_package(client, args)
File "C:\Python279\exporting\export_package.py", line 54, in export_package
= export_threat_rulebase(show_package.data["name"], threat_layer["name"], client)
File "C:\Python279\exporting\export_threat_rulebase.py", line 32, in export_threat_rulebase
export_threat_exception_rulebase(package, layer, rulebase_rule, exception_groups, client)
File "C:\Python279\exporting\export_threat_exception_rulebase.py", line 16, in export_threat_exception_rulebase
{"name": layer, "rule-uid": threat_rule["uid"]})
File "C:\Python279\exporting\export_objects.py", line 57, in get_query_rulebase_data
payload={"name": payload["name"], "package": payload["package"]}):
KeyError: 'package'
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I'm very sorry for this inconvenience. There is a bug in our code that causes this exception.
We will fix it on sunday next week and inform you.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, we fixed the bug and uploaded a new version to GitHub repo (the link is on top).
Please inform if this works for you.
Thanks, Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, it seems to be working now.
How far along are you with exporting manual NAT rules?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Glad to hear.
Manual NAT export is on our roadmap during coming weeks.
We will update this forum when done.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
When will you be able to include Manual NAT ? we have big migration and it can help alot.
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I've already answered this question here several times.
This task is on our roadmap for upcoming weeks, I cannot tell exactly when.
What are you migrating? Another vendor's configuration to Check Point?
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Sorry for repeating it again, No, from one SMS to another SMS. same R80.10
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I've used this script to export 2 policy package from a SMS running R80.10. One policy is a regular/basic Access Control and the other one is an Inline Layer Access Control.
I can successfully export the regular one but not the Inline one. The size of the regular one is 63KB and the Inline is 1KB (and if I try to open the Archive it gives me an error). The export of the Inline seems to stop at one time but I don't get any error message.
Is there a way to debug the export process?
Any assistance would be appreciated.
Thanks,
Nader
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Run the tool with a flag "--debug on", and it will produce a log file named "import_export.log".
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I ran the following command and it did not generate any output file neither update the "import_export.log"
C:\Python27>python.exe "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\import_export_package.py" --op export -n NV_AA_Prod_QA_Inline_Policy -u admin -p ********* n -m ******** --debug on
I ran the same command with 'debug off' and got the same result.
In both cases it shows during the execution the following messages:
Traceback (most recent call last):
File "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\import_export_package.py", line 45, in <module>
export_package(client, args)
File "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\exporting\export_package.py", line 38, in export_package
= export_access_rulebase(show_package.data["name"], access_layer["name"], client, timestamp, tar_file)
File "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\exporting\export_access_rulebase.py", line 47, in export_access_rulebase
timestamp, ["access-rule", "access-section"], client.api_version)
File "C:\Users\Administrator\Documents\CP Scripts\Pyhon tool for exporting-importing a policy package\ExportImportPolicyPackage-master\utils.py", line 208, in create_tar_file
with tarfile.open(layer_tar_name, "w:gz") as tar:
File "C:\Python27\lib\tarfile.py", line 1693, in open
return func(name, filemode, fileobj, **kwargs)
File "C:\Python27\lib\tarfile.py", line 1740, in gzopen
fileobj = gzip.GzipFile(name, mode, compresslevel, fileobj)
File "C:\Python27\lib\gzip.py", line 94, in __init__
fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: u'exported__access_layer__AA Prod to AA QA/PDS/OTS__2017_12_14_12_52.tar.gz'
Thought?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The output states that there was an error to create an archive file.
The name of the archive file is very strange - "exported__access_layer__AA Prod to AA QA/PDS/OTS__2017_12_14_12_52", where "AA Prod to AA QA/PDS/OTS" is a name of an inline layer.
Do you have an inline layer with such name?
If yes, try to change it to something simple and see what happens.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It worked after replacing the '/' with '-'. Now I need to test the import.
The other thing I noticed is that it wasn't able to export a cluster object, for instance I got the error message "Object of type CpmiGatewayCluster with uid 3855de8e-dfc3-4b29-8088-aed930789947 named ANNTMSCXL01 is not exportable. Its name was changed to export_error_CpmiGatewayCluster_3855de8e-dfc3-4b29-8088-aed930789947_ANNTMSCXL01"
Thanks for your help!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Robert Decker I tested the import and it worked for both Regular and Inline policy packages. However for the Inline one I noticed that it created multiple policies associated with the Inline package. Each policy is basically a copy of an Inline layer sub-rules. I've attached a screenshot below that should help understand what I'm seeing:
And I've also pasted below a copy of the Command line output during the import:
Importing Access_Layer [AA ICCP to AA Prod]
Adding access-rules
Imported 13 out of 13 access-rules (100%)
Importing Access_Layer [AA ICCP to NV ICCP]
Adding access-rules
Imported 7 out of 7 access-rules (100%)
Importing Access_Layer [AA ICCP to NV Prod]
Adding access-rules
Imported 5 out of 5 access-rules (100%)
Importing Access_Layer [AA Prod to AA ICCP]
Adding access-rules
Imported 15 out of 15 access-rules (100%)
Importing Access_Layer [AA Prod to AA QA-PDS-OTS]
Adding access-rules
Imported 6 out of 6 access-rules (100%)
Importing Access_Layer [AA Prod to NV ICCP]
Adding access-rules
Imported 6 out of 6 access-rules (100%)
Importing Access_Layer [AA Prod to NV Prod]
Adding access-rules
Imported 20 out of 34 access-rules (58%)
Imported 34 out of 34 access-rules (100%)
Importing Access_Layer [AA Prod to NV QA-PDS]
Adding access-rules
Imported 6 out of 6 access-rules (100%)
Importing Access_Layer [AA QA-PDS-OTS to NV QA-PDS]
Adding access-rules
Imported 20 out of 39 access-rules (51%)
Imported 39 out of 39 access-rules (100%)
Importing Access_Layer [NV ICCP to AA ICCP]
Adding access-rules
Imported 9 out of 9 access-rules (100%)
Importing Access_Layer [NV ICCP to AA Prod]
Adding access-rules
Imported 5 out of 5 access-rules (100%)
Importing Access_Layer [NV ICCP to NV Prod]
Adding access-rules
Imported 13 out of 13 access-rules (100%)
Importing Access_Layer [NV Prod to AA ICCP]
Adding access-rules
Imported 8 out of 8 access-rules (100%)
Importing Access_Layer [NV Prod to AA Prod]
Adding access-rules
Imported 20 out of 40 access-rules (50%)
Imported 40 out of 40 access-rules (100%)
Importing Access_Layer [NV Prod to AA QA]
Adding access-rules
Imported 5 out of 5 access-rules (100%)
Importing Access_Layer [NV Prod to NV ICCP]
Adding access-rules
Imported 18 out of 18 access-rules (100%)
Importing Access_Layer [NV Prod to NV QA-PDS]
Adding access-rules
Imported 9 out of 9 access-rules (100%)
Importing Access_Layer [NV QA-PDS to AA QA-PDS-OTS]
Adding access-rules
Imported 20 out of 41 access-rules (48%)
Imported 40 out of 41 access-rules (97%)
Imported 41 out of 41 access-rules (100%)
Importing Access_Layer [IMPORTED LAYER NV_AA_Prod_QA_Inline_Policy Network]
Adding access-rules
Imported 20 out of 90 access-rules (22%)
Imported 40 out of 90 access-rules (44%)
Imported 60 out of 90 access-rules (66%)
Imported 80 out of 90 access-rules (88%)
Imported 90 out of 90 access-rules (100%)
Adding access-sections
Imported 20 out of 30 access-sections (66%)
Imported 30 out of 30 access-sections (100%)
Is it by design?
Thanks,
Nader
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'll check this and get back with answers ASAP.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Would just like to add a note, that we experienced similar behavior. Had a couple of gateway objects in the database (admin error), and all were exported as export_error.... objects.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Gateways/Clusters/Virtual Systems are legacy database objects (R77.30), and therefore cannot be exported/imported directly.
In the case of Simple Gateway object, it has a wrapper in R80.X database, hence it can be manipulated.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Nader,
You are correct, but this doesn't look right for me.
I'll check with the tool's developer to verify if this behaviour is intended.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is a bug, not a desired behaviour.
We will fix it ASAP.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Robert,
Any luck on updating the script?
Thanks and happy new year!
Nader
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sure, we are working on a solution. Not an easy one...
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Problem fixed!
Please go to the GitHub repo link on top of this post and download the updated source.
Robert.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi all,
There is a major fix for this tool, if you are exporting/importing a policy with inline layers.
Please go to the Github repo link (at the top of this post) and download the updated source.
Thanks to Nader for reporting this problem.
Robert.