Python tool for exporting/importing a policy package or parts of it

[Inbar_Moskovich]

Overview

ExportImportPolicyPackage tool enables you to export a policy package from a R80.x management database to a .tar.gz file, which can then be imported into any other R8x management database.

This tool can be used for backups, database transfers, testing, and more.

In the case you are exporting a policy package from a CMA, please verify that a global policy was NOT assigned to that CMA.
The tool doesn't support exporting a policy with global policy assigned!

The tool is referenced in https://support.checkpoint.com/results/sk/sk180923 

Description

This tool enables you to export a policy package (Access Policy, Threat Policy or both) from a management server into a .tar.gz file.

Notice

There are some types of objects that the script might not be able to export.
In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the log files to notify you of this.
In the Check Point SmartConsole you can easily replace each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

Instructions

Download the latest version from our GitHub repository: https://github.com/CheckPointSW/ExportImportPolicyPackage 
First, make sure you have [2.7.9 <= Python <= 2.7.14] installed on the machine running the script.
To export a package, run the import_export_package.py script. An interactive menu will guide you the rest of the way.
Command line flags may also be set in order to skip some or all of the menu.
A lot more details can of course be accessed with the [-h] option. This option also prints the current version of the tool.

Current tool version is V3.0.

Limitations

This export/import script does not gather all data from a given management server/CMA.
In general, it is limited by the R80.x Management APIs.
Specifically, this means:

• CMAs with a Global Policy assigned cannot be exported
  • Workaround: unassign the Global Policy prior to export
• Gateway/Cluster objects have to be recreated
  • Placeholder objects will be created
• UserCheck messages have to be recreated
  • Placeholder objects will be created
• The Internal Certificate Authority will not be copied. This means:
  • Re-establishing SIC with the appropriate gateways
  • Re-generating VPN certificates
  • Manually recreating HTTPS Inspection and DLP Rules
• Other objects not currently readable/writable via the R80.x API will not be copied

Tested on version

R8x
Releases earlier than R80 lack the necessary API support and are not supported.

Source Code Availability

The source code is available through GitHub: https://github.com/CheckPointSW/ExportImportPolicyPackage 

FAQ

Replies to this thread have locked.
Please refer to the FAQ below before you create a new post with your question.

When I run this tool, I get the message: APIResponse received a response which is not a valid JSON.

This most likely means you haven't enabled the API server yet.
See: https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Enabling-web-api/m-p/32641

I get an error message related to server fingerprint

Use the --unsafe option to ignore this error.

Can this tool export more than one policy package at a time?

Not currently, but you could call the tool in a script multiple times.

[Marcos_Garcia]

So, in the case of migrate a cma containg vsx objects, I will have to create manually all the vsx objects and with the script migrate the policy and objects, am I right?

[Robert_Decker]

There are some types of objects that the script might not be able to export.

In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the "export_error_log.elg" log file to notify you of this.

After the export process ends, you import the data into another server, and open the SmartConsole GUI.

In the SmartConsole GUI you can easily locate each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

Robert.

[John_White]

Im running R80.10 MDM with a few CMAs no JHF installed

From root I ran the following commands: 

==============================================

# ./opt/CPsuite-R80/fw1/Python/bin/python2.7 /home/admin/ExportImportPolicyPackage-master/import_export_package.py -m <CMA IP HERE>

Welcome to the Policy Package Import/Export Tool.
What would you like to do?
1. Import a package
2. Export a package
99. Exit
2
Please enter a Policy Package name to export:
MY_POLICY_NAME
Please select a login method:
1. Enter user credentials manually
2. Login as Root
3. Use an existing session file
4. Use an existing session UID
99. Back
2
The script will run with the following parameters:
Export Access-Control layers = True
Export Threat-Prevention layers = False
Output-file name = None
Management Server IP = 10.74.254.36
Management Server Port = 443
Management Server Domain = None
1. Change Settings
2. Run
99. Back
2
You currently do not have a record of this server's fingerprint.
Server's fingerprint: A5D408DBE6CCDD60296E615E858DEA1E96F3C0A3
Do you accept this fingerprint? [y/n] y
Fingerprint saved.
No package named 'MY_POLICY_NAME' found. Cannot export.

I tried mdsenv <CMA NAME> and running same command ... no luck. 

Confirmed MY_POLICY_NAME exists ... 

Tried several different policy names ... same issue.

Any thoughts?

Is there something I need to do special with Multi-Domain? 

PS: I tried an external JumpBox also tried user / pass ... same issues. 

[John_Tam]

You need run with this parameter import_export_package.py -m <MDS IP> -d <CMA IP>

[Robert_Decker]

Hi,

For MDM you should also specify the domain name you are acting on.

Either use a "-d" switch (-d "domain-name"), or select option 1 (Change Settings) and provide a domain name.

Robert.

[John_White]

Thanks for the quick response. The export worked however the import failed:

From Root within R80.10 MDM:

#  ./opt/CPsuite-R80/fw1/Python/bin/python2.7 /home/admin/ExportImportPolicyPackage-master/import_export_package.py -op import --all -m 10.0.0.1 -d MY_DOMAIN –r -n My_Policy -f /My_Policy.out.tar.gz

The import ran for a while then failed with the below message:

<---SNIP--->

Importing Access_Layer [IMPORTED LAYER My_Policy Security]

Nothing to import...

Importing Access_Layer [IMPORTED LAYER 1 My_Global_Policy Security]

Adding access-rules

Imported 1 out of 1 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER My_Policy Application]

Adding access-rules

Imported 2 out of 2 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER My_Policy Security]

Traceback (most recent call last):
File "/home/admin/ExportImportPolicyPackage-master/import_export_package.py", line 47, in <module>
import_package(client, args)
File "/home/admin/ExportImportPolicyPackage-master/importing/import_package.py", line 52, in import_package
layers_to_attach = import_objects(args.file, client, {})
File "/home/admin/ExportImportPolicyPackage-master/importing/import_objects.py", line 86, in import_objects
import_objects(rulebase_object_file.name, client, changed_layer_names, layer_name)
File "/home/admin/ExportImportPolicyPackage-master/importing/import_objects.py", line 19, in import_objects
export_tar = tarfile.open(file_name, "r:gz")
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/tarfile.py", line 1693, in open
return func(name, filemode, fileobj, **kwargs)
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/tarfile.py", line 1740, in gzopen
fileobj = gzip.GzipFile(name, mode, compresslevel, fileobj)
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/gzip.py", line 94, in __init__
fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: 'exported__access_layer__My_Policy Security__2018_03_03_15_50.tar.gz'

Any thoughts? 

[Robert_Decker]

Hi John,

I've noticed in your logs that an access policy layer named "My_Policy Security" appears twice.

Is it possible that you have defined that layer both in global domain and in CMA?

It seems that you have assigned a global policy to a CMA before exporting the policy, right?

Robert.

[John_White]

Hi Robert,

Thanks for your quick response. 

No, I do not have a global policy assigned to any CMA. 

I only have 2 layers in my Access Control policy: 

Security  (Firewall Blade Only)
Application (Application Control Blade Only)

This is the command I ran to export: 

# ./opt/CPsuite-R80/fw1/Python/bin/python2.7 /home/admin/ExportImportPolicyPackage-master/import_export_package.py -op export --all -m 10.0.0.1 -d LAB_DOMAIN -n LAB_Policy -r -o LAB_Policy.out

I'm trying to import this into a another Domain and it failed.

What I provided you previously is a scrub of the original log where I removed all customer related object information and IPs.

I deleted the policy in the target Domain and tried again and got the same results... ALL objects are imported just fine but no rule. 

===================================================================================================

Importing Access_Layer [IMPORTED LAYER 1 LAB_Policy Security]

Nothing to import...

Importing Access_Layer [IMPORTED LAYER 2 LAB_Global_Policy Security]

Adding access-rules

Imported 1 out of 1 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER 1 LAB_Policy Application]

Adding access-rules

Imported 2 out of 2 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER 1 LAB_Policy Security]

Traceback (most recent call last):
File "/home/admin/ExportImportPolicyPackage-master/import_export_package.py", line 47, in <module>
import_package(client, args)
File "/home/admin/ExportImportPolicyPackage-master/importing/import_package.py", line 52, in import_package
layers_to_attach = import_objects(args.file, client, {})
File "/home/admin/ExportImportPolicyPackage-master/importing/import_objects.py", line 86, in import_objects
import_objects(rulebase_object_file.name, client, changed_layer_names, layer_name)
File "/home/admin/ExportImportPolicyPackage-master/importing/import_objects.py", line 19, in import_objects
export_tar = tarfile.open(file_name, "r:gz")
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/tarfile.py", line 1693, in open
return func(name, filemode, fileobj, **kwargs)
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/tarfile.py", line 1740, in gzopen
fileobj = gzip.GzipFile(name, mode, compresslevel, fileobj)
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/gzip.py", line 94, in __init__
fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: 'exported__access_layer__LAB_Policy Security__2018_03_03_15_50.tar.gz'

===================================================================================================

I'm going to try exporting but this time I'll remove --all 

Any advice is appreciated. 

[John_White]

Tried exporting again without --all and imported both directly on the MDS and via a jump box with python 2.7.9 installed. Both came back with the exact same error ... from the error it says: 

IOError: [Errno 2] No such file or directory: 'exported__access_layer__LAB_Policy Security__2018_03_04_18_02.tar.gz' 

I searched for this file and I could not find it anywhere ... 

I extracted the Exported File I created and I found it in here ... there seems to be a problem with gzip.py unable to find this package within the Exported File. 

Any thoughts? 

[Robert_Decker]

Hi John,

From the logs I see that you have some data corruption there.

Please contact me via email when you are available.

Robert.

[John_White]

Hi Robert,

Thank you for your time today. You are right there was something wrong with the exported__access_layer__LAB_Policy Security__2018_03_03_15_50.tar.gz file ... apparently when opening the My_Policy.out.tar.gz file with WinRAR there was two files with the name "exported__access_layer__LAB_Policy Security__2018_03_03_15_50.tar.gz" ... what was happening during the import process it would extract the 1st file (aka GOOD File) then it would extract the 2nd file (aka BAD file) and overwrite the GOOD file. To resolve the issue I used WinRAR to extract the good file 1st, uploaded the My_Policy.out.tar.gz file to my MDS, extracted it, replaced the BAD file with the GOOD file, re-compressed the file into a tar.gz file and VOILA! The import was successful. 

SO ... I checked the CMA and unfortunately although the import was successful the in-line layer is not expanding

Not sure where to go from here ... 

[Robert_Decker]

I'll investigate your data to find out what is wrong and inform you.

If there is a bug in tool's code, it will be fixed.

Robert.

[David_Walker]

Good morning Robert, do you have any more information related to this issue from John?  We need to update the customer tomorrow and we will miss our window to get this completed.

David Walker

Team Manager, DMD Services

[Robert_Decker]

Hi David,

We started to investigate the issue on Monday, but I had a problem to use his management server afterwards due to credentials problem.

I sent him email, waiting for a response...

Robert.

[David_Herselman]

Firstly many thanks for the valuable time and effort you and your colleagues continue to invest, it's much appreciated!

I'm successfully able to run the tool and it generates an export archive. Upon extracting it and exploring the content it initially looked good but the actual access layer archives unfortunately contain no data. There don't appear to be any error messages, apart from 4 unexportable objects. Herewith a link to the content of the import_export.log file, I replaced references to the customer name with 'Client Name' and the first 3 octets of public IPs with 55.66.77:

http://txt.do/dxzcl 

The archive does contain archives within it but only the 'nat_rule' and 'threat-layer__Standard' contain csv and json data files:

-rw-r--r-- 1 root root 103 Mar 8 22:27 exported__access_layer__Application__2018_03_08_22_22.tar.gz
-rw-r--r-- 1 root root  99 Mar 8 22:26 exported__access_layer__Network__2018_03_08_22_22.tar.gz
-rw-r--r-- 1 root root 761 Mar 8 22:27 exported__nat_layer__Standard__2018_03_08_22_22.tar.gz
-rw-r--r-- 1 root root  95 Mar 8 22:27 exported__threat_layer__IPS__2018_03_08_22_22.tar.gz
-rw-r--r-- 1 root root 525 Mar 8 22:28 exported__threat_layer__Standard Threat Prevention__2018_03_08_22_22.tar.gz

More questions:

• Is this an official Check Point project?
• I can share the generated export archive, should it be beneficial. Perhaps you could provide a method of me sharing the file or a link to the file outside the public domain.
• Could I possibly ask for a switch to exclude exporting MDS global objects and policy rules? We can probably work around this by repackaging the resulting archive though. I'm a little strapped for time at the moment but intend to create an additional test domain to play with importing the exported policies, once we have one...

Regards

David Herselman

[Robert_Decker]

Hi David,

Thank you for your feedback.

This new version of tool contains fixes for issues raised mainly by you:

1. Using layer UIDs instead of names for compatibility with global policy assigned domains

2. Hiding a password typed by the user at the prompt

3. Removal of redundant output file

Are these items working now? Does a fix for the first item raised the current problem of empty archives for access layers?

In addition, there were some important stabilization fixes.

Now, for your questions:

1. This is NOT an official Check Point project. This is an open source initiative for the community, but is actively maintained by me.

2. Yes, I'd like to examine the generated archive. Maybe Dameon Welch Abernathy can assist here.

3. Any ideas or improvements are welcome and will be handled as soon as I can. Please remember that this is an open source and anyone can contribute!

Recently, I'm investing more time for fixes and developments related to MDM. It seems that these parts are not supported as needed.

regards,

Robert.

[PhoneBoy]

I'll set up an account on an FTP server for you David Herselman‌

I will send you a private message with the credentials.

[David_Herselman]

Hi Robert,

Apologies, yes the new version fixed my initial problems and I can now safely show other staff how to use the tool without revealing my credentials. The export now completes without errors and doesn't generate the redundant output file but it unfortunately contains everything except the actual policy rules (objects, categories, groups, nat rules, etc).

PS: I received an email from Dameon Welch Abernathy and will upload the generated archive and log files shortly...

Regards

David Herselman

[Marco_Valenti]

Hi Rob , that tool can be used for migrate cma with a vs too?

[Robert_Decker]

Hi Marco,

The main purpose of this tool is to export a policy package (access, threat) and all its objects from a management server database (smart center or CMA/domain), to be imported later into another management server database, or just for backup.

There are some types of objects that the script might not be able to export. In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the log files to notify you of this. In the Check Point SmartConsole you can easily replace each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

In this link you can see a list of all exportable objects types.

Unfortunatelly, virtual systems are not supported as they are not fully supported by R80/R80.10 yet.

Robert.

[Marco_Valenti]

Thanks for the detailed reply , let say that vs object are not exported they will be replaced by "export_error" object? my hope is to have at least the object from the cma where a vs where deployed

[Robert_Decker]

right, the placeholder should be there.

Robert.

[Robert_Decker]

IMPORTANT NOTE!!!

In the case you are exporting a policy package from a CMA, please verify that a global policy was NOT assigned to that CMA. The tool doesn't support exporting a policy with global policy assigned!

Robert.

[Robert_Decker]

Hi David,

I recieved your data, thanks to Dameon, and analyzed it.

As I've suspected, you were migrating from a CMA with global policy assigned.

I've to admit that recently I recieved many requests from inside and outside Check Point, facing the same issue, and posted here (bellow and on top in red) an important caveat about what this tool is supposed to support.

There is no way to export a CMA that has assigned a global policy.

One way to workaround this, is a manual work to un-assign the global policy for that CMA and then export.

Hope this helps,

Robert.

P.S. if you need any additional clarifications or assistance, please just ask for.

[PhoneBoy]

Can you export the global policy itself, though?

[Robert_Decker]

Yes, of course.

[PhoneBoy]

It's just you can't export the two when they are linked together.

Got it, thanks for confirming.

[Nader_Assi__Old]

Hi Robert Decker I just downloaded the latest version of this script and ran it to try to export my policy package. However it failed during the export as shown below:

Traceback (most recent call last):
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\import_export_package.py", line 45, in <module>
export_package(client, args)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_package.py", line 39, in export_package
= export_access_rulebase(show_package.data["name"], access_layer["name"], access_layer["uid"], client, timestamp, tar_file)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_access_rulebase.py", line 44, in export_access_rulebase
export_access_rulebase(package, access_layer["name"], access_layer["uid"], client, timestamp, tar_file)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_access_rulebase.py", line 59, in export_access_rulebase
export_general_objects(data_dict, api_type, object_dictionary[api_type], unexportable_objects, client)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_objects.py", line 313, in export_general_objects
api_type, group_object, client, unexportable_objects)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_objects.py", line 400, in get_group_objects
member_objects.extend(group_object[container])
TypeError: 'NoneType' object has no attribute '__getitem__'

Any thought?

Thanks,

Nader

[Robert_Decker]

Hi Nader,

It looks like a bug in the code due to some unique type/value of your data.

Please run again the tool and send to my email the API logs file "api.elg" from "$MDS_FWDIR/logs" folder on management server, and tool's log file from your working directory.

I'll look at it on Sunday, at work.

Robert.

[Nader_Assi__Old]

Hi Robert,

I’ve sent you the requested log files to your email address.

Thanks,

Nader