Python tool for exporting/importing a policy package or parts of it

[Inbar_Moskovich]

Overview

ExportImportPolicyPackage tool enables you to export a policy package from a R80.x management database to a .tar.gz file, which can then be imported into any other R8x management database.

This tool can be used for backups, database transfers, testing, and more.

In the case you are exporting a policy package from a CMA, please verify that a global policy was NOT assigned to that CMA.
The tool doesn't support exporting a policy with global policy assigned!

The tool is referenced in https://support.checkpoint.com/results/sk/sk180923 

Description

This tool enables you to export a policy package (Access Policy, Threat Policy or both) from a management server into a .tar.gz file.

Notice

There are some types of objects that the script might not be able to export.
In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the log files to notify you of this.
In the Check Point SmartConsole you can easily replace each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

Instructions

Download the latest version from our GitHub repository: https://github.com/CheckPointSW/ExportImportPolicyPackage 
First, make sure you have [2.7.9 <= Python <= 2.7.14] installed on the machine running the script.
To export a package, run the import_export_package.py script. An interactive menu will guide you the rest of the way.
Command line flags may also be set in order to skip some or all of the menu.
A lot more details can of course be accessed with the [-h] option. This option also prints the current version of the tool.

Current tool version is V3.0.

Limitations

This export/import script does not gather all data from a given management server/CMA.
In general, it is limited by the R80.x Management APIs.
Specifically, this means:

• CMAs with a Global Policy assigned cannot be exported
  • Workaround: unassign the Global Policy prior to export
• Gateway/Cluster objects have to be recreated
  • Placeholder objects will be created
• UserCheck messages have to be recreated
  • Placeholder objects will be created
• The Internal Certificate Authority will not be copied. This means:
  • Re-establishing SIC with the appropriate gateways
  • Re-generating VPN certificates
  • Manually recreating HTTPS Inspection and DLP Rules
• Other objects not currently readable/writable via the R80.x API will not be copied

Tested on version

R8x
Releases earlier than R80 lack the necessary API support and are not supported.

Source Code Availability

The source code is available through GitHub: https://github.com/CheckPointSW/ExportImportPolicyPackage 

FAQ

Replies to this thread have locked.
Please refer to the FAQ below before you create a new post with your question.

When I run this tool, I get the message: APIResponse received a response which is not a valid JSON.

This most likely means you haven't enabled the API server yet.
See: https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Enabling-web-api/m-p/32641

I get an error message related to server fingerprint

Use the --unsafe option to ignore this error.

Can this tool export more than one policy package at a time?

Not currently, but you could call the tool in a script multiple times.

[Marcel_M]

This guide helped me alot:

https://www.hurricanelabs.com/blog/check-point-api-merging-management-servers-with-r80-10 

But if I do a Import, I have all the Objects and Section but no Rules, I get this Output, it seems the Object Internet is the issue:

Internet is a ANY Object which excluded the the Privat RFC 1918 address spaces. Anyone any idea or hint what to try?

Fingerprint saved.
Creating a Policy Package named [Office]

Importing general objects

Adding services-udp

Imported 9 out of 9 services-udp (100%)

Adding services-tcp

Imported 18 out of 18 services-tcp (100%)

Adding simple-gateways

Imported 1 out of 1 simple-gateways (100%)

Adding access-layers

Imported 2 out of 2 access-layers (100%)

Adding networks

Imported 12 out of 12 networks (100%)

Adding hosts

Imported 19 out of 19 hosts (100%)

Adding groups

Imported 4 out of 4 groups (100%)

Adding groups-with-exclusion

Imported 1 out of 1 groups-with-exclusion (100%)

Importing Access_Layer [Mom-Policy Network]

Adding access-rules

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Imported 20 out of 35 access-rules (57%)

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Not unique name problem "Internet" - changing payload to use UID instead.

Imported 35 out of 35 access-rules (100%)

Failed to publish import of access-rules from tar file #1! Access-rules from said file were not imported!. Error: Publish failed because of validation errors

Adding access-sections

Imported 7 out of 7 access-sections (100%)

Importing Access_Layer [AppCtrl]

Adding access-rules

Imported 1 out of 1 access-rules (100%)

[Robert_Decker]

Hi,

First of all, I read the link that you mentioned. Very good document, but not accurate - Gaia server has several python versions installed. You should use "python2.7" one.

Regarding your problem, can you please attach a screenshot of one of access rules that has Internet object reference and fails?

Robert.

[Marcel_M]

Sure here is a screenshot of it:

I am not sure but I guess this could be the Object which cause the failure:

Because of this Error Message which appeared 12 times in the log and the Object is also 12 times used in the Rulebase.

Not unique name problem "Internet" - changing payload to use UID instead.

Failed to publish import of access-rules from tar file #1! Access-rules from said file were not imported!. Error: Publish failed because of validation errors

and here the Details of the Group "Net-Group-RF-191" which is the except:

[Robert_Decker]

Hi Marcel,

Check Point system database already contains an object named Internet. Just open "Object Explorer" window and you will see it - 

When you work in UI and select "Internet", it knows by context which type is selected.

In API we cannot say what is the context and which type should be used.

Just rename you object to Internet1 and it will be ok.

Robert.

[Marcel_M]

Thx for the fast help this worked perfect now !!!

But one more question off topic, what exactly is this system defined Internet Object --> I don't see it in the "Objects Bar (F11)" But I can see it in the Object Explorer also like you on your screenshot.

Does this Object kind of the same which I created or which networks are included and excluded of this object?

[Robert_Decker]

Objects Bar - Displayes objects by predefined system categories.

Objects Explorer - Displayed all objects and allows filtering by system categories and user defined categories (object tags).

Internet - predefined system network object - for use in rulebase to represent gateway's external interfaces.

Robert.

[Robert_Decker]

Hi all,

There is a new fix for this tool, if you are exporting/importing a policy with rulebase sections.

Please go to the Github repo link (at the top of this post) and download the updated source (including the linked python sdk).

Robert.

[David_Herselman]

We are running MDS R80.10 and would like to use a domain as a template. We already use a Global policy to enforce standard Network and Application layers but would like to avoid the lengthy process of subsequently editing the domain's inline policies (also called 'Network' and 'Application').

Does the tool work with MDS?

[Robert_Decker]

Yes, the tool works with MDS as well.

Use -h flag to see all options to run the tool.

[Laurent_Raynaud]

Hi all,

We are currently running this script to import our policies and we met this error message :

"

Traceback (most recent call last):
File "/home/admin/script/import_export_package.py", line 45, in <module>
export_package(client, args)
File "/home/admin/script/exporting/export_package.py", line 38, in export_package
= export_access_rulebase(show_package.data["name"], access_layer["name"], client, timestamp, tar_file)
File "/home/admin/script/exporting/export_access_rulebase.py", line 16, in export_access_rulebase
get_query_rulebase_data(client, "access-rulebase", {"name": layer, "package": package})
File "/home/admin/script/exporting/export_objects.py", line 131, in get_query_rulebase_data
section["to"] = rulebase_item["to"]
KeyError: 'to'

"

Can you help me to understand where the problem come from ?

Thanks.

LR.

[Robert_Decker]

Hi,

Right, this was already reported yesterday in tool's github repo and we've just uploaded the fix.

Please download the updated source and try again.

Thanks,

Robert.

[Robert_Decker]

Hi all,

Added a support for manual NAT rules.
Enjoy!

[Robert_Decker]

Please note that NAT rulebase is different from Access rulebase and doesn't contain ordered layers.

Therefore, imported NAT rulebase cannot be created asside of existing NAT rulebase.

In order to avoid merging of rules, here is the importing process, in pictures:

Original NAT rulebase that is exported

NAT rulebase imported into another database

As you can see, rules order is preserved, under dedicated new sections, but existing original sections are omitted.

This work should be completed manually, if needed.

Any comments are welcomed!

Robert.

[Laurent_Raynaud]

Still the same error :

Traceback (most recent call last):
File "/home/admin/script/import_export_package.py", line 45, in <module>
export_package(client, args)
File "/home/admin/script/exporting/export_package.py", line 38, in export_pack age
= export_access_rulebase(show_package.data["name"], access_layer["name"], cl ient, timestamp, tar_file)
File "/home/admin/script/exporting/export_access_rulebase.py", line 16, in exp ort_access_rulebase
get_query_rulebase_data(client, "access-rulebase", {"name": layer, "package" : package})
File "/home/admin/script/exporting/export_objects.py", line 131, in get_query_ rulebase_data
section["to"] = rulebase_item["to"]
KeyError: 'to'

How can I debug it ?

Thank you for your help.

LR.

[Robert_Decker]

You are NOT using the updated version.

Please download it from github repo. Here is the updated code snippet that was fixed - 

Robert.

[Robert_Decker]

Delivered!

[Robert_Decker]

Delivered!

[David_Herselman]

Hi Robert,

I setup a dedicated CentOS 7 VM, altinstalled Python 2.7.14, cloned the Github project and fired off the tool. I'm hoping to ultimately save time and avoid mistakes deploying new tenants in our MDS environment and am happy to see that we can extract the export file and edit the resulting files in a standard text editor. Nice work on the NAT rules, they export perfectly and it would be allot faster creating section titles and simply moving some rules around, instead of creating everything from scratch every time.

The actual domain policies don't export and receive the following errors in the 'export_error_log.elg' file:

Failed to retrieve layer named 'Network'! Error: Requested object name [Network] is not unique.. Layer was not exported!

Failed to retrieve layer named 'Application'! Error: Requested object name [Application] is not unique.. Layer was not exported!

Failed to retrieve layer named 'Network'! Error: Requested object name [Network] is not unique.. Layer was not exported!

Failed to retrieve layer named 'Network'! Error: Requested object name [Network] is not unique.. Layer was not exported!

Failed to retrieve layer named 'Application'! Error: Requested object name [Application] is not unique.. Layer was not exported!

I assume this has to do with the Global policy assignment on domains, herewith a screenshot:

PS: The above screenshot is from the 'Network' policy where rules 1-8 and 9 are globally assigned and customers can exclusively edit the inline domain policy. I assume the export tool subsequently sees multiple 'Network' and 'Application' policy layers.

Really, really hoping the API can reference layers using unique IDs instead of names, to overcome this limitation, as it essentially doesn't work with Multi-Domain Server instances...

PS: Not sure why the export tool complains about 'Network' three times and 'Application' twice as the structure there is essentially the same:

PS: Snipped out rules 8.1 and 8.2 as they contain customer references.

Herewith the content of the 'import_export.log' file:

Checking existence of package [Standard]

Exporting Access Control layers

Exporting Access Layer [Network]

Getting layer information for layer [Network]

Failed to retrieve layer named 'Network'! Error: Requested object name [Network] is not unique.. Layer was not exported!

Exporting Access Layer [Application]

Getting layer information for layer [Application]

Failed to retrieve layer named 'Application'! Error: Requested object name [Application] is not unique.. Layer was not exported!

Exporting Access Layer [Network]

Getting layer information for layer [Network]

Failed to retrieve layer named 'Network'! Error: Requested object name [Network] is not unique.. Layer was not exported!

Exporting Access Layer [Network]

Getting layer information for layer [Network]

Failed to retrieve layer named 'Network'! Error: Requested object name [Network] is not unique.. Layer was not exported!

Exporting Access Layer [Application]

Getting layer information for layer [Application]

Failed to retrieve layer named 'Application'! Error: Requested object name [Application] is not unique.. Layer was not exported!

Exporting NAT policy

Getting information from show-nat-rulebase

Retrieved 4 out of 4 rules (100%)

##Show presented object of type CpmiAnyObject with name All

##Show presented object of type CpmiAnyObject with name Any

##Show presented object of type network with name CP_default_Office_Mode_addresses_pool

##Show presented object of type host with name External IP 2

##Show presented object of type group with name Internal

##Show presented object of type Global with name Original

##Show presented object of type Global with name Policy Targets

Analysing rulebase items...

##Show presented dependent rule of type nat-rule under section Exempt NAT

##Show presented section of type nat-section with name Exempt NAT

##Show presented dependent rule of type nat-rule under section Outbound NAT

##Show presented section of type nat-section with name Outbound NAT

Processing rules and sections

Updating data for rule #3

Updating data for rule #4

Exporting hosts

Exporting host with uid 882d76fe-87fe-4335-902e-2e9a994ba17b named External IP 2

Exporting groups

Exporting groups from group [Internal]

Exporting networks from group [Network - Client Name]

Exporting network with uid 2582f788-a426-421c-9aa6-6590e98c0d09 named LAN - Bloemfontein

Exporting network with uid b7a7cfab-8bee-4a8b-9d9c-ea776bd5d67a named LAN - Cape Town

Exporting network with uid af52ae76-5783-4f24-b035-da17bb9a72af named LAN - Polokwane

Exporting network with uid f80dd08c-ea84-4c19-9811-ac7e00114e38 named LAN - Durban

Exporting network with uid c13b2949-40f3-4bb2-a509-3baa1a09e330 named LAN - Johannesburg

Exporting network with uid f7c169bf-2140-4db7-b16f-16c411b940d5 named LAN - Nelspruit

Exporting network with uid a50a97cb-68fa-4b4d-ac3d-204a66d6a5b0 named LAN - Port Elizabeth

Exporting networks from group [Network - Guest WiFi]

Exporting network with uid b3b1a4e2-0f01-48c0-8d29-d10ff90e9c46 named LAN - Johannesburg - Guest WiFi

Exporting networks from group [Network - Teraco - Syrex]

Exporting network with uid e7248c6c-f2a7-404f-946f-1237ac1b038b named LAN - Teraco - Core uplink

Exporting network with uid 8b4171f6-894e-474e-9847-87b2f8f17472 named LAN - Teraco - Hosting

Exporting group with uid f8be31fc-2bcc-44ca-bac5-7ab25d1bf3d5 named Network - Client Name

Exporting group with uid da6a0028-6a76-42b8-86bb-02dc17958b0c named Network - Guest WiFi

Exporting group with uid 9531c411-5e31-4241-a44a-5cd1a9d6e3d4 named Network - Teraco - Syrex

Exporting group with uid 2dbc672f-a834-497a-b154-e9211f6d79b2 named Internal

Exporting networks

Exporting network with uid 22e75d37-812c-46f7-ad40-7eda6f193329 named CP_default_Office_Mode_addresses_pool

Exporting NAT rules

Exporting nat-rule with uid bcd6bdbe-30ca-47f0-8098-6898cd67cd98

Exporting nat-rule with uid 2359440f-d4dd-4adc-b1c2-f6eb38b80e98

Exporting placeholders for unexportable objects from NAT rulebase

Done exporting NAT rulebase.

Exporting general objects to TAR...

Regards

David Herselman

[David_Herselman]

PS: Perhaps consider renaming the script from 'import_export_package.py' to 'export_import_package.py' so that it matches the project name... Just my OCD...

[Robert_Decker]

Hi David,

Thank you very much for this detailed feedback, really appreciated!

NAT sections - we intentionally omitted exporting sections in order to avoid merging them during the import into another database. This is a real pain and error prone process. We prefer this process done manually.

Your assumptions regarding the name uniqueness errors are correct, they are due to global policy assignment. Using UID instead of name will fix this.

I'll take this important usability input and fix the tool to face with such situations.

Please stay tuned.

Regards,

Robert.

[Robert_Decker]

I would be glad to do it, but people are already using this name in their scripts and I do not want to ruin their automations...

[David_Herselman]

Hi Robert,

A relatively minor little quirk with the generated export file:

• When specifying '/root/CustomerA.tar.gz':

-rw-r--r-- 1 root root    0 Jan 26 14:04 CustomerA.tar.gz
-rw-r--r-- 1 root root 1944 Jan 26 14:04 CustomerA.tar.gz.tar.gz

• When specifying '/root/CustomerA':

-rw-r--r-- 1 root root    0 Jan 26 14:12 CustomerA
-rw-r--r-- 1 root root 1929 Jan 26 14:12 CustomerA.tar.gz

This occurs when either running the application interactively or specifying commands via switches:

cd /root/cptool;
source /root/python_2.7.14/bin/activate;
export HISTFILE='/dev/null';
python /root/ExportImportPolicyPackage/import_export_package.py -op export -n Standard --all -m 100.127.200.2 \
  -d 100.127.200.11 -o /root/cptool/CustomerA -u davidh -p secret;

PS: It would additionally be nice to hide the password when running the tool interactively and to have the tool run without having to choose '2' for run when supplying all parameters.

import getpass

user = raw_input("Username:")
passwd = getpass.getpass("Password for " + user + ":")

[Marcos_Garcia]

Hello, and if the vsx is in r80.10, is it possible to use this tool?

[PhoneBoy]

As far as I know, yes.

[Robert_Decker]

No. As I mentioned above, virtual systems are not supported yet.

Any object in R80 SmartConsole GUI that is edited inside a legacy editor window, is not native R80 object.

Our R&D works to fully migrate these objects into R80.

Robert.

[PhoneBoy]

I assume you could still pull the policies out, right?

The VSX objects are problematic, I get it (and why).

[Robert_Decker]

Yes, the process works ok.

BTW, each object that is not exportable, there is a detailed report in the log file about it.

Furthermore, we create an empty placeholder object called "export_error_xxx" to indicate this situation and help the user to quickly find that object in GUI.

[Robert_Decker]

Fix delivered to GitHub repo.

[Robert_Decker]

Fix delivered to GitHub repo.

The "2" option is still needed.

[Robert_Decker]

Hi all,

There is a new fix for this tool, following up valuable input from David Herselman:

Better support for MDS environment, by using policy layer uid instead of a name

Hiding a password typed by the user at the prompt

Bug fix related to redundant output file

Please go to the Github repo link (at the top of this post) and download the updated source (including the linked python sdk).

Robert.