How to merge database in R80.10

[Vishnu_Kumar]

Hi All,

Currently we are using 5 managment server having os R77.30.

Each management server is managing 1 HA pair of secuity gateways.

Now I am palnning to merge the database of all Managment server, so that we can manat all gateways from single managment server.

Can anybody help me to know how we can merge the database of these managment servers.

As CP_merge is has been obsolute in R80.10.

[PhoneBoy · Accepted solution]

https://community.checkpoint.com/t5/API-CLI-Discussion-and-Samples/Python-tool-for-exporting-importi...

[G_W_Albrecht]

I would suggest to first unite the R77.30 management servers using CP_merge, then run the pre-upgrade verifier on the R77.30 SMS. After performing the needed changes, you can do a migrate export with R80.20 tools (i would suggest R80.20) and a migrate import into a fresh R80.20 SMS.

[Vishnu_Kumar]

Hi Dameon and Günther,

Thanks for an update. 

Firstly Apologies for a typo mistake in my initial query. Actually All management Server's current OS is R80.10.

Earlier by mistake I mentioned R77.30.

So I think, I will not be able to follow the procedure suggested by Mr. Gunther.

So now available option is :

Python tool for exporting/importing a policy package or parts of it 

During follow this option I have done following steps:

1.  have downloaded the ZIP file from the link "https://github.com/CheckPointSW/ShowPolicyPackage"

2. Now "cp_mgmt_api_python_sdk" is blank by default.

     As per instructions I will need to manually download and copy the [Check Point API Python SDK]  (https://github.com/CheckPoint-APIs-Team/cpapi-python-sdk) content into this folder.

PROBLEM:

1. From this link, I am not able to locate the "Check Point API Python SDK" file. So that I can manually download it.

2. "import_export_package.py " script is not executing.

Can any one share exact steps or relevant artical which i can follow.

[PhoneBoy]

The Python SDK consists of the files in the lib directory on that github repository.

They would need to be copied somewhere your Python interpreter will look for them. 

That will likely fix the issue you are having executing the import_export_package.py script.

[Vishnu_Kumar]

Hi Dameon,

Thanks for update. Now donloaded all relavant files and import_export_package.py script is exeting from my PC.

But now once I fill all required details into interactive session to export the policy package. In last script gets termanated automatically with following error message:

============== ===========++++++++++++++++=======================

Login to management server failed. lib::APIResponse
{
    "data": null,
    "error_message": "APIResponse received a response which is not a valid JSON.",
    "res_obj": {},
    "status_code": 403,
    "success": false
}

==============XXXXXXXXXXXXXXXXXXXXXXXX====================

Please also veryfybelow mentioned  the procedure, which I followed during Intactive session (Script execution)

I hope i need to run this scrip on PC machine, not on managmnet Server.

==================== ============================================

Welcome to the Policy Package Import/Export Tool.
What would you like to do?
1. Import a package
2. Export a package
99. Exit
2
Please enter a Policy Package name to export:
Policy_Package_1
Please select a login method:
1. Enter user credentials manually
2. Login as Root
3. Use an existing session file
4. Use an existing session UID
99. Back
1
The script will run with the following parameters:
Export Access-Control layers = True
Export Threat-Prevention layers = False
Output-file name = None
Management Server IP = 127.0.0.1
Management Server Port = 443
Management Server Domain = None
1. Change Settings
2. Run
99. Back
1
Please select a setting to change:
1. Disable export of Access-Control Rulebases
2. Enable export of Threat-Prevention Rulebases
3. Output file name
4. Change Management Server IP
5. Change Management Server Port
6. Change the domain name
99. Back
4
Please enter the IP address of the management server:
172.16.1.221
The script will run with the following parameters:
Export Access-Control layers = True
Export Threat-Prevention layers = False
Output-file name = None
Management Server IP = 172.16.1.221
Management Server Port = 443
Management Server Domain = None
1. Change Settings
2. Run
99. Back
2
Please enter your username:
admin

Please enter your password:

((((((  AFTER IT SCRIPT GETS CLOSED AUTOMATICALLY)

=======================XXXXXXXXXXXXXXXXXX ======================== 

Below is the API Status (even after restart API service, IP is 127.0.0.1)

[As some online document reflecting that it should not be 127.0.0.1 to resolve this issue]

------------- ----------------------------------

[Expert@MGMT-1:0]# api stat
Usage: api {start [-d]|stop|restart [-d]|status|reconf|logging [on|off|warn|info                                                                                        |debug|trace]|fingerprint}
[Expert@MGMT-1:0]# api status

API Settings:
---------------------
Accessibility:                      Require ip 127.0.0.1
Automatic Start:                    Enabled

Processes:

Name      State     PID       More Information
-------------------------------------------------
API       Started   3611
CPM       Started   3369      Check Point Security Management Server is running                                                                                         and ready
FWM       Started   2922

Port Details:
-------------------
JETTY Internal Port:      50276
APACHE Gaia Port:         443
                          Apache port retrieved from: httpd-ssl.conf

--------------------------------------------
Overall API Status: Started
--------------------------------------------

API readiness test SUCCESSFUL. The server is up and ready to receive connections

Notes:
------------
To collect troubleshooting data, please run 'api status -s <comment>'

[PhoneBoy]

There was a bug in early versions of R80.10 where even if you set the allowed IPs for the API to something other than 127.0.0.1, it would remain that way.

Re: R80.10 API bug: fallback to "SmartCenter Only" after reboot‌

Please install the latest Jumbo Hotfix and try again

[Vishnu_Kumar]

Hi Dameon,

Thanks for your last input. Now that issue is resolved, but now facing below error in last

============XXXXX######################

Exporting Access Control layers

Exporting Access Layer [Policy_Package_1 Network]

Traceback (most recent call last):
File "D:\GIT_HUB\SCRIPT\import_export_package.py", line 45, in <module>
export_package(client, args)
File "D:\GIT_HUB\SCRIPT\exporting\export_package.py", line 39, in export_package
= export_access_rulebase(show_package.data["name"], access_layer["name"], access_layer["uid"], client, timestamp, tar_file)
File "D:\GIT_HUB\SCRIPT\exporting\export_access_rulebase.py", line 16, in export_access_rulebase
get_query_rulebase_data(client, "access-rulebase", {"name": layer, "uid": layer_uid, "package": package})
File "D:\GIT_HUB\SCRIPT\exporting\export_objects.py", line 38, in get_query_rulebase_data
if compare_versions(client.api_version, "1.1") != -1:
File "D:\GIT_HUB\SCRIPT\utils.py", line 126, in compare_versions
v1_nums = version1.split('.')
AttributeError: 'NoneType' object has no attribute 'split'

#################XXXXXXXXXXXXXXXXXXXXXXX#####################

Also request you to please verify my below given procedure as well, which I am following to export a policy

======== ==============

D:\GIT_HUB\SCRIPT>
D:\GIT_HUB\SCRIPT>import_export_package.py

Welcome to the Policy Package Import/Export Tool.
What would you like to do?
1. Import a package
2. Export a package
99. Exit
2
Please enter a Policy Package name to export:
Policy_Package_1
Please select a login method:
1. Enter user credentials manually
2. Login as Root
3. Use an existing session file
4. Use an existing session UID
99. Back
1
The script will run with the following parameters:
Export Access-Control layers = True
Export Threat-Prevention layers = False
Output-file name = None
Management Server IP = 127.0.0.1
Management Server Port = 443
Management Server Domain = None
1. Change Settings
2. Run
99. Back
1
Please select a setting to change:
1. Disable export of Access-Control Rulebases
2. Enable export of Threat-Prevention Rulebases
3. Output file name
4. Change Management Server IP
5. Change Management Server Port
6. Change the domain name
99. Back
4
Please enter the IP address of the management server:
172.16.1.221
The script will run with the following parameters:
Export Access-Control layers = True
Export Threat-Prevention layers = False
Output-file name = None
Management Server IP = 172.16.1.221
Management Server Port = 443
Management Server Domain = None
1. Change Settings
2. Run
99. Back
2
Please enter your username:
admin

Please enter your password:

Exporting Access Control layers

Exporting Access Layer [Policy_Package_1 Network]

Traceback (most recent call last):
File "D:\GIT_HUB\SCRIPT\import_export_package.py", line 45, in <module>
export_package(client, args)
File "D:\GIT_HUB\SCRIPT\exporting\export_package.py", line 39, in export_package
= export_access_rulebase(show_package.data["name"], access_layer["name"], access_layer["uid"], client, timestamp, tar_file)
File "D:\GIT_HUB\SCRIPT\exporting\export_access_rulebase.py", line 16, in export_access_rulebase
get_query_rulebase_data(client, "access-rulebase", {"name": layer, "uid": layer_uid, "package": package})
File "D:\GIT_HUB\SCRIPT\exporting\export_objects.py", line 38, in get_query_rulebase_data
if compare_versions(client.api_version, "1.1") != -1:
File "D:\GIT_HUB\SCRIPT\utils.py", line 126, in compare_versions
v1_nums = version1.split('.')
AttributeError: 'NoneType' object has no attribute 'split'

D:\GIT_HUB\SCRIPT>

[Vishnu_Kumar]

With this error, a  <Policy-Package-Name>.tar.tgz file is exported but having only 1 KB size.

[PhoneBoy]

What version of Python?

Note I’ve only run this on Linux myself.

Believe you also need to install this: Python library for using R80 management server APIs‌

[Vishnu_Kumar]

Hi Dameon,

Currently, python version is 2.7.9 on my Windows 10 machine. May I know which Linux OS you used for it.

Please let me know whether I need to install python Library “ Python library for using R80 management server APIs <https://community.checkpoint.com/docs/DOC-1091> ”on Windows machine or on checkpoint Management Server.

Currently I am trying to install it on Management server. Here “pip install command is not working. Is it possible to install it locally after upload the library using WinSCP.

[PhoneBoy]

I used an Ubuntu variant but any modern distribution should work.

The libraries must be on the machine running the Python interpreter.

Have no idea if you can copy the libraries over to a Windows version of Python.

[Dawei_Ye]

Hi Dameon,

How could this script deal with the same name object when I import the old exported rules into a new SMC?

We are facing a huge problem when we merge a old SSG (with 1400 objects and 1200 policies)to Check Point.

Regards,

Dawei Ye

[PhoneBoy]

The API won't (or shouldn't) allow you to create objects of the same name.

Which means, as is, the script will probably fail on import with duplicate objects in your existing management.

Then again, because the script is Open Source, you could probably enhance it to deal with this situation.

[Vishnu_Kumar]

Hi Dameon,

Thanks very much, Now i am able to exucure script scussfully on ubuntu.

I was able to export policy scusfully from source managmnet server.

Also able to import it on target Managmnet server. All the thinkg liooking fine except VPN cummunities.

VPN communities are not imorting, getting below error message:

Failed to import vpn-community-meshed with name [HO-to-Brach-VPN]. Error: message: Invalid parameter for [shared-secrets]. Invalid value

code: generic_err_invalid_parameter.

[PhoneBoy]

The parameter should be called shared-secret (without the s on the end).

Which means it's possible there is a bug in the script.

[Vishnu_Kumar]

Hi Dameon,

Actually i wanted to know using this scripts is it possiblle to import VPN cummuniteis or not.

[PhoneBoy]

Like I said to your previous comment, it looks like there is a bug in the script related to shared secrets.

Amiad Stern‌ can you have someone look at Python tool for exporting/importing a policy package or parts of it‌ and confirm?

[Amiad_Stern]

Hi Vishnu Kumar‌ , we will check Dameon Welch-Abernathy‌ comment to see if there is a bug. Meanwhile, if you're interested in only in exporting VPN communities (and not the whole package), try this repository GitHub - CheckPointSW/ExportObjects: Check Point ExportObjects tool enables you to export specific t...  

[Brandon_Chan_Th]

Hi Amiad, I am facing the same issue as Vishnu where the VPN communities are not being merged due to the error, 

Error: message: Invalid parameter for [shared-secrets]. Invalid value

code: generic_err_invalid_parameter.

Therefore, I have tried your suggestion to export out the VPN communities separately as per https://github.com/CheckPointSW/ExportObjects

However, when I have tried importing the VPN communities I am getting the error,

Line 25: code: "generic_err_object_not_found"
message: "Requested object [IPX--peer-IP] not found"

The object IPX--peer-IP is an Interoperable Devices which for some reason is not being added in after executing the import_export_package.py. Is there a way for me to export and import only Interoperable Devices? Unfortunately, I do not see the export script for Interoperable Devices in https://github.com/CheckPointSW/ExportObjects.

[Vishnu_Kumar]

Hi Amiad,

Actually I need VPN communities as well as rest of the database also.

Now just for an update, If I am removing pre-shared-key from the cummunities then able to import VPN communities sucessfully with rest of VPN parameters.

But if i dont remove the pre-shared-key then not able to import VPN communities completely. 

[Vishnu_Kumar]

Hi Dameon,

From MGMT-Server-A, i need to export two policy packages and then import these on MGMT-Server-B.

I successfully imported the "Policy-Package-1"  from MGMT-server-A to MGMT-Server-B, including all objects.

Now trying to import "Policy-Package-2" from server-A to server-B, but as objects database has been already imported during importing "Policy-Package-1".

So it is throwing multiple errors of colliding objects when same object database is being importing again with "Policy-Package-2".

So please suggest, if this time i can only import Policy-Package without database objects or suggest other best way to resolve  this issue.

[PhoneBoy]

At a high level, you could probably make it work by changing a call in the script to use set-if-exists true, which would tell the API server it's ok to overwrite an existing object.

However, there are several methods being referred to in this thread and I'm not sure exactly which one you're using right now.

[Vishnu_Kumar]

Hi Dameon,

After merging the database, how will be the behave of NAT rulres.

In my LAB, after consolidated the database of 4 management  servers, sequence of NAT rules has been changed  drastically into each policy package. 

Even policy target (Gateway) is separate for each policy.  

Some rules are commonly added to each policy package.

[Vishnu_Kumar]

Hi Dameon,

Through this script I am not able to export Sub-Policies.

Please suggest if there is any way to import sub policies

Getting below error message:

===== =================

Failed to import access-rule. Error: message: Invalid parameter for [type]. The invalid value [29e53e3d-23bf-48fe-b6b1-d59bd88036f9] should be replaced by one of the following values: [none, log, extended log, detailed log]

code: generic_err_invalid_parameter

.