cancel
Showing results for 
Search instead for 
Did you mean: 
Create a Post

Re: Python tool for exporting/importing a policy package or parts of it

So, in the case of migrate a cma containg vsx objects, I will have to create manually all the vsx objects and with the script migrate the policy and objects, am I right?

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

There are some types of objects that the script might not be able to export.

In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the "export_error_log.elg" log file to notify you of this.

After the export process ends, you import the data into another server, and open the SmartConsole GUI.

In the SmartConsole GUI you can easily locate each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

Robert.

0 Kudos
Employee
Employee

Re: Python tool for exporting/importing a policy package or parts of it

Im running R80.10 MDM with a few CMAs no JHF installed

From root I ran the following commands: 

==============================================

# ./opt/CPsuite-R80/fw1/Python/bin/python2.7 /home/admin/ExportImportPolicyPackage-master/import_export_package.py -m <CMA IP HERE>

Welcome to the Policy Package Import/Export Tool.
What would you like to do?
1. Import a package
2. Export a package
99. Exit
2
Please enter a Policy Package name to export:
MY_POLICY_NAME
Please select a login method:
1. Enter user credentials manually
2. Login as Root
3. Use an existing session file
4. Use an existing session UID
99. Back
2
The script will run with the following parameters:
Export Access-Control layers = True
Export Threat-Prevention layers = False
Output-file name = None
Management Server IP = 10.74.254.36
Management Server Port = 443
Management Server Domain = None
1. Change Settings
2. Run
99. Back
2
You currently do not have a record of this server's fingerprint.
Server's fingerprint: A5D408DBE6CCDD60296E615E858DEA1E96F3C0A3
Do you accept this fingerprint? [y/n] y
Fingerprint saved.
No package named 'MY_POLICY_NAME' found. Cannot export.

I tried mdsenv <CMA NAME> and running same command ... no luck. 

Confirmed MY_POLICY_NAME exists ... 

Tried several different policy names ... same issue.

Any thoughts?

Is there something I need to do special with Multi-Domain? 

PS: I tried an external JumpBox also tried user / pass ... same issues. 

0 Kudos
John_Tam
Ivory

Re: Python tool for exporting/importing a policy package or parts of it

You need run with this parameter import_export_package.py -m <MDS IP> -d <CMA IP>

Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Hi,

For MDM you should also specify the domain name you are acting on.

Either use a "-d" switch (-d "domain-name"), or select option 1 (Change Settings) and provide a domain name.

Robert.

Highlighted
Employee
Employee

Re: Python tool for exporting/importing a policy package or parts of it

Thanks for the quick response. The export worked however the import failed:

From Root within R80.10 MDM:

#  ./opt/CPsuite-R80/fw1/Python/bin/python2.7 /home/admin/ExportImportPolicyPackage-master/import_export_package.py -op import --all -m 10.0.0.1 -d MY_DOMAIN –r -n My_Policy -f /My_Policy.out.tar.gz

The import ran for a while then failed with the below message:

<---SNIP--->

Importing Access_Layer [IMPORTED LAYER My_Policy Security]

Nothing to import...

Importing Access_Layer [IMPORTED LAYER 1 My_Global_Policy Security]

Adding access-rules

Imported 1 out of 1 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER My_Policy Application]

Adding access-rules

Imported 2 out of 2 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER My_Policy Security]

Traceback (most recent call last):
File "/home/admin/ExportImportPolicyPackage-master/import_export_package.py", line 47, in <module>
import_package(client, args)
File "/home/admin/ExportImportPolicyPackage-master/importing/import_package.py", line 52, in import_package
layers_to_attach = import_objects(args.file, client, {})
File "/home/admin/ExportImportPolicyPackage-master/importing/import_objects.py", line 86, in import_objects
import_objects(rulebase_object_file.name, client, changed_layer_names, layer_name)
File "/home/admin/ExportImportPolicyPackage-master/importing/import_objects.py", line 19, in import_objects
export_tar = tarfile.open(file_name, "r:gz")
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/tarfile.py", line 1693, in open
return func(name, filemode, fileobj, **kwargs)
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/tarfile.py", line 1740, in gzopen
fileobj = gzip.GzipFile(name, mode, compresslevel, fileobj)
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/gzip.py", line 94, in __init__
fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: 'exported__access_layer__My_Policy Security__2018_03_03_15_50.tar.gz'

Any thoughts? 

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Hi John,

I've noticed in your logs that an access policy layer named "My_Policy Security" appears twice.

Is it possible that you have defined that layer both in global domain and in CMA?

It seems that you have assigned a global policy to a CMA before exporting the policy, right?

Robert.

0 Kudos
Employee
Employee

Re: Python tool for exporting/importing a policy package or parts of it

Hi Robert,

Thanks for your quick response. 

No, I do not have a global policy assigned to any CMA. 

I only have 2 layers in my Access Control policy: 

Security  (Firewall Blade Only)
Application (Application Control Blade Only)

This is the command I ran to export: 

# ./opt/CPsuite-R80/fw1/Python/bin/python2.7 /home/admin/ExportImportPolicyPackage-master/import_export_package.py -op export --all -m 10.0.0.1 -d LAB_DOMAIN -n LAB_Policy -r -o LAB_Policy.out

I'm trying to import this into a another Domain and it failed.

What I provided you previously is a scrub of the original log where I removed all customer related object information and IPs.

I deleted the policy in the target Domain and tried again and got the same results... ALL objects are imported just fine but no rule. 

===================================================================================================

Importing Access_Layer [IMPORTED LAYER 1 LAB_Policy Security]

Nothing to import...

Importing Access_Layer [IMPORTED LAYER 2 LAB_Global_Policy Security]

Adding access-rules

Imported 1 out of 1 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER 1 LAB_Policy Application]

Adding access-rules

Imported 2 out of 2 access-rules (100%)

Importing Access_Layer [IMPORTED LAYER 1 LAB_Policy Security]

Traceback (most recent call last):
File "/home/admin/ExportImportPolicyPackage-master/import_export_package.py", line 47, in <module>
import_package(client, args)
File "/home/admin/ExportImportPolicyPackage-master/importing/import_package.py", line 52, in import_package
layers_to_attach = import_objects(args.file, client, {})
File "/home/admin/ExportImportPolicyPackage-master/importing/import_objects.py", line 86, in import_objects
import_objects(rulebase_object_file.name, client, changed_layer_names, layer_name)
File "/home/admin/ExportImportPolicyPackage-master/importing/import_objects.py", line 19, in import_objects
export_tar = tarfile.open(file_name, "r:gz")
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/tarfile.py", line 1693, in open
return func(name, filemode, fileobj, **kwargs)
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/tarfile.py", line 1740, in gzopen
fileobj = gzip.GzipFile(name, mode, compresslevel, fileobj)
File "/opt/CPsuite-R80/fw1/Python/lib/python2.7/gzip.py", line 94, in __init__
fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: 'exported__access_layer__LAB_Policy Security__2018_03_03_15_50.tar.gz'

===================================================================================================

I'm going to try exporting but this time I'll remove --all 

Any advice is appreciated. 

0 Kudos
Employee
Employee

Re: Python tool for exporting/importing a policy package or parts of it

Tried exporting again without --all and imported both directly on the MDS and via a jump box with python 2.7.9 installed. Both came back with the exact same error ... from the error it says: 

IOError: [Errno 2] No such file or directory: 'exported__access_layer__LAB_Policy Security__2018_03_04_18_02.tar.gz' 

I searched for this file and I could not find it anywhere ... 

I extracted the Exported File I created and I found it in here ... there seems to be a problem with gzip.py unable to find this package within the Exported File. 

Any thoughts? 

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Hi John,

From the logs I see that you have some data corruption there.

Please contact me via email when you are available.

Robert.

0 Kudos
Employee
Employee

Re: Python tool for exporting/importing a policy package or parts of it

Hi Robert,

Thank you for your time today. You are right there was something wrong with the exported__access_layer__LAB_Policy Security__2018_03_03_15_50.tar.gz file ... apparently when opening the My_Policy.out.tar.gz file with WinRAR there was two files with the name "exported__access_layer__LAB_Policy Security__2018_03_03_15_50.tar.gz" ... what was happening during the import process it would extract the 1st file (aka GOOD File) then it would extract the 2nd file (aka BAD file) and overwrite the GOOD file. To resolve the issue I used WinRAR to extract the good file 1st, uploaded the My_Policy.out.tar.gz file to my MDS, extracted it, replaced the BAD file with the GOOD file, re-compressed the file into a tar.gz file and VOILA! The import was successful. 

SO ... I checked the CMA and unfortunately although the import was successful the in-line layer is not expanding Smiley Sad

Not sure where to go from here ... 

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

I'll investigate your data to find out what is wrong and inform you.

If there is a bug in tool's code, it will be fixed.

Robert.

0 Kudos
Employee
Employee

Re: Python tool for exporting/importing a policy package or parts of it

Good morning Robert, do you have any more information related to this issue from John?  We need to update the customer tomorrow and we will miss our window to get this completed.

David Walker

Team Manager, DMD Services

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Hi David,

We started to investigate the issue on Monday, but I had a problem to use his management server afterwards due to credentials problem.

I sent him email, waiting for a response...

Robert.

0 Kudos

Re: Python tool for exporting/importing a policy package or parts of it

Firstly many thanks for the valuable time and effort you and your colleagues continue to invest, it's much appreciated!

I'm successfully able to run the tool and it generates an export archive. Upon extracting it and exploring the content it initially looked good but the actual access layer archives unfortunately contain no data. There don't appear to be any error messages, apart from 4 unexportable objects. Herewith a link to the content of the import_export.log file, I replaced references to the customer name with 'Client Name' and the first 3 octets of public IPs with 55.66.77:

http://txt.do/dxzcl 

The archive does contain archives within it but only the 'nat_rule' and 'threat-layer__Standard' contain csv and json data files:

-rw-r--r-- 1 root root 103 Mar 8 22:27 exported__access_layer__Application__2018_03_08_22_22.tar.gz
-rw-r--r-- 1 root root  99 Mar 8 22:26 exported__access_layer__Network__2018_03_08_22_22.tar.gz
-rw-r--r-- 1 root root 761 Mar 8 22:27 exported__nat_layer__Standard__2018_03_08_22_22.tar.gz
-rw-r--r-- 1 root root  95 Mar 8 22:27 exported__threat_layer__IPS__2018_03_08_22_22.tar.gz
-rw-r--r-- 1 root root 525 Mar 8 22:28 exported__threat_layer__Standard Threat Prevention__2018_03_08_22_22.tar.gz

More questions:

  • Is this an official Check Point project?
  • I can share the generated export archive, should it be beneficial. Perhaps you could provide a method of me sharing the file or a link to the file outside the public domain.
  • Could I possibly ask for a switch to exclude exporting MDS global objects and policy rules? We can probably work around this by repackaging the resulting archive though. I'm a little strapped for time at the moment but intend to create an additional test domain to play with importing the exported policies, once we have one...

Regards

David Herselman

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Hi David,

Thank you for your feedback.

This new version of tool contains fixes for issues raised mainly by you:

1. Using layer UIDs instead of names for compatibility with global policy assigned domains

2. Hiding a password typed by the user at the prompt

3. Removal of redundant output file

Are these items working now? Does a fix for the first item raised the current problem of empty archives for access layers?

In addition, there were some important stabilization fixes.

Now, for your questions:

1. This is NOT an official Check Point project. This is an open source initiative for the community, but is actively maintained by me.

2. Yes, I'd like to examine the generated archive. Maybe Dameon Welch Abernathy can assist here.

3. Any ideas or improvements are welcome and will be handled as soon as I can. Please remember that this is an open source and anyone can contribute!

Recently, I'm investing more time for fixes and developments related to MDM. It seems that these parts are not supported as needed.

regards,

Robert.

0 Kudos
Admin
Admin

Re: Python tool for exporting/importing a policy package or parts of it

I'll set up an account on an FTP server for you David Herselman

I will send you a private message with the credentials.

0 Kudos

Re: Python tool for exporting/importing a policy package or parts of it

Hi Robert,

Apologies, yes the new version fixed my initial problems and I can now safely show other staff how to use the tool without revealing my credentials. The export now completes without errors and doesn't generate the redundant output file but it unfortunately contains everything except the actual policy rules (objects, categories, groups, nat rules, etc).

PS: I received an email from Dameon Welch Abernathy and will upload the generated archive and log files shortly...

Regards

David Herselman

0 Kudos

Re: Python tool for exporting/importing a policy package or parts of it

Hi Rob , that tool can be used for migrate cma with a vs too?

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Hi Marco,

The main purpose of this tool is to export a policy package (access, threat) and all its objects from a management server database (smart center or CMA/domain), to be imported later into another management server database, or just for backup.

There are some types of objects that the script might not be able to export. In such a case, an appropriate dummy object will be exported instead, and a message will be logged into the log files to notify you of this. In the Check Point SmartConsole you can easily replace each of these objects by searching "export_error" in the search field, see where each object is used, create the necessary object manually, then replace it.

In this link you can see a list of all exportable objects types.

Unfortunatelly, virtual systems are not supported as they are not fully supported by R80/R80.10 yet.

Robert.

0 Kudos

Re: Python tool for exporting/importing a policy package or parts of it

Thanks for the detailed reply , let say that vs object are not exported they will be replaced by "export_error" object? my hope is to have at least the object from the cma where a vs where deployed

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

right, the placeholder should be there.

Robert.

Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

IMPORTANT NOTE!!!

In the case you are exporting a policy package from a CMA, please verify that a global policy was NOT assigned to that CMA. The tool doesn't support exporting a policy with global policy assigned!

Robert.

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Hi David,

I recieved your data, thanks to Dameon, and analyzed it.

As I've suspected, you were migrating from a CMA with global policy assigned.

I've to admit that recently I recieved many requests from inside and outside Check Point, facing the same issue, and posted here (bellow and on top in red) an important caveat about what this tool is supposed to support.

There is no way to export a CMA that has assigned a global policy.

One way to workaround this, is a manual work to un-assign the global policy for that CMA and then export.

Hope this helps,

Robert.

P.S. if you need any additional clarifications or assistance, please just ask for.

0 Kudos
Admin
Admin

Re: Python tool for exporting/importing a policy package or parts of it

Can you export the global policy itself, though?

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Yes, of course.

0 Kudos
Admin
Admin

Re: Python tool for exporting/importing a policy package or parts of it

It's just you can't export the two when they are linked together.

Got it, thanks for confirming.

0 Kudos

Re: Python tool for exporting/importing a policy package or parts of it

Hi Robert Decker I just downloaded the latest version of this script and ran it to try to export my policy package. However it failed during the export as shown below:

Traceback (most recent call last):
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\import_export_package.py", line 45, in <module>
export_package(client, args)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_package.py", line 39, in export_package
= export_access_rulebase(show_package.data["name"], access_layer["name"], access_layer["uid"], client, timestamp, tar_file)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_access_rulebase.py", line 44, in export_access_rulebase
export_access_rulebase(package, access_layer["name"], access_layer["uid"], client, timestamp, tar_file)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_access_rulebase.py", line 59, in export_access_rulebase
export_general_objects(data_dict, api_type, object_dictionary[api_type], unexportable_objects, client)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_objects.py", line 313, in export_general_objects
api_type, group_object, client, unexportable_objects)
File "C:\Users\nassi\Documents\Vendors\Check Point\Scripts\ExportImportPolicyPackage-master\exporting\export_objects.py", line 400, in get_group_objects
member_objects.extend(group_object[container])
TypeError: 'NoneType' object has no attribute '__getitem__'

Any thought?

Thanks,

Nader

0 Kudos
Employee++
Employee++

Re: Python tool for exporting/importing a policy package or parts of it

Hi Nader,

It looks like a bug in the code due to some unique type/value of your data.

Please run again the tool and send to my email the API logs file "api.elg" from "$MDS_FWDIR/logs" folder on management server, and tool's log file from your working directory.

I'll look at it on Sunday, at work.

Robert.

0 Kudos

Re: Python tool for exporting/importing a policy package or parts of it

Hi Robert,

I’ve sent you the requested log files to your email address.

Thanks,

Nader

0 Kudos