topic Re: Python tool for exporting specific types of objects from the management server in API / CLI Discussion

Python tool for exporting specific types of objects from the management server

Inbar_Moskovich — Thu, 13 Oct 2016 12:01:43 GMT

Overview

This tool enables you to export specific types of objects from a R80.10 Management database to a .csv file, which can then be imported into any other R80.10 Management database.

This tool can be used for backups, database transfers, testing, and more.

A typical run of the script to export all hosts will be along the lines of:

export_hosts.py -m [management server IP] -o hosts.csv

A lot more details can of course be accessed with the '-h' option.

Later on, to import these hosts into other R80.10 Management database, you execute the command mgmt_cli with the batch flag [-b]:

mgmt_cli add host -b hosts.csv

Tested on version

R80.10

API version 1.1

Source Code Availability

The source code is now public on GitHub repository:

ExportObjects Tool

NOTICE: By using this sample code you agree to terms and conditions in this Terms and Conditions

...

Re: Python tool for exporting specific types of objects from the management server

Giuseppe_Devito — Fri, 28 Jul 2017 12:23:45 GMT

Hi, I got this error:

$ python export_hosts.py -m CP-MGMT -o hosts.csv --port 18190
Trying to get login credentials...
Trying to login via `mgmt_cli login -r true`.
Trying to get username and password.
Enter username: admdevitog
Enter password:
Got the following login credentials:
Username: ************
Password: ************
Session ID: None
Traceback (most recent call last):
File "export_hosts.py", line 15, in <module>
get_objects("hosts", fields)
File "/cygdrive/c/Users/************/Desktop/Fdown/import_export_objects/get_objects.py", line 129, in get_objects
output_file, output_file_format, user_created, client, args = process_args_and_login(showparameter=showparameter, fields=fields)
File "/cygdrive/c/Users/************/Desktop/Fdown/import_export_objects/get_objects.py", line 68, in process_args_and_login
raise mgmt_api_lib.APIClientException("The server's fingerprint is different than your local record of it. The script cannot operate in this unsecure manner (unless running with --unsafe). Exiting...")
mgmt_api_lib.api_exceptions.APIClientException: The server's fingerprint is different than your local record of it. The script cannot operate in this unsecure manner (unless running with --unsafe). Exiting...

Can you help me?

Re: Python tool for exporting specific types of objects from the management server

Ofir_Shikolski — Sun, 30 Jul 2017 06:06:38 GMT

Please try this :
1.Delete the following files:
get_objects.pyc
get_objects.log
fingerprints.txt
hosts.csv

2. Run

# python export_hosts.py -m <IP> -o hosts.csv

For working scenario , you should see :

You currently do not have a record of this server's fingerprint.
Server's fingerprint: <XXXXXXXXX>
Do you accept this fingerprint? [y/n] y
Fingerprint saved.
1 object received.
Done.

Re: Python tool for exporting specific types of objects from the management server

jongsoo — Mon, 23 Oct 2017 03:23:29 GMT

Hello,

I have a problem. :

C:\import_export_objects>export_hosts.py -m 192.168.20.90 -o hosts.csv
Trying to get login credentials...
Trying to login via `mgmt_cli login -r true`.
Trying to get username and password.
Enter username: admin
Enter password:
Got the following login credentials:
    Username: admin
    Password: *********
    Session ID: None
50/33159 objects retrieved.
100/33159 objects retrieved.
150/33159 objects retrieved.
200/33159 objects retrieved.
                     ...
4750/33159 objects retrieved.
4800/33159 objects retrieved.
4850/33159 objects retrieved.
Traceback (most recent call last):
File "C:\import_export_objects\export_hosts.py", line 15, in <module>
    get_objects("hosts", fields)
File "C:\import_export_objects\get_objects.py", line 130, in get_objects
    data = get_data(client, showparameter=showparameter if showparameter else args.showparameter[0], dependency_solver_args=dependency_solver_args)["objects"]
File "C:\import_export_objects\get_objects.py", line 104, in get_data
    data = get_raw_data(param=showparameter, client=client)
File "C:\import_export_objects\get_objects.py", line 157, in get_raw_data
    for current_res in results:
File "C:\import_export_objects\mgmt_api_lib\cp_mgmt_api.py", line 232, in gen_api_query
    raise APIException(api_res.error_message)
mgmt_api_lib.api_exceptions.APIException: APIResponse received a response which is not valid JSON.

C:\import_export_objects>

When 4850 obejcts were retrieved, API Daemon died.
I think this problem happens because of too many objects.
Do you have any solution to get all objects?

Please help me

Additionally, I applied this commands. :

export NGM_WEB_API_JRE_64=1

export NGM_WEB_API_MAX_MEMORY=2048

Re: Python tool for exporting specific types of objects from the management server

Russell_Seifert — Tue, 14 Nov 2017 13:25:31 GMT

Exporting "group", "group-with-exclusion", "service-group", "time-group" fails.

$ python export_groups.py -m 172.25.197.232 -o output/groups.csv
Trying to get login credentials...
Trying to login via `mgmt_cli login -r true`.
Trying to get username and password.
Enter username: admin
Enter password:
Got the following login credentials:
    Username: admin
    Password: ******
    Session ID: None
28/28 objects retrieved.
Traceback (most recent call last):
File "export_groups.py", line 14, in <module>
    get_objects("groups", fields, dependency_solver_args)
File "/cygdrive/c/Users/rseifert/Desktop/import_export_objects/get_objects.py", line 130, in get_objects
    data = get_data(client, showparameter=showparameter if showparameter else args.showparameter[0], dependency_solver_args=dependency_solver_args)["objects"]
File "/cygdrive/c/Users/rseifert/Desktop/import_export_objects/get_objects.py", line 120, in get_data
    data = dependency_loader.order_data()
File "/cygdrive/c/Users/rseifert/Desktop/import_export_objects/get_objects.py", line 738, in order_data
    if item["type"] == self.type:
TypeError: 'NoneType' object has no attribute '__getitem__'

Also, export_times.py has "get_objects("networks", fields)". Changing it to "times" fails to save the results.

Re: Python tool for exporting specific types of objects from the management server

Robert_Decker — Thu, 16 Nov 2017 09:55:23 GMT

Hello all,

This library code is obsolete and is not maintained any more.

Instead, you should use a newer library package - ExportImportPolicyPackage, hosted on GitHub, and discussed here - ExportImport on Developers Network.

Re: Python tool for exporting specific types of objects from the management server

Robert_Decker — Mon, 20 Nov 2017 12:40:46 GMT

Hi all,

We have decided to continue maintaining this useful tool, fixed some bugs and uploaded the source code to GitHub repo (the link on top of the page).

Enjoy!

Re: Python tool for exporting specific types of objects from the management server

Steve_Moran1 — Wed, 13 Feb 2019 18:29:04 GMT

after accepting the fingerprint, I recieve the following error...

Do you accept this fingerprint? [y/n] y
Fingerprint saved.
Login failed: APIResponse received a response which is not a valid JSON.

Re: Python tool for exporting specific types of objects from the management server

Eric_Oakeson — Wed, 13 Feb 2019 20:56:21 GMT

Does this script pull only custom objects? It says it's retrieved 103 objects, but then when we open the csv file there will only be a handful in there. Does it not pull anything that's there by default?