CLI API Example for exporting group members to a CSV file for later import

Eric_Beasley — Thu, 30 Jun 2016 20:30:13 GMT

Overview

This set of example shell scripts for bash on Gaia, demonstrate how to export a CSV file containing all members of all groups (within the define limits value) with the show groups and show group command, and the requisite import operation using the generated CSV file to populate the existing groups with their respective members with the set group command utilizing the --batch <csv_input_file> optional switch.

Description

Bash shell script "cli_api_dump_objects_group-members_build_import_csv_v00x01.sh" exports the members of the identified groups, first generating an array of groups based on the limit constraint value, and then using that array to dump the individual members of each group collected in the array to a CSV file with the first column being the group name and the second column being the group member. The group members are selected from the output of "show groups" using the jq command, parsing the generated json output. This script also generates an output file for process reference, in addition to the CSV file. and the console output.

Output files are generated as:

CSV File :

Name: group-members-add_<DateTimeGroup>.csv

Path: dump/<DateTimeGroup>/

Example: dump/2016-06-30-1044CDT/group-members-add_2016-06-30-1044CDT.csv

Operations Output File:

Name: group-members_build_import_csv__<DateTimeGroup>.txt

Path: dump/<DateTimeGroup>/

Example: dump/2016-06-30-1044CDT/group-members_build_import_csv_2016-06-30-1044CDT.txt

Example output CSV file group-members-add_2016-06-30-1044CDT.csv :

name,members.add

CP-Management,"R80-Demo-SMS-01"

CP-Management,"R80-Demo-SmartEvent"

CP-Gateways,"Perimeter-Jupiter-R77x30"

Management-Workstations,"Joe-Lenovo-W500"

Management-Workstations,"Hans-Dell-M4800"

Management-Workstations,"Madoka-Asus-X900"

Management-Workstations,"Terry-Apple-iBook"

Bash shell script "cli_api_import_objects_csv_group-members_v00x01.sh" imports a CSV file via the "set group" operation, using the columns of the CSV file to populate the group. This script stipulates output from the shell script "cli_api_dump_objects_group-members_build_import_csv_v00x01.sh" or a similar CSV file that has the group name to set as the first column. For this operation the expected action is that each member added to a group is the second value on a new row of the CSV file with the target group's name as the first column.

It is stipulated that the groups and member objects to populate the groups were created in a previous process before this Example

The import operation will generate an output file with the json details of the "set" operation, which can be reviewed for errors during the import operation, since missing member objects or groups will not be handled.

Output files are generated as:

Operations Output File:

Name: dump_add_group-members_via_set_<DateTimeGroup>.json

Path: dump/<DateTimeGroup>/

Example: dump/2016-06-30-1044CDT/dump_add_group-members_via_set_2016-06-30-1217CDT.json

Instructions

To utilize the Bash shell scripts, download and copy them to the target host, most likely the management server for the specific operation (export and/or import). It may be wise to execute a dos2unix operation against the scripts to ensure they are no longer in DOS text format. Tailoring the shell scripts can provide personalized control over root paths for target output and CSV files as well as any desired tweaks to file naming and operational output. Executing each script will start with requiring authentication to the SmartConsole administrator account used for the mgmt_cli operations, set with variable "APICLIadmin"; successful authentication starts the process of output generation or import of the identified CSV file.

To generate the required CSV file with groups and group members, execute the "cli_api_dump_objects_group-members_build_import_csv_v00x01.sh" script on the management server that has the groups with members to export. The export will handle up to the defined limit number of groups as defined in the variable "APICLIObjectLimit", which can be modified to suite testing or complete execution.

NOTE: the array population with the groups is a quiet process without visible action, so patience is advised when specifying a large value for the variable "APICLIObjectLimit".

Transfer the generated CSV file to the target management server that will import the group members.

Execute the "cli_api_import_objects_csv_group-members_v00x01.sh" script with the generated CSV (identified with variable "CSVImportPathFile") in the expected directory (identified with variable "CSVImportPathRoot")

Configuration

These script examples attempt to provide some detail tailoring and configuration via variables set for the specific script. This version does not make the approach overly generic (e.g. name of exported CSV file is hardcode in the import), and future versions of this example set may clearly abstract and configure command line input variables.

Key values to configure for:

Both Export and Import

Variable	Definition
APICLIadmin	SmartConsole administrator name to use for operations
APICLIsessionfile	filename and path to mgmt_cli session ID file generated by login and used for all subsequent mgmt_cli operations

Export

Variable	Definition
APICLIpathroot	root of path for output files
APICLIpathbase	base path for output files, generally uses $APICLIpathroot and for operations time delineation can utilize the $DATE variable
APICLIfileoutputpre	General prefix for the output file, prefixes the filename in the full output file path
APICLIfileoutputext	File extension for operational output file, default is .txt
APICLIfileoutputsufix	File suffix for the operational output file, default is $DATE.$APICLIfileoutputext so generally<date_time_group>.txt
APICLIJSONfileoutputext	File extension for mgmt_cli json output file, default is .json NOTE: this is not used in this example
APICLIJSONfileoutputsufix	File suffix for the mgmt_cli json output file, default is $DATE.$APICLIJSONfileoutputext so generally<date_time_group>.json NOTE: this is not used in this example
APICLICSVfileoutputext	File extension for generated CSV file, default is .csv
APICLICSVfileoutputsufix	File suffix for the operational output file, default is $DATE.$APICLICSVfileoutputext so generally<date_time_group>.csv NOTE: this was purposely done for the work utilizing this example, which stipulates a defined state of CSV output to export based on the time of execution. For those wanting a generic approach, the value can be set to be more static and not include the $DATE value element.
*APICLIObjectLimit*	This is the maximum number of groups to export, providing the limit value for the mgmt_cli show groups command to populate the array of groups to export members from.
APICLIoutput	full file path to operational output file for later review of actions

Import

Header 1	Header 2
APICLIfileoutputpre	General prefix for the output file, prefixes the filename in the full output file path
APICLIfileoutputext	File extension for mgmt_cli json output file, default is .json
APICLIfileoutputsufix	File suffix for the mgmt_cli json output file, default is $DATE.$APICLIfileoutputext so generally<date_time_group>.json
OutputPathRoot	root of path for output files
OutputPathBase	base path for output files, generally uses $OutputPathRoot and for operations time delineation can utilize the $DATE variable
CSVImportType	mgmt_cli type for import operation, in this example it is group
CSVImportPathRoot	This is the path root for the location of the CSV file to import, in the example it is a sub-directory relative to the location of the script
CSVImportPathFile	This is the file name of the CSV file to import, in this case hard-coded based the CSV output generated by the export operation. NOTE: this was purposely done for the work utilizing this example, which stipulates a defined state of CSV output to import. For those wanting a generic approach, the value can be set to be more static and not include the $DATE value element.
CSVImportPath	This is the path to the CSV file to import based on the $CSVImportPathRoot and $CSVImportPathFile variables.
OutputPath	full file path to operational output file for later review of actions

Modification of the script sections to suit personal preference and requirements is strongly encouraged via the copy-paste operation.

I may be updating these later, with some harmonization of common variables required and some abstraction options via command line parameters.

Why and What for...

These scripts were developed to address a pressing need in my own basement cloud laboratory, after some issues cropped up with my migrated management server, which has an original data base starting from R70 and migrated, upgraded, imported to Multi-Domain Management, and now exported from Multi-Domain Management, which has left the system a bit wonky and questionable. By creating scripts to handle the output of objects from my existing management server, I can then use the CSV data to import to a clean, new installation, where I can start fresh, with all my objects, but probably none of the baggage or garbage from almost 9 years of lab/home use operations. It is an excellent learning opportunity and mentors like Uri Bialik help with this very much.

However, these scripts can also help with some other operations that may be necessary, probably requiring some tweaks, but the example can help a bunch for starting out, operations like:

Duplicating group members after group import for laboratory environments, when building a from-scratch test environment, but wanting to use familiar objects
Duplicating group members after group import to a different Domain in Multi-Domain Management, when not wanting to use global objects. This will require some adjustments to handle the Domain specification variables, but should not be rocket magic to make happen.
Operations in Professional Services for either recovering from an exported baseline, or assisting with pre-migration testing operations, and rebuild operations
Having a backup of objects on hand to utilize for bare-metal rebuild where a import or restore is not plausible or advisable.

Code Version

Code version 0.1.0

Tested on version

R80, API version 1.0

NOTICE: By using this sample code you agree to terms and conditions in this Not authorized to view the specified document 1042

...

topic CLI API Example for exporting group members to a CSV file for later import in API / CLI Discussion

CLI API Example for exporting group members to a CSV file for later import

Overview

Description

Instructions

Configuration

Why and What for...

Code Version

Tested on version