topic Re: Parsing the output of "mgmt_cli" in API / CLI Discussion

Parsing the output of "mgmt_cli"

Uri_Bialik — Wed, 13 Jan 2016 12:10:05 GMT

When running a mgmt_cli command the output of the command is presented in text format.

Although Unix offers many tools for parsing text (e.g. cut, grep and awk), a better approach for parsing the output of mgmt_cli is to use json format

Once the output of the mgmt_cli command is in json, it is possible to use standard tools such as jq to do all the parsing work for you.

This document provides examples for using jq to handle the output of a mgmt_cli command.

To learn more about jq, you can visit https://stedolan.github.io/jq/ and https://jqplay.org https://jqplay.org/

Making mgmt_cli produce json instead of text output is easy. Simply add --format json to your mgmt_cli command.

For example:

mgmt_cli show host name "my host" --format json

Extract values from a mgmt_cli json response:

Extract the a top-level field

mgmt_cli show-host name h1 --root true --format json | $CPDIR/jq/jq ".color"

"black"

Extract a field inside another field:

mgmt_cli show-host name h1 --root true --format json | $CPDIR/jq/jq ".domain.name"

"SMC User"

Extract a field that contains non-alphanumeric characters like “-“

mgmt_cli show-package name Standard --root true --format json | $CPDIR/jq/jq '.["installation-targets"]'

"all"

Extract a field inside another field (both have hyphen in their name):

mgmt_cli show-host name h1 --root true --format json | $CPDIR/jq/jq '.["meta-info"]["last-modifier"]'

"administrator1"

Tip adding the “-r” parameter remove the quotation marks from the output:

Example:

mgmt_cli show-host name h1 --root true --format json | $CPDIR/jq/jq ".color"

"black"

mgmt_cli show-host name h1 --root true --format json | $CPDIR/jq/jq ".color" -r

black

Working with arrays:

Get the array:

mgmt_cli show-group name "My Group" --root true --format json | $CPDIR/jq/jq ".members"

[

{

"domain": {

"domain-type": "domain",

"uid": "41e821a0-3720-11e3-aa6e-0800200c9fde",

"name": "SMC User"

"type": "group",

"name": "IPs that should reach the sales application",

"uid": "2a83b2c4-8f82-41a4-9763-356e49c0947c"

{

"domain": {

"domain-type": "domain",

"uid": "41e821a0-3720-11e3-aa6e-0800200c9fde",

"name": "SMC User"

"type": "host",

"name": "h1",

"uid": "ce18c250-98c5-464e-94ff-04845e64a233"

{

"domain": {

"domain-type": "domain",

"uid": "41e821a0-3720-11e3-aa6e-0800200c9fde",

"name": "SMC User"

"type": "host",

"name": "h2",

"uid": "13faa04f-a6d8-4b72-aab7-33fadaf0f134"

}

]

Get the array elements (it would remove the [ ] around the array items and the commas between the items)

mgmt_cli show-group name "My Group" --root true --format json | $CPDIR/jq/jq ".members[]"

{

"domain": {

"domain-type": "domain",

"uid": "41e821a0-3720-11e3-aa6e-0800200c9fde",

"name": "SMC User"

"type": "group",

"name": "IPs that should reach the sales application",

"uid": "2a83b2c4-8f82-41a4-9763-356e49c0947c"

}

{

"domain": {

"domain-type": "domain",

"uid": "41e821a0-3720-11e3-aa6e-0800200c9fde",

"name": "SMC User"

"type": "host",

"name": "h1",

"uid": "ce18c250-98c5-464e-94ff-04845e64a233"

}

{

"domain": {

"domain-type": "domain",

"uid": "41e821a0-3720-11e3-aa6e-0800200c9fde",

"name": "SMC User"

"type": "host",

"name": "h2",

"uid": "13faa04f-a6d8-4b72-aab7-33fadaf0f134"

}

Get a single field from every array element

mgmt_cli show-group name "My Group" --root true --format json | $CPDIR/jq/jq ".members[].name"

"IPs that should reach the sales application"

"h1"

"h2"

Get a single field from every array element, key name contains a hyphen

mgmt_cli show-package name standard --root true --format json | $CPDIR/jq/jq '.["access-layers"][].name'

"Network"

"my layer"

Get multiple fields from every array element

mgmt_cli show-group name "My Group" --root true --format json | $CPDIR/jq/jq ".members[] | [.name , .type] "

[

"IPs that should reach the sales application",

"group"

]

[

"h1",

"host"

]

[

"h2",

"host"

]

mgmt_cli show-group name "My Group" --root true --format json | $CPDIR/jq/jq ".members[] | [.name , .type] | @csv" -r

"IPs that should reach the sales application","group"

"h1","host"

"h2","host"

Using jq in a bash script:

In the bash script below, you can see how to extract simple fields and arrays from a mgmt_cli command into bash variables.

#!/bin/bash

#points to where jq is installed

JQ=${CPDIR}/jq/jq

# login as root without providing credentials

mgmt_cli login --root true > id.txt

# save the details of a host object called "h1" and save them to a file called "host_details.json"

mgmt_cli show host name h1 -s id.txt --format json > host_details.json

# get some of the host's fields and store them as bash variables

HOST_NAME=$($JQ -r ".name" host_details.json)

HOST_LAST_MODIFY_TIME=$($JQ -r '.["meta-info"]["last-modify-time"]["iso-8601"]' host_details.json)

echo "host name: $HOST_NAME"

echo "last modified: $HOST_LAST_MODIFY_TIME"

# Working with json arrays

# MGMT_CLI_OUTPUT is a string with multiple lines. Each line contains a name of a group members.

# in this example the output of mgmt_cli is not sent to a file, instead it is passed to jq directly using a pipe.

MGMT_CLI_OUTPUT="`mgmt_cli show group name "my group" -s id.txt --format json | $JQ ".members[].name | @sh" -r`"

# break the string into an array - each element of the array is a line in the original string

# there are simpler ways, but this way allows the names to contain spaces. Gaia's bash version is 3.x so readarray is not available

ARR=()

while read -r line; do

ARR+=("$line")

done <<< "$MGMT_CLI_OUTPUT"

# print the elements in the array

for i in "${ARR[@]}"

echo "$i"

done

# using bash variables in a jq expression

MEMBERS_COUNT=$(mgmt_cli show group name "my group" -s id.txt --format json | $JQ ".members | length")

echo "number of members in the group: $MEMBERS_COUNT"

COUNTER=0

while [ $COUNTER -lt $MEMBERS_COUNT ]; do

MEMBER_NAME=$(mgmt_cli show group name "my group" -s id.txt --format json | $JQ ".members[$COUNTER].name")

echo $MEMBER_NAME

let COUNTER=COUNTER+1

done

# sometime the jq expression includes the character ' which means that it will not handle the variable properly

GROUP_LAST_MODIFY_TIME=$(mgmt_cli show group name "my group" -s id.txt --format json | $JQ '.["meta-info"]["last-modify-time"]["iso-8601"]')

echo $GROUP_LAST_MODIFY_TIME

# this will not work:

#key="iso-8601"

#group_last_modify_time=$(mgmt_cli show group name "my group" -s id.txt --format json | $JQ '.["meta-info"]["last-modify-time"][$key]')

# using jq's --arg option it is possible to define variables that can later be used inside the jq expression even if the expression is surrounded by single quotes

KEY="iso-8601"

GROUP_LAST_MODIFY_TIME_WITH_VARIABLE=$(mgmt_cli show group name "my group" -s id.txt --format json | $JQ --arg myvar "$KEY" '.["meta-info"]["last-modify-time"][$myvar]')

echo $GROUP_LAST_MODIFY_TIME_WITH_VARIABLE

Re: Parsing the output of "mgmt_cli"

Eric_Beasley — Thu, 30 Jun 2016 16:40:41 GMT

Thanks for the examples above, it really helps understand some of the nuances.

Silly question regarding this section of the example code:

# Working with json arrays

# MGMT_CLI_OUTPUT is a string with multiple lines. Each line contains a name of a group members.

# in this example the output of mgmt_cli is not sent to a file, instead it is passed to jq directly using a pipe.

MGMT_CLI_OUTPUT="`mgmt_cli show group name "my group" -s id.txt --format json | $JQ ".members[].name | @sh" -r`"

# break the string into an array - each element of the array is a line in the original string

# there are simpler ways, but this way allows the names to contain spaces. Gaia's bash version is 3.x so readarray is not available

ARR=()

while read -r line; do

ARR+=("$line")

done <<< "$MGMT_CLI_OUTPUT"

# print the elements in the array

for i in "${ARR[@]}"

echo "$i"

done

I played around with that to build a script that exports the members of all groups to a two column (group name, member name) CSV file to allow me to rebuild the group membership quickly from the CSV file. The while loop section above is used to generate the array of groups, for subsequent processing.

When running the code, the while loop operates in a closed system, and no output is visible until it completes, something that can take some time, based on number of groups defined, so it is not clear that anything is happening. When the loop completes, it dumps any output and the proceeds to the output for loop.

I tailored the while loop like this:

ARR=()

while read -r line; do

ARR+=("$line")

echo -n '.'

done <<< "$MGMT_CLI_OUTPUT"

echo

But when it executes, you see nothing then it blasts out a line of dots equaling the number of groups read.

Is there a better way to achieve this type of feedback output that will actually display in real time, not postpartum?

Re: Parsing the output of "mgmt_cli"

Sven_Glock — Fri, 11 May 2018 12:02:08 GMT

Very helpful - thanks!

Re: Parsing the output of "mgmt_cli"

Matthias_Kring — Mon, 12 Oct 2020 05:03:39 GMT

What's the correct syntax to extract IP-Adresses? When I say
mgmt_cli show host name "mypc" --format json | jq ".ipv4-address", I get the error
error: address is not defined
.ipv4-address 1 compile error

Thanks in advance

Re: Parsing the output of "mgmt_cli"

PhoneBoy — Mon, 12 Oct 2020 05:15:47 GMT

Put "ipv4-address" in quotes.

Re: Parsing the output of "mgmt_cli"

Matthias_Kring — Mon, 12 Oct 2020 06:33:32 GMT

Hmm. As you can see above, it already is in quotes.

This one works: jq '.["ipv4-address"]'

Re: Parsing the output of "mgmt_cli"

Norbert_Bohusch — Wed, 14 Oct 2020 11:26:47 GMT

you put the quotes around the dot as well ( ".ipv4-address" ), but it needs to be after the dot: ."ipv4-address"

Re: Parsing the output of "mgmt_cli"

Matthias_Kring — Wed, 14 Oct 2020 11:34:42 GMT

Sorry, but this doesn't work for me either:

mgmt_cli show host name "mypc" --format json | jq ."ipv4-address"
error: address is not defined
.ipv4-address 1 compile error

Re: Parsing the output of "mgmt_cli"

Jim_Oqvist — Wed, 14 Oct 2020 11:59:58 GMT

You need to include the jq filter in single quotes ' my_filter'

like this

mgmt_cli show host name "mypc" -r true --format json | jq '."ipv4-address"'

Re: Parsing the output of "mgmt_cli"

Norbert_Bohusch — Wed, 14 Oct 2020 12:06:45 GMT

Sorry, you are right. My part is only relevant if it is part of a longer command like below (disconnect all WEB_API sessions):

mgmt_cli -r true show sessions details-level full --format json | jq -r '.objects[] | select(."user-name"=="WEB_API") | "mgmt_cli -r true disconnect uid " + .uid + " discard true"' | bash

Here the outer quotes you mentioned are there but enclose the whole jq statement.

PS: you don't need the brackets though