Export Objects - Applications / Categories and Network Objects

[George_Sas]

Hi guys.

Is there a way to export all the objects defined on my management server ?

I have around 3K network objects and 12K Applications / Categories.
I want to export everything and make a general cleanup on unused objects or double defined custom apps / urls.

I see no other way of doing this in any other way.

Thanks in advance.

[Danny]

Create a Gaia backup of your SmartCenter Server and perform a migrate_server export of your management database before you start cleaning up your objects.

My Policy-Audit-Extension might be of help with the cleanup.

You could also SSH login into the expert mode of your SmartCenter Server and run $MDS_FWDIR/scripts/web_api_show_package.sh -n 443 -c to export your security policies and network objects to a HTML file for further analysis.

[George_Sas]

I was thinking more of an export of Apps / Objects in CSV format or something so I can check it directly in a file.

[Tal_Paz-Fridman]

In SmartConsole, open the Object Explorer (Ctrl + E).

Select a single category (or subcategory) for clear separation, then choose Actions > Export.

[George_Sas]

My hero ! 🙂 Totally forgot even if I previously used it !

Works fine to export the objects , but not exactly useful in this situation.
I can see object name but not the actual data.

So for example if I have an URL_xxx and inside I have an URL List ... I can not see the list of URL's. This is what I am after as I can have 2 custom Apps with diferent name but same URL definitions inside.

Example:

In CSV I can not see the URLs.

Thanks.

[Bob_Zimmerman]

For application-site objects specifically, this script run on the management in expert mode will get the raw data for your custom objects:

portNumber=$(api status | grep "APACHE Gaia Port" | awk '{print $NF}')

showAll() {
IFS=$(printf "\377")
sharedArguments=( --port "${portNumber}" -f json ${cmaAddress:+-d} ${cmaAddress:+${cmaAddress}} -r true show "${1}" details-level full limit 500 )
if ! firstResult=$(mgmt_cli "${sharedArguments[@]}");then return 1;fi
toReturn="$(<<<"${firstResult}" jq -c '.objects[]|.')
";objectCount=$(<<<"${firstResult}" jq -c '.total')
if [ "${objectCount}" -lt 501 ];then echo -n "${toReturn}";return 0;fi
for offsetVal in $(seq 500 500 "${objectCount}" 2>/dev/null | tr "\n" "${IFS}");do
toReturn+="$(mgmt_cli "${sharedArguments[@]}" offset "${offsetVal}" \
| jq -c '.objects[]|.')
";done;echo -n "${toReturn}";}

allAppSites=$(mktemp)
showAll application-sites >"${allAppSites}"
wc -l "${allAppSites}"

customAppSites=$(mktemp)
jq -c 'select(true == ."user-defined")' "${allAppSites}" >"${customAppSites}"
rm "${allAppSites}"

The custom objects will be in the file "${customAppSites}". There are a few ways to convert it to CSV output, depending on how you want to represent the data. For example, like this:

cat "${customAppSites}" \
| jq '{uuid:.uid,
name:.name,
regEx:."urls-defined-as-regular-expression",
containedUrl:."url-list"[]}' \
| jq -r -s '(map(keys) | add | unique) as $cols 
| $cols, map(. as $row | $cols | map($row[.] | tostring))[] 
| @csv'

That will output one line per URL which tells you what object it's in.

[George_Sas]

Thanks , will try that as a last resort , thought there is something someone already made and used before.

[Danny]

@George_Sas: Please check if this backup extension for SmartConsole does what you are looking for.

[George_Sas]

Hmmm ... strange ... not working for me ?
Link works fine in a browser but nor when I try to add it in Extensions.

[George_Sas]

Tried with Python just to see if I can reach the site and in cli I get ssl error ?

python3 -c "import urllib.request; print(urllib.request.urlopen('https://dannyjung.de/backup.json').getcode())"
Traceback (most recent call last):
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/urllib/request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/http/client.py", line 1286, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/http/client.py", line 1332, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/http/client.py", line 1281, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/http/client.py", line 1041, in _send_output
self.send(msg)
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/http/client.py", line 979, in send
self.connect()
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/http/client.py", line 1458, in connect
self.sock = self._context.wrap_socket(self.sock,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/ssl.py", line 1075, in _create
self.do_handshake()
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/ssl.py", line 1346, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/urllib/request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/urllib/request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/urllib/request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/CPsuite-R82/fw1/Python/lib/python3.11/urllib/request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)>

[Danny]

@George_Sas: Are you using a Proxy which might prevent access to the SmartConsole extension?
Someone had a similar issue. Another possible solution is described here.

[George_Sas]

Issue found .... the error is "Error 40" ... Extension is installed on the SmartConsole itself and not management server.... so the Local Antivirus Firewall is blocking the SmartConsole.exe to access any external URL's !!! Took me a few hours ...
I will test on monday now , thanks !

[Danny]

I'm glad you solved it.
My backup extension should export your URL list successfully: