IPs to Rules as JSON or CSV

[Bob_Zimmerman]

I recently had the need to build a table out of all of the rules referencing any IP address in a list of addresses. Basically a rule audit for all the rules involved in a given application.

This post originally had several versions of the script attached directly, but I have since moved it to Github:

https://github.com/Bob-Zimmerman/ipsToRulesCP

USAGE

The script should be run as root (in expert mode, and with elevated privileges if you use low-privilege users) on the SmartCenter or MDS. Doesn't need any credentials. It does everything via the API in read-only mode.

Usage is given right at the top of the script. It also prints the usage if you run the script with no switches or if you run it with the -h switch:

[Bob_Zimmerman@MySmartCenter]# ./ipsToRules.sh -h
Usage:
./ipsToRules.sh [-d] [-h] [-J file] [-j file] [-c file] [-O] 
Default output is pretty-print JSON to STDOUT, suitable for output redirection.
	-d		Increase debug level, up to twice.
	-h		Print this usage information.
	-J file	Write pretty-print JSON output to .
	-j file	Write compact JSON output to . One line per rule.
	-c file	Write quote-delimited CSV output to .
	-O		Write pretty-print JSON output to STDOUT.
	list	List of IPs to search for, separated by spaces.

As you can see, it currently has options for compact JSON output, pretty JSON output, and quote-delimited CSV output. It should be pretty clear from the code how to write a new output formatter. Just needs a new variable for the name, a new switch in the getopts case statement, a little output prep work, and a new item in the "masterOutput" function.

The only privileged commands it uses right now are 'cpprod_util FwIsFirewallMgmt' (to detect if it is run on a firewall instead of a management) and 'mdsstat' (to detect if it is a SmartCenter or MDS), within a few lines of each other at the bottom. You can make a version which will work only on a SmartCenter or only on an MDS, and it would work as an unprivileged user.

KNOWN LIMITATIONS

It currently accepts only IP addresses. Haven't yet gotten around to writing logic for spotting CIDR notation, or for looking up networks once I've found them in the input.

There's a big case statement in the middle for dereferencing objects. It includes all the object types I personally needed, but I'm sure there are plenty which are not included.

I'm pretty sure there are error cases I don't handle properly, such as if none of the IP addresses are found.

[Bob_Zimmerman]

New version is available.

CHANGES

• Added object UID output to dereferencer failures to help find the object in question and write a new dereferencer case.
• Added four object types to the object dereferencer (service-other, CpmiHostCkp, address-range, Internet).
• Normalized multi-line pipeline styling.
• Fixed object dereferencer handling for spaces in object names.

[Bob_Zimmerman]

New version is available.

CHANGES

• Rearranged dereferencer objects to case-insensitive alphabetical order.
• Added more types to the dereferencer (group-with-exclusion, service-dce-rpc)
• Added basic object output for dereferencer failures.

[Bob_Zimmerman]

New version is available. I think this one is a pretty significant update.

Changes

• Added CpmiGatewayPlain to the dereferencer.
• Updated most of my sed scripts to use ASCII Unit Separators rather than # for the field separation. This is important because Unit Separator is a non-printing character. It can't naturally occur in the comments of an object, which enables the next feature:
• Added object comments to dereferencer output.

[Bob_Zimmerman]

I have decided to start pushing this script to Github:

https://github.com/Bob-Zimmerman/ipsToRulesCP

I will try to share major changes here, but smaller changes will only show up there. If you look at the commit history, you may see I have broken down the changes in a more granular way than I did here. The prior batches of changes were to avoid forum noise by only posting when I felt there were somewhat significant updates. Version control systems like Git, however, are really built with the idea of small, focused changes. That is the model I will be using in the future.

As an added benefit, now I get an issue tracker, and other people can send proposed changes more easily.

[PhoneBoy]

@_Val_ looks like a Toolbox addition. 

[Bob_Zimmerman]

Actually, this reminded me. Some time last year, I did get around to testing whether you can make a cyclic set of groups (group A contains group B, group B contains group A), and confirmed the management won't let you. Removed that concern from the known limitations!