automation time bases rulebases

[Ram1]

HI Team,

I need to check and extract all the rule bases which are going to expire in next 15 days ( time object), is there any script in ansible/shell/python and mgmt cli api call satisfies above condition. and below are my needs as well.

1) to get full file bases which are disabled using a script.

2) how alert rules that are going to expire through email.

3) how to get the details of expired rule bases

also please let me know how to start with mgmt_cli_api call.

Regards,

Ram

[PhoneBoy]

Just to be clear, rules never really "expire."
That said, you can add a time element to a rule and specify an "end" date so the rule will no longer apply after that date. 

Regardless, I've never seen anyone post a script that looks for these rules.
If I were to write such a script, I'd probably do something like:

• Iterate over each time object to figure out which one(s) are about to expire (or have): https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/show-times~v1.7%20 
• Use "where-used" to figure out where each object is being used: https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/where-used~v1.7%20 

[Ram1]

HI Phone Boy,

Thanks you  for your reply, since im new to mgmt_cli api and scripting , im stuck with how to proceed. i tried but couldn't make up syntaxes correct, could you pls help me to start up with.

my requirement is , we have rules bases which are time bounded and we want to alert our team by 10 days  before it rules go to expire.  how can i check time object of all the rule bases by iterating and extracting the same... since im new to mgmt_cli...im not sure of syntaxes and how to proceed. could you please help me.

Regards,

Ram

[PhoneBoy · Accepted solution]

If you're new to the API and scripting in general, I recommend going through our Automation and Orchestration lab on CheckMates Labs: https://community.checkpoint.com/t5/CheckMates-Labs/ct-p/checkmates-labs 
Here's a couple of starters for the points I mentioned above (will need more work to fully automate):

• mgmt_cli -s xxx.txt --format json show times details-level full | jq '.objects[]'
• mgmt_cli -s xxx.txt --format json where-used uid xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx

[Bob_Zimmerman · Accepted solution]

Expanding a little, looks like you need to filter for time objects where "end-never" is set to false. Something like this should do:

mgmt_cli -r true --format json show times details-level full limit 500 offset 0 \
| jq -c '.objects[]|.' \
| grep '"end-never":false' \
| jq -c '.uid,.end."iso-8601"' \
| xargs -L 2 sh -c 'if [[ "$1" < "$(date -d "+10 days" -Iseconds)" ]];then echo $0;fi'

That should output the UUIDs of time objects with end dates before ten days in the future, one per line. You can assign them to a shell variable for iteration, or pipe them through xargs. Whichever way you do it, you will need to find where the time objects with those UUIDs are used.

The pipeline above will only show the first 500 time objects you have. If you have more than 500 time objects, you will need to run it several times increasing the offset 500 each time.

[Ram1]

thanks, bob...but what if i want to export the whole rule bases to a csv, also..will this one script checks for all the rule bases across all the policies in management server?.

[PhoneBoy]

This is one of the reasons I suggested looking at the time objects directly and using where-used against them: it will show all the rules in all policy layers where these objects are used.
If you start with a given policy layer and try and work out what rules will expire soon, it will likely be a whole lot more work, especially if the policy layer has hundreds of rules.

[JozkoMrkvicka]

In general, in case of time objects, we need to take into consideration following conditions:

1. Time objects can be used in already disabled rules

2. Time objects can be used in more rules

3. Some rules can have more than 1 time object (one of them already expired, second one still valid)

4. Not all created time objects must be used in rule

5. Time objects can be created as Global Objects used in different Domains (in case of MDS)

[Bob_Zimmerman]

Sure, but the code above should get you to the time objects you need to care about within the current domain. Everything you're talking about (except item 5) is something you do after you have the time objects you need to care about.

[JozkoMrkvicka]

On the other hand, why we should take care about scripting such a thing at all ?

In R77.30 there was nice feature where you typed "expired" in rulebase search bar and you will get all rules which are expired.

Why we need to script such a basic function which was there in R77.30, but is still missing in R8x versions ?

By the way, isnt it already baked in R8x Compliance blade ? Isnt Compliance blade checking all disabled/expired rules?

@PhoneBoy 

[Corinne_Vakulen]

Hello Jozko

Indeed we had a default BP “FW-145” that was part of R77.30, showing access rules that have zero hit counts on the last 6 months. Hit Count search was broken from R80.

We have introduced a new option to fix the hit count on custom Best Practices for the previous 3 months as well, so customers can create their own BPs according to their needs.  The fix will be merged into R81.10, unless there is an urgency from the customer’s side.

TX

Corinne

[JozkoMrkvicka]

Hi Corinne,

Thanks for great info.

What about expired/rules with time objects ? Can we expect this to be included in R81.10 BP as well ?

For example, I would like to know how many + which rules are already expired (based on time objects), or will expire in next XY days/months.

[Corinne_Vakulen]

Hi Jozko,  just to add that indeed it's a different feature.

Today Compliance cannot check on the time stamp, but if you create a FW User Best Practice and define the search on all rules with a Time object= Expire, Compliance will search and show all rules with Time=Expire. (see attached).  By double clicking on the rules found, you can drill down up to the rule base, from the Compliance DB.

I hope it helps !

Corinne

[PhoneBoy]

The architecture of R77.x and R80.x is very different on the management side.
When you load up a rulebase in SmartDashboard, pretty much everything you saw was loaded into memory. 
This was one of the reasons concurrent admins weren’t supported.
However, it did allow SmartDashboard to perform queries on the data very easily.
I didn’t even realize that SmartDashboard had an option to show expired rules.

In R8x, most everything you see in SmartConsole is the result of API calls and little is cached client-side.
This means we pretty much had to re-implement every feature that SmartDashboard had or, in the few cases where CPMI is still used, reuse those components.
In any case, this was clearly not a feature we have (yet) re-implemented in R8x.

My guess is we could implement this as a SmartConsole Extension similar to what was done for the Change Report.
However, it’d officially be an RFE.
If it’s something you need, I recommend engaging with your local Check Point office.

[Ram1]

thanks bob, one more query can i able to loop it here itself or do i need to run another script,. how can i loop to check all the time objects that i have? or is there a way in which i can delete unused time objects. 

[PhoneBoy]

What we're providing you is fragments of a potential script.
This could be done as a single script, but would need more work to develop.

[fwmeister · Accepted solution]

I'd read the API documentation and the jq documentation. Yes, it is a bit tricky for a beginner like me but it gets easier with use.

This should get you started with what you need...   

DAYS=20
DOMAIN="MGMT"
TMPDIR="/var/log/expiring"
STARTH=$(date +%m/%d/%Y )
echo Starting now: $STARTH
START=$(date -d "$STARTH" +%s%N | cut -b1-13)
ENDH=$(date +%m/%d/%Y -d '+'$DAYS' days')
echo Ending $ENDH
END=$(date -d "$ENDH" +%s%N | cut -b1-13)
TOFILE=$TMPDIR/time-object-name.txt
REFILE=$TMPDIR/rules_expiring.txt
ORFILE=$TMPDIR/old_rules.txt
PFILE=$TMPDIR/policies.tmp

printf "\nSearching for Rules that are within $DAYS days of expiring in $DOMAIN.\n"

mgmt_cli -r true -d $DOMAIN show times details-level full limit 500 --format json | jq --arg START ${START} --arg END ${END} --raw-output '.objects[] | select( (.end.posix|tonumber) >= ($START|tonumber) and (.end.posix|tonumber) <= ($END|tonumber) ) | .name ' > $TOFILE

mgmt_cli -r true -d $DOMAIN show access-layers limit 500 --format json | jq --raw-output '."access-layers"[] | (.name)' | grep "\ Security" > $PFILE

OFS=$IFS
IFS=$'\n'
for POL_NAME in $(cat $PFILE); do

IFS=$OFS

echo "Search policies for the expiring time objects.."
for line in $(cat $TOFILE);
do
echo "Searching for time object $line in the $POL_NAME"
f_log "Searching for time object $line in the $POL_NAME" $LOGFILE
mgmt_cli -r true -d $DOMAIN show access-rulebase limit 500 name "$POL_NAME" details-level "standard" use-object-dictionary true filter "$line" --format json | jq --raw-output '.rulebase[] .rulebase[] | ."rule-number"' > $ORFILE
done

for rule_num in $(cat $ORFILE ) ;
do
echo "Expiring Rules on $DOMAIN $POL_NAME:"
mgmt_cli -r true -d $DOMAIN show access-rule layer "$POL_NAME" rule-number "$rule_num" --format json |jq --raw-output --arg PN "$POL_NAME" --arg RN "$rule_num" '($PN + "," + $RN + "," + .source[].name + "," + .destination[].name + "," + .service[].name + "," + .action.name + "," + .time[].name + "," + .comments)' >> $REFILE
done

done
cat $REFILE