Dynamic revisions in R80.x SmartConsole

[ED]

This document has its content based from this great video https://community.checkpoint.com/videos/6461-lightboard-series-dynamic-revisions by Yair Herling which use a nice example to show how the logic behind R80.10 dynamic revisions happens. Hope it's ok that I use it this way. 

Pre R80 revisions:

Revisons were saved as backup of the management database. Each revision replicates the management database completely.This had effect on the available free space of the security manager server. Used to restore the database in case of disaster. Many users on this community have been missing this option and not quite understood the new architecture for revisions in R80.x.

R80.x revisions:

R80 SmartConsole architecture has built-in revisions. Every time you publish a session, an additional revision is created and automatically stored. Consuming far less disk space with very little footprint. Contains only the changes from the previous revision. Faster policy verification using the difference between installed revisions. Enables the collaboration of multiple admins without conflict. More efficient management high availability based on the changes made since previous synchronization. 

Databases in R80.x SMS: contains two databases

SOLR: 

Events are stored in the SOLR database. SOLR enables efficient textual searchers across all the data. When you run cpwd_admin list command it shows SOLR. Log file is located at $RTDIR/log/solr.log and $RTDIR/log/solrRun.log. To start or stop SOLR you can run evstop or evstart. 

PostgreSQL: 

Stores management configuration. Based on tables and the relationship between them. Contains hosts, revisions, etc..

Data is segmented into multiple database domains. Database domain types are explained in this document R80.10 Security Management Architecture Overview

A logical process called CPM manages these databases and provides reliable validations and revisions for the users. CPM is reponsible for writing all information to the PostgreSQL and SOLR databases. SmartConsole GUI application and the REST APIs are the only ones that can interact with the CPM. 

Dynamic revisions example: a user wants to enable NAT on a host (example from video)

Host is stored in PostgreSQL like this:

Hosts

Object ID	Name	IP address	Enable NAT	Version ID
123456	London VM	10.10.10.45	NO	541

That same host when it was created was stored like this in PostgreSQL 

Version

Object ID	Active admin	Description	Publish time
541	Bob	Created new host London VM	1/4/18 13:00

Notice the connection between these two tables, the object ID and version ID marked with green. 

After an admin decides to enable NAT and publish his changes, an additonal row for that host object will be created and the version that it was modified from will get updated.  

Now the Hosts table looks like this:

Hosts

Object ID	Name	IP address	Enable NAT	Version ID
123456	London VM	10.10.10.45	NO	541
123456	London VM	10.10.10.45	YES	542

Notice that the object ID stays the same for the host but the value for 'enable NAT' changed to 'YES'. Important also, a new version ID 542 was added (marked with red). This version ID is linked to Object ID in the version table as shown below.

Now the Version table looks like this:

Version

Object ID	Active admin	Description	Publish time
541	Bob	Created new host London VM	1/4/18 13:00
542	Dan	Enable auto NAT for London VM	2/4/18 15:00

We can see that another admin, Dan enabled NAT for the London VM. This change is linked from version table to hosts table with the ID 542. 

Above example shows that versions are really small and only contain the incremental change or difference. Because of this you can store a large number of versions without worrying about disk space. 

Read-only view of revisions

The object ID from each version is linked as we saw by the objects that were changed in that version which makes it posssible to calculate the state of each object in any given session. 

The login and 'which object to see' mechanism are made by the CPM process so it's not something that the GUI client needs to worry about. This way you can log in with read-only mode for version '541' from our example above and see that the London VM had not enabled NAT. An audit log for versions '542' was generated when admin Dan enabled NAT from the list of rows that are linked to versions '542' as well as their previous value. To get the previous value a silent login is made to the previous version '541' and gets the value of 'enable NAT' column for the host London VM. 

How do you manage crisis using revisions? There are different scenarios. Examples from CheckPoint. 

Case 1:  Connectivity/security problem after changes on the policy are performed and installed.

Solution: Security Policy -> Installation History > Policy installation history > choose last known good version > Install specific version. After a gateway is safely re-installed, query the audits and changes made in each revision on the management.

Note! This does not change any configuration that you see in SmartConsole but ONLY changes what is installed on the gateway back to a previously working configuration. The problem is if you reinstall the current security policy again, then you will get back where you started. Your job is to do investigation and redo the changes. Tim Hall  got you covered with this great investigation document R80+ Change Control: A Visual Guide

Case 2:  Network problem after downloading Threat Prevention update and installing it on gateways.

Solution:  Threat Prevention > Updates, choose an update known to be good.

Case 3: Full environment restore to a certain point in time is needed

Solution: Restore from Backup. This cause a loss of all the work done after that point.

Case 4: A change in your policy that you want to revert

Solution: Select your security policy > Actions > History.. > select previous revision > Revert. This will revert to a previous state. It will revert the rulebase structure but not the objects used in the rulebase. 

I'm open for corrections in this document as I might have misunderstood some parts. How was your transition from R77.x to R80.x new architecture? Do you understand the new revisions architecture and how to revert things back? 

[Vladimir]

Enis,

Thank you for this post.

I think that the decision of decoupling "emergency fallback" on the gateways from active version of the policy is extremely poor decision. Depending on how many changes were made between revisions by number of administrators, investigation in to the root cause of the problem may take a while and is inconvenient.

If the "Version" you are depicting in your table pertains to the policy revision and not the revision of the individual object, the better approach would probably be to add one more column to the table with "Active; Yes/No" values.

When rolling back, the object's properties could then be changed accordingly.

The new "unsuccessful" policy could be then displayed with "Inactive" marker and all the changes in place.

For the life of me, I do not see a single good reason for the current policy NOT depicting actual working state of your environment.

[ED]

I agree that the investigation may take a while and is inconvenient based on the number of changes and number of administrators. The "version" table for me looks like it pertains to the policy revision if the "object ID" in that table is linked to several objects in the "hosts" table if you did several changes. 

[Tomer_Sole]

You are correct that every change is a new link in the revisions table.

One more thing - everything on the Management server is revisioned, not just policies and objects. Gateway interfaces, global policy assignment operations, connecting CloudGuard IaaS Data Center Objects, if we save it, we revision it. If I can think of a common limitation then maybe if you want to know the history of your Active Directory roles, that's not something we can provide, but we can tell if you picked and unpicked them in Access Role objects.

[Tomer_Sole]

Hi,

Thank you for this feedback. I can see your point on this one. The reason why we chose not to do that was because we have many triggered changes for various operations. For example, creating a new security policy adds 1 layer and a default rule to it. Or: Enabling IPS on a pre-R80 gateway adds a rule to the IPS Shared Layer - an ordered rulebase where first-match and positions matter. So simply discarding a single revision from the middle of the stack would result in a state that the user never had to begin with. 

We hope that things like the audit logs and show-changes API command can help you manually revert a single change from a list of published revisions. 

We do have plans to increase the core capability of the dynamic revisions in our next releases. I think I might contact you to describe this as the developers make the improvements.

[Vladimir]

Thanks!

I'd be happy to assist.

[GG27]

great topic