R80.10 API policies installation benchmark

[Intelliment_Sec]

I have performed a benchmark of times for policies installation comparing R77.30 through dbedit, R80.10 through mgmt_cli and R80.10 through web services. I will appreciate any suggestion to reduce the R80.10 API times and corrections about the tests.

Testing has been performed with a server virtual machine and a different gateway virtual machine. Both of them are deployed in my local network, so network latency is minimum. The table below shows the devices settings:

Device	Memory	Processors	Hard disk
Check Point R77.30 (server)	1 GB	1	21 GB
Check Point R77.30 (gateway)	1 GB	1	21 GB
Check Point R80.10 (server)	6 GB	2	60 GB
Check Point R80.10 (gateway)	4 GB	2	15 GB

The benchmark consists of 254 host_plains, 254 network_object_groups, 254 tcp_services, 254 service_groups and 254 security_rules.

The table below shows the final results:

Method	Time (hh:mm:ss)
Check Point R77.30 (dbedit)	00:03:46
Check Point R80.10 (mgmt_cli)	00:31:20
Check Point R80.10 (web services)	01:07:50

The command below shows an example of the mgmt_cli schema used:

mgmt_cli add host -b hosts.csv --root true

The request below shows an example of the requests sent through Postman to the web services:

Method:
POST

URL:
{{server}}/add-host

Headers:
Content-Type:application/json
X-chkp-sid:{{session}}

Body:
{
 "name": "h1.1.1.1",
 "color": "orange",
 "comments": "1.1.1.1",
 "ip-address": "1.1.1.1"
}

[PhoneBoy]

While 6GB of RAM and 2 cores is the minimum requirement, performance with this configuration won't be that great.

I would repeat the test with a more realistic allocation of resources to the R80.10 Management VM (16GB of RAM and 4 cores), which is similar to what the Smart-1 405 appliance includes, as documented here: Smart-1 405 and 410

[Uri_Bialik]

Behind the scenes the "mgmt_cli" tool is making web-services request so I guess there should be a different reason why the mgmt_cli test was faster than the web-service test.

In R80.10 the policy installation process was changed: In R77.30, the management server verifies the entire policy every time a policy is pushed. In R80.10, the management server looks at the changes since the last policy push and verifies only the relevant parts. This can explain why a second policy push might be faster than the first one.
In large policies the typical performance gain from this change is a policy push twice as fast compared to R77.30.

Having said that, yes, using dbedit is faster for the following reasons:

* It does not perform validations.

* It does not create a database revision.

* It uses a more simple global lock on the database.

* It does not require a "publish" operation.

Many of us in Check  Point's RnD are constantly working to minimize this gap and improve the performance of the management server in general and the APIs in particular.

[Uri_Bialik]

I created a python script that uses web-services to creates 254 host objects, 254 group objects, 254 TCP objects, 254 service groups and 254 firewall rules (just like your benchmark but without the policy installation part).

On a VM running on my laptop (6 GB of RAM and 2 cores), it took the script 12:35 minutes to complete.

I'll contact you to share the script and understand the differences between our benchmark results.

[Daniel_]

I'm interested in if you have found the problem?

Is your script using Keep-Alive Connections? I think with Keep-Alive connections you should get a much better performance.

You have mentioned that mgmt_cli is making web-service request. If you use mgmt_cli and publish your changes does it do one keep-alive connection to publish all changes? This would explain the differences.

Can you share your script here?

Thanks in advanced!

[RickLin]

Does anyone ever observe the CPU clock rate on R80.10 management server ?

If use physical server with CPU Intel Xeon E5-2660v3, 2.60GHz *2 , will it better?

[PhoneBoy]

Stronger CPUs will definitely be a net-positive here.