Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
FK
Participant
Jump to solution

Mgmt_cli limitation when publishing a large number of newly created objects?

Hi CheckMates,

 

I have a question regarding mgmt_cli and limitation.

Is there a limit when publishing a large amount of newly created objects?

Let´s say you have a script which creates a network group for azure IPs with thousands of network objects.

Updatable objects is not an option -> 1400 appliances.

 

Thank you.

 

Greetings from Austria.

0 Kudos
2 Solutions

Accepted Solutions
_Val_
Admin
Admin

You need to mind API time out error, which is 10 min by default. If you push a long list of objects, and the command takes more than 10 minutes to post, API will return an error. The best practice is to push in smaller portions, multiple times. If each POST is short enough, you should be okay.

View solution in original post

Jim_Oqvist
Employee
Employee

Hi, In addition to all good recommendations you have received in terms of publishing 100 changes and  upgrade the hardware to be able to use updateable objects, I would like to point you to add/set/delete-object-batch https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/add-objects-batch~v1.8%20 wich are API endpoints that where added in R80.40. 

Batch API Significantly increase API performance in multiple object operations (add/set/delete).

  •          Object creation time reduced by up to 92% when compared to adding one-by-one*
  •          Object deletion time reduced by up to 87% when compared to deleting one-by-one*

*In the tested scenario of adding and deleting 256 hosts objects.

Commands:

  •          add-objects-batch
  •          set-objects-batch
  •          delete-objects-batch

Here is an example of adding 250 hosts objects

https://github.com/checkpointsw-devsec/chkp_api_examples/blob/master/mgmt_cli/mgmt_api_add-objects-b...

here is an example of deleting host patterns

https://community.checkpoint.com/t5/API-CLI-Discussion/mgmt-cli-to-delete-all-objects-matching-a-pat...

Please note that usually when calling mgmt_cli with credentials (without doin a explicit login) results in four different operations including Publish.

  • login
  • make change
  • publish
  • logout

Using the object-batch operation with mgmt_cli require you to do a explicit publish.

You can pobrobly be able to make more that hundred changes when using object-batch operation with out hitting any time out value as mentioned by Val, but the exact limit is dependent of your environment I suggest that you test with different number of changes in order to get to a value that suitable for your scenario.

Kind Regards

Jim

View solution in original post

12 Replies
_Val_
Admin
Admin

You need to mind API time out error, which is 10 min by default. If you push a long list of objects, and the command takes more than 10 minutes to post, API will return an error. The best practice is to push in smaller portions, multiple times. If each POST is short enough, you should be okay.

FK
Participant

Hi Guys,

 

thank you for inputs.

I will try to seperate it into smaller groups per publish.

 

Greetings.

Fabian

0 Kudos
Henrik_Noerr1
Advisor

This complicates rollbacks and revision control however. Not sure there is a great solution with the time it takes in general to communitate with the API on larger installations / changes

0 Kudos
Robert_Decker1
Participant

One solution is to increase the timeout on Apache server from default 10 minutes.

G_W_Albrecht
Legend Legend
Legend

How to achieve that ? Could not find it in 

/opt/CPshrd-R81/web/Apache/conf/cp-httpd.conf

CCSE / CCTE / CTPS / CCME / CCSM Elite / SMB Specialist
0 Kudos
PhoneBoy
Admin
Admin

The more changes you try to publish at the same time, the longer the process will take.
If you’re making a bunch of changes at once, it’s generally recommended to publish every few hundred changes or so.

JozkoMrkvicka
Mentor
Mentor

Is there some easy way how to force publish to be done once number of XY changes were done in one session ?

For example, I want to automatically publish 50 changes if performed in the same session.

If I did 160 changes in the same session, the first publish will be done after first 50 changes, next publish after next 50 changes (100 changes already done). The last 10 changes will be handled by "manual" publish.

Kind regards,
Jozko Mrkvicka
0 Kudos
PhoneBoy
Admin
Admin

On the API side of things? No.
You would have to handle that in the logic of your script.
You can commit with each mgmt_cli invocation by using -r true, however that incurs a lot of extra overhead (login/do/commit/logout with each command).

0 Kudos
Tomer_Noy
Employee
Employee

I would seriously consider upgrading the gateways to newer hardware (1500 appliances) in order to use the Azure Updatable Objects.

For anyone that is coming across this thread, it's a simple as choosing it in the Access policy:

 

Azure Updatable Objects.png

Filling your management with thousands of network objects and huge network groups isn't good practice and might introduce future slowness in some scenarios beyond the challenge of the initial publish via API.

Robert_Decker1
Participant

100 changes per publish is recommended.

Same goes for group members.

FK
Participant

Thanks for this info.

0 Kudos
Jim_Oqvist
Employee
Employee

Hi, In addition to all good recommendations you have received in terms of publishing 100 changes and  upgrade the hardware to be able to use updateable objects, I would like to point you to add/set/delete-object-batch https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/add-objects-batch~v1.8%20 wich are API endpoints that where added in R80.40. 

Batch API Significantly increase API performance in multiple object operations (add/set/delete).

  •          Object creation time reduced by up to 92% when compared to adding one-by-one*
  •          Object deletion time reduced by up to 87% when compared to deleting one-by-one*

*In the tested scenario of adding and deleting 256 hosts objects.

Commands:

  •          add-objects-batch
  •          set-objects-batch
  •          delete-objects-batch

Here is an example of adding 250 hosts objects

https://github.com/checkpointsw-devsec/chkp_api_examples/blob/master/mgmt_cli/mgmt_api_add-objects-b...

here is an example of deleting host patterns

https://community.checkpoint.com/t5/API-CLI-Discussion/mgmt-cli-to-delete-all-objects-matching-a-pat...

Please note that usually when calling mgmt_cli with credentials (without doin a explicit login) results in four different operations including Publish.

  • login
  • make change
  • publish
  • logout

Using the object-batch operation with mgmt_cli require you to do a explicit publish.

You can pobrobly be able to make more that hundred changes when using object-batch operation with out hitting any time out value as mentioned by Val, but the exact limit is dependent of your environment I suggest that you test with different number of changes in order to get to a value that suitable for your scenario.

Kind Regards

Jim

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events