Heads-up: Management API Remote calls frequency limit

[Dima_M]

Hi all,

If you're working with remote scripts utilizing Management REST API, please read the below post.

To enable efficient work of Management REST API clients, starting R81 (expected release date – September 2020)  we are going to limit the allowed frequency of the login command for remote API calls only, to 3 logins per admin per domain per minute.

Any request that will exceed this limit – will by failed by the server with a new error ("Too many requests in a given amount of time").To rule out possible impact, we recommend you to test your solutions which are based REST API once R81 Public EA is available for download.

Below you can find:

1. Guidelines for adapting your code to handle the new error
2. General best practices for login API

Please do not hesitate to contact us for further consultation.

Handling the new error in case too many login requests in a given time

Starting  R81, login from a remote machine is limited to 3 logins per minute for each admin to a specific domain. Your scripts should catch the error below and try again later:

retries = 0

DO

    wait for (2^retries) seconds

    result = Do login operation.

    IF result.is_success = true

        retry = false

    ELSE IF result.is_success = false

       IF result.get_error_message = "Too many requests in a given amount of time"

                  retry = true

      ELSE

                  Some other error occurred, stop calling the API.

                  retry = false

    END IF

    retries = retries + 1

WHILE (retry AND (retries < MAX_RETRIES))

General best practices for login API:

• Login to last published session (enter-last-published-session) when you don't need to make any changes or updates. For example, if all commands are of type "show". This is because login for reading and writing has more database overhead than a login for read-only and other publishes will not affect your session.

Example 1(mgmt_cli):

mgmt_cli login user "aa" password "aaaa" enter-last-published-session true -f json

Example 2(Web Services):

POST {{server}}/login

Content-Type: application/json



{

  "user" : "aa",

  "password" : "aaaa"

  "enter-last-published-session" : "true"

}

• When using the mgmt_cli, reduce the number of logins and logouts to the minimum possible by using sessions, and working within one session..
• Using one session is faster. If you do not explicitly use a session ID (sid), then each command results in this set of operations: login, action, publish, and logout. All these extra operations cause a higher management and database overhead.
• Reducing the number of sessions helps you avoid reaching maximum allowed number of concurrent read/write sessions. The maximum is 100.

Bad Practice Example (pseudo code):
In this example, API call is being executed without an explicit session-id. This means that each time, four commands are being executed (login,add-host,publish and logout)

for i=0 to i<100 do:

         mgmt_cli -r true add-host name hosts_list[i] ip-address ip_list[i]

Good Practice Example (pseudo code):
In this example, login is done just once. All changes are made in one session, and at the end of the session there is a publish and logout. This saves the overhead of managing multiple login and logout operations on the server.

session=`mgmt_cli -r true login --format json| jq -r '.sid'` //  login once and set session id (sid) into a variable

for i=0 to i<100  do:

   mgmt_cli add-host name hosts_list[i] ip-address ip_list[i]  --session-id $session // use the session id for adding hosts in loop

mgmt_cli publish --session-id $session  // publish all changes in one session. Publish occur only once

mgmt_cli logout --session-id $session  // logout once

Note- If you have many hundreds of changes, it is best to avoid publishing once at the end of the session. Instead, publish a few times within your session. For example, publish every 100 changes. However, you only need to log in once at the beginning of the session, and log out at the end of the session.

[nissimv]

Hi Dima,

Thank you for sharing this valuable information. 

I have R81 - Build 287 in my lab and I'm able to run API calls (using my application) with more then the above ratio you mentioned.

I made sure there is more then 3 logins per admin per domain per minute so In my lab this issue is not observed while one of my customers has the above described issue and he get's the following error

2021-03-26 11:29:23,417 ERROR com.checkpoint.management.web_api_is.exceptions.WebApiGeneralException.log:50 [qtp901620835-94] - Error message: [Too many requests in a given amount of time]

What can be the reason this happens only to my customer while according to this post this should be default REST configs for all new R81 installations ? 

2021-04-05 10:33:43,040 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:44,403 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:47,892 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:49,146 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:49,243 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:50,380 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login

[nissimv]

Hi,

Just realize that is working because I was running in MDS environment. This issue happens only with single management environment.