topic Re: Heads-up: Management API Remote calls frequency limit in API / CLI Discussion

Heads-up: Management API Remote calls frequency limit

Dima_M — Tue, 21 Jul 2020 09:53:22 GMT

Hi all,

If you're working with remote scripts utilizing Management REST API, please read the below post.

To enable efficient work of Management REST API clients, starting R81 (expected release date – September 2020) we are going to limit the allowed frequency of the login command for remote API calls only, to 3 logins per admin per domain per minute.

Any request that will exceed this limit – will by failed by the server with a new error ("Too many requests in a given amount of time").To rule out possible impact, we recommend you to test your solutions which are based REST API once R81 Public EA is available for download.

Below you can find:

Guidelines for adapting your code to handle the new error
General best practices for login API

Please do not hesitate to contact us for further consultation.

Handling the new error in case too many login requests in a given time

Starting R81, login from a remote machine is limited to 3 logins per minute for each admin to a specific domain. Your scripts should catch the error below and try again later:

retries = 0 DO wait for (2^retries) seconds result = Do login operation. IF result.is_success = true retry = false ELSE IF result.is_success = false IF result.get_error_message = "Too many requests in a given amount of time" retry = true ELSE Some other error occurred, stop calling the API. retry = false END IF retries = retries + 1 WHILE (retry AND (retries < MAX_RETRIES))

General best practices for login API:

Login to last published session (enter-last-published-session) when you don't need to make any changes or updates. For example, if all commands are of type "show". This is because login for reading and writing has more database overhead than a login for read-only and other publishes will not affect your session.

Example 1(mgmt_cli):

mgmt_cli login user "aa" password "aaaa" enter-last-published-session true -f json

Example 2(Web Services):

POST {{server}}/login Content-Type: application/json { "user" : "aa", "password" : "aaaa" "enter-last-published-session" : "true" }

When using the mgmt_cli, reduce the number of logins and logouts to the minimum possible by using sessions, and working within one session..

Using one session is faster. If you do not explicitly use a session ID (sid), then each command results in this set of operations: login, action, publish, and logout. All these extra operations cause a higher management and database overhead.
Reducing the number of sessions helps you avoid reaching maximum allowed number of concurrent read/write sessions. The maximum is 100.

Bad Practice Example (pseudo code):
In this example, API call is being executed without an explicit session-id. This means that each time, four commands are being executed (login,add-host,publish and logout)

for i=0 to i<100 do: mgmt_cli -r true add-host name hosts_list[i] ip-address ip_list[i]

Good Practice Example (pseudo code):
In this example, login is done just once. All changes are made in one session, and at the end of the session there is a publish and logout. This saves the overhead of managing multiple login and logout operations on the server.

session=`mgmt_cli -r true login --format json| jq -r '.sid'` // login once and set session id (sid) into a variable for i=0 to i<100 do: mgmt_cli add-host name hosts_list[i] ip-address ip_list[i] --session-id $session // use the session id for adding hosts in loop mgmt_cli publish --session-id $session // publish all changes in one session. Publish occur only once mgmt_cli logout --session-id $session // logout once

Note- If you have many hundreds of changes, it is best to avoid publishing once at the end of the session. Instead, publish a few times within your session. For example, publish every 100 changes. However, you only need to log in once at the beginning of the session, and log out at the end of the session.

Re: Heads-up: Management API Remote calls frequency limit

nissimv — Mon, 05 Apr 2021 08:04:46 GMT

Hi Dima,

Thank you for sharing this valuable information.

I have R81 - Build 287 in my lab and I'm able to run API calls (using my application) with more then the above ratio you mentioned.

I made sure there is more then 3 logins per admin per domain per minute so In my lab this issue is not observed while one of my customers has the above described issue and he get's the following error

2021-03-26 11:29:23,417 ERROR com.checkpoint.management.web_api_is.exceptions.WebApiGeneralException.log:50 [qtp901620835-94] - Error message: [Too many requests in a given amount of time]

What can be the reason this happens only to my customer while according to this post this should be default REST configs for all new R81 installations ?

2021-04-05 10:33:43,040 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:44,403 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:47,892 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:49,146 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:49,243 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login
2021-04-05 10:33:50,380 DEBUG {R81} [com.skybox.view.agent.rest.client.generic.platform.EntityLoggingFilter] (EntityLoggingFilter.java:60) <@127.0.0.1:TASK:1> https://172.x.x.x/web_api/login

Re: Heads-up: Management API Remote calls frequency limit

nissimv — Sun, 11 Apr 2021 08:00:13 GMT

Hi,

Just realize that is working because I was running in MDS environment. This issue happens only with single management environment.

Re: Heads-up: Management API Remote calls frequency limit

Alexander_Wilke — Wed, 27 Apr 2022 06:47:09 GMT

Hello,

is this the correct command to disable api throttling?

api throttling off

Is this the correct file to increase the number of logins per time interval?

/opt/CPsuite-R81.10/fw1/api/conf/api-load-control.xml

Regards
Alex

Re: Heads-up: Management API Remote calls frequency limit

SimonMeadows — Tue, 28 Jun 2022 13:52:38 GMT

Hi,

I have used 'api throttling off' so I can run some pretty heavy logs requests against the api but have notice that throttling is re-enabled when the management software is patched or upgraded.

Does anyone know a way to maintain the throttling setting through a patch or upgrade?

Thanks

Simon

Re: Heads-up: Management API Remote calls frequency limit

PhoneBoy — Tue, 28 Jun 2022 17:38:03 GMT

Perhaps a regular cron job that executes this command to ensure its always turned off?
That said, it's not recommended to do this.

Re: Heads-up: Management API Remote calls frequency limit

Alexander_Wilke — Mon, 11 Jul 2022 08:38:31 GMT

Edit this file and turn it off. If you want to have it off the we do not have to care about any changes on this file in the future.
/opt/CPsuite-R81.10/fw1/api/conf/api-load-control.xml

Make the file immuteable. This prevents the system to modify the file.

chattr +i /opt/CPsuite-R81.10/fw1/api/conf/api-load-control.xml

Re: Heads-up: Management API Remote calls frequency limit

PhoneBoy — Tue, 12 Jul 2022 12:55:53 GMT

The main issue with doing that is that you also prevent other necessary changes from being made to this file on upgrade.
Which could create other issues down the road.

Re: Heads-up: Management API Remote calls frequency limit

Alexander_Wilke — Wed, 13 Jul 2022 05:33:03 GMT

Do not agree. This file is only for throttling opting. The main option on top is to turn it on or off. The other options a detailed options to throttle. So if you do not like throttling, you can disable it and other changes in the future are not relevant.

To be hones - the only reason for that throttling is that the MDS API is slow, is crashing and working much worse than other APIs. To circumvent these issue you (CheckPoint) just limited access and that's it.

Re: Heads-up: Management API Remote calls frequency limit

ias_gc-dk — Fri, 10 Feb 2023 14:10:10 GMT

I have made a python version, because we use the cpapi library to access the mgmt-api. Perhaps others can find a use for this:

from cpapi import APIClient, APIClientArgs from time import sleep import getpass warning_color = '\033[91m' #red reset_color = '\033[0m' #return to normal colour cp_username="testadmin" if sys.stdin.isatty(): cp_password = getpass.getpass("Enter password: ") apiserver="fwmgmt01.test" cp_domain="10.20.30.40" client_args = APIClientArgs(server=apiserver) with APIClient(client_args) as client: if client.check_fingerprint() is False: print("Could not get the server's fingerprint - Check connectivity with the server.") exit(1) # login to server: cp_login_retries = 0 cp_login_retry = True while cp_login_retries < 3 and cp_login_retry == True: login_res = client.login(cp_username, cp_password, domain=cp_domain) if login_res.success is True: cp_login_retry = False elif login_res.success is False: print(warning_color,"Login failed for domain "+cp_domain+":\n{}".format(login_res.error_message),reset_color) if login_res.data['code'] == "err_too_many_requests": print(f"Retrying {3 - cp_login_retries} time(s)") cp_login_retry = True for i in range(20, 0, -1): print(i, end= ' \r') sleep(1) cp_login_retries = cp_login_retries+1 else: print(warning_color,"Login failed for domain "+cp_domain+":\n{}".format(login_res.error_message),reset_color) cp_login_retry = False continue else: print("Login unsuccessful to",cp_domain) exit()

Note that this is for logging into a VSX domain. I suppose you can just omit the domain-parameter to log into a non-VSX platform.

After logging in, issue the calls to the API you want. Example:

show_vpn_star_response = client.api_call("show-vpn-communities-star",{}) print(show_vpn_star_response)

Re: Heads-up: Management API Remote calls frequency limit

oconnork — Fri, 13 Sep 2024 14:36:13 GMT

Hey everyone,

Does the change of the value 3 to 100 ( in the red rectangle) can cause any side effect over all the systems using the API ? For example the way that the request is done or the output done by the login command or anything else ?

I tried to look for documentation on this file and how It works but can't see anything.

Thanks in advance for your help,

Kalei

Re: Heads-up: Management API Remote calls frequency limit

PhoneBoy — Fri, 13 Sep 2024 14:57:46 GMT

What precise file are you're looking at?
From what I understand, these limits are in place to ensure the API server remains performant.
Changing these limits to higher values may cause performance issues and should only be done per explicit instructions from TAC.

Re: Heads-up: Management API Remote calls frequency limit

oconnork — Fri, 13 Sep 2024 15:09:29 GMT

the file is api-load-control.xml and actually need it for multiples usage for scripts..

For example, in MDS environment, if I want to connect to a domain, I will login on that domain, received the sid linked to this domain and do the stuff with this sid, but now if I want to do another action on another domain, I must login again and can't use the previous sid for that.

Thus, If we have then more than 3 domains, It's very limited the action you can do over all the domains...

=> Is there a way to use the same sid to connect to a different domain without using the API login again ?

Re: Heads-up: Management API Remote calls frequency limit

PhoneBoy — Fri, 13 Sep 2024 19:30:48 GMT

Looks like it's in $FWDIR/api/conf/api-load-control.conf.
To do something across multiple domains, you would have to serialize the requests (versus attempting to do them in parallel).

In any case, I don't believe it is possible to reuse a sid across multiple domains currently.
@Omer_Kleinstern any other ideas here?

Re: Heads-up: Management API Remote calls frequency limit

Alexander_Wilke — Fri, 13 Sep 2024 20:45:14 GMT

I believe CheckPoint can not provide performant REST API so to avoid tickets they created this throttling.

However - we are running this official command to disable the limit to make sure ource business processes are working.

api throttling off

Re: Heads-up: Management API Remote calls frequency limit

PhoneBoy — Fri, 13 Sep 2024 23:56:36 GMT

That sounds like a much better approach IMO than editing configuration files that aren't documented.

Re: Heads-up: Management API Remote calls frequency limit

JozkoMrkvicka — Sat, 14 Sep 2024 06:39:13 GMT

The session ID "sid" is unique and valid ONLY for one login call.

If you want to execute API commands on different domain, you need to login to that domain and get new "sid". If you want to execute any API calls from the domain for which you already logged in (but not logged out yet), you just need to specify "sid" relevant for that ONE domain. If you know your API calls will last longer than 10 minutes (default session timeout), then you can increase session duration by adding parameter "session-timeout" into initial login API call.

As an example, you have MDS with 3 domains. You dont know name, you dont know IP of these 3 domains. But you do know MDS IP so you can get data from MDS relevant for all 3 domains. You need to create 4 sessions as following:

1. login into MDS to get data from all 3 domains.

2. login into first domain and do whatever you need within this session

3.login into second domain and do whatever you need within this session

4. login into third domain and do whatever you need within this session

If you are done with your job on MDS or any of 3 domains, you need to publish the changes (if some were made) within that one session for specific domain, and logout using that one session for specific domain.

In short, you cannot expect that all shop's doors will be opened in your town if you approach one specific shop. You need to be physically present in front of that shop first.

Please also read API management Tips & Best Practises guide.

Re: Heads-up: Management API Remote calls frequency limit

oconnork — Mon, 16 Sep 2024 08:25:55 GMT

@PhoneBoy Well, my script is already serializing the request but It needs to go through like 9 domains in less than a minute to do some host deletion on them.

Simple usecase, removing a local host with a certain IP from all domains... I will need obviously to create 9 sessions to theses 9 domains and this will be done in less than one minute..

@JozkoMrkvicka I know how to login on different domains, the question was to avoid using multiples sid to get in multiples domains and by so avoiding to have more than 3 login api calls but looks not possible following all feedback here..

@Alexander_Wilke , I was more thinking of at least rising the number of allowed login api calls per minutes to the number of domains that I have - IMO It'll solve all login issue as soon as I can have at least one open session for each domains that I have within a single minute. Sounds more reasonable solution instead of disabling the whole thing.

But still, the file is not at all documented so I'm not confident on how to proceed and the possible outcome from doing that..

Re: Heads-up: Management API Remote calls frequency limit

PhoneBoy — Mon, 16 Sep 2024 15:11:56 GMT

Curious what's driving the requirement to do nine domains in under a minute.
Also curious if the script works when you disable the throttling with the CLI command?

Re: Heads-up: Management API Remote calls frequency limit

oconnork — Tue, 17 Sep 2024 06:36:23 GMT

It's not "required" to run the script under a minute but will for sure be way faster than waiting 1 minute every 3 domains crossed... And actually I don't have 9 domain but 22 domains!! So to cross all of them I'll need 22/3 = 8 minutes in total to go through all domains...

Didn't tried yet to disable the api throttling as i don't have any documentation on this command either.. Where Is It possible to have information on this ? Impact ? Need to restart the MDM ? Need to restart the api ? anything else to know about this command ?