API. How to get all objects in one request

[Dmitriy2]

Hello.

I am looking for a way to get all objects from the Smart-1 management server (for example, all hosts and networks, all rules, etc.) At the moment, the api allows you to get only 500 items per request, which can be thousands. Is there a way to get all the objects?

Moreover, when I'm start manipulating the offset parameter increasing it value by 500 with each subsequent request and the database contains 510 hosts, then the second request with offset=500 will return 500 hosts instead of 10. So I can't get all the objects even in a loop. 

Here is an example of the code I use to get hosts. 

def cp_api_call(command, payload, cp_session):
	if cp_session == '':
		headers = {'Content-Type': 'application/json'}
	else:
		headers = {'Content-Type': 'application/json', 'X-chkp-sid': cp_session}
	try:
		r = requests.post(cp_url + command, json = payload, headers = headers, verify = False)
		code = r.status_code
		if code == 200:
			return r.json()
		else:
			print(r.json())
	except ConnectionError:
		print('ERROR: Connection error, retry after 10 seconds.')
		sleep(10)
		r = requests.post(cp_url + command, json = payload, headers = headers, verify = False)
		return r.json()

def cp_show_hosts(cp_session):
	payload = {
		'limit' : 500,
		'offset' : 0,
		'details-level' : 'standard'
	}
	r = cp_api_call('show-hosts', payload, cp_session)

[Bob_Zimmerman]

There's not a way to. With Check Point's APIs, the maximum number of things you can get from a single API request is 500.

If there are 510 hosts and you send a request with limit 500, offset 500, it definitely does not give you 500 results back on any version I've tested. It would only give you the 10 above the 500 offset.

[PhoneBoy · Accepted solution]

You can attempt to set the limit higher but we do not guarantee the API will return all the requested results if you do so.
Multiple calls with limit/offset is the correct approach.
Your code sample above does not appear to increment offset with each call, which is required here.

[Dmitriy2]

Apparently I had an error in the script with the loop earlier. Now I've done it again and everything works, thank you!

def cp_show_hosts(cp_session, offset):
	payload = {
		'limit' : 500,
		'offset' : offset,
		'details-level' : 'standard'
	}
	return cp_api_call('show-hosts', payload, cp_session)

offset = 0

while True:
	cp_hosts = cp_show_hosts(cp_session, offset)
	hosts_from = cp_hosts['from']
	hosts_to = cp_hosts['to']
	hosts_total = cp_hosts['total']
	print('-'*80)
	print(f'total: {hosts_total} offset: {offset} from: {hosts_from} to: {hosts_to}')

	if hosts_to >= hosts_total:
		break
	else:
		offset += 500

Script output:

-----------------------------------------------
total: 3459 offset: 0 from: 1 to: 500
-----------------------------------------------
total: 3459 offset: 500 from: 501 to: 1000
-----------------------------------------------
total: 3459 offset: 1000 from: 1001 to: 1500
-----------------------------------------------
total: 3459 offset: 1500 from: 1501 to: 2000
-----------------------------------------------
total: 3459 offset: 2000 from: 2001 to: 2500
-----------------------------------------------
total: 3459 offset: 2500 from: 2501 to: 3000
-----------------------------------------------
total: 3459 offset: 3000 from: 3001 to: 3459
CheckPoint logout result: {'message': 'OK'}

[Dmitriy2]

Apparently I had an error in the loop code. I redid the script and now all objects are unloaded, thank you!

def cp_show_hosts(cp_session, offset):
	payload = {
		'limit' : 500,
		'offset' : offset,
		'details-level' : 'standard'
	}
	return cp_api_call('show-hosts', payload, cp_session)

offset = 0

while True:
	cp_hosts = cp_show_hosts(cp_session, offset)
	hosts_from = cp_hosts['from']
	hosts_to = cp_hosts['to']
	hosts_total = cp_hosts['total']
	print('-'*80)
	print(f'total: {hosts_total} offset: {offset} from: {hosts_from} to: {hosts_to}')

	if hosts_to >= hosts_total:
		break
	else:
		offset += 500

Script output:

-----------------------------------------------
total: 3459 offset: 0 from: 1 to: 500
-----------------------------------------------
total: 3459 offset: 500 from: 501 to: 1000
-----------------------------------------------
total: 3459 offset: 1000 from: 1001 to: 1500
-----------------------------------------------
total: 3459 offset: 1500 from: 1501 to: 2000
-----------------------------------------------
total: 3459 offset: 2000 from: 2001 to: 2500
-----------------------------------------------
total: 3459 offset: 2500 from: 2501 to: 3000
-----------------------------------------------
total: 3459 offset: 3000 from: 3001 to: 3459
CheckPoint logout result: {'message': 'OK'}