Optimized rule addition via API/ Need help regarding CMA Filter | R80.40/API 1.7.1

[mahendra]

Hi. 

I am trying to add rules into domains, having rules >10k via API.

I am adding in way so that needless creation of new rules in minimized/eliminated.

For this, i am using filters based on input source/destination/service. as rulebase if quite big.

Also, checking each rule would take a lot of time, "show-as-ranges" gives only 20 results in about 20-30 seconds and in case of without "show-as-ranges", we will need to make multiple nested queries for each rule to resolve a "uid"

Now, problem is that filter opion which used CMA filter is not very accurate. It works only in few cases. I have checked and verified that directly in smartdomain, it does not filter accurately.

Someone please guide me how should i proceed.

[PhoneBoy]

For a rulebase above 10k rules, you might want to do an optimization exercise with Professional Services.

Without knowing exactly what you’re doing at a code level, it’s difficult to tell you how to proceed.
Precise examples of calls and outputs (both expected and actual) will help.

To keep the API server performant, API calls will return only a limited amount of results by design.
Pretty much every API produced by every vendor has similar limits in place for the same reason.
API performance is better in the most recent versions (R81.10 and above).

Even so, it may be easier to download the necessary data from the API offline and worth with it that way.
You can capture the JSON data and feed the data to jq, which can query the captured data locally.

[mahendra]

ok. first of all, its a big organization with tens of thousands of application with different requirements.

However, you did not answer my query.

my problem is that filter option does not work very well, or is there a way to make it function more accurately.

if that happens, I will be able to handle it very well.

loading rulebase or querying loaded rulebase for every rule addition request is certainly not very feasible option as we receive 100+  requests daily.

Can you please help me on how to use filter more accurately and efficiently.

[PhoneBoy]

Again, without knowing exactly what you’re doing at a code level, it’s difficult to tell you how to proceed.
Please provide detailed examples of the calls you're making and the results you're getting from them.

[mahendra]

this is what i am trying to do:

there are three inputs: for source, detination and service.

based on inputs, filter can become like "src:10.10.10.10 AND dst:20.20.20.20 AND svc:8080"

This type of filter sometimes work and sometimes it doesn't (i am not talking about it working in code but smartdomain). it sometimes gives no results even if there re many rules satisfying thiis condition.

Again, i repeat, i am not talking about filter not working in code, but this type of filter sometimes doesn't work in smartdomain  (on entring filter manually in search bar of smartdomain). 

[PhoneBoy]

If your ultimate question wasn't related to the API/CLI, but rather why SmartConsole search is returning unexpected results, why not post it to the Management space?

If you have specific queries that you feel aren't working correctly, then I recommend a TAC case: https://help.checkpoint.com 
However, given you're on R80.40, I suggest upgrading to R81.10 or later since CMA searches should be a bit faster due to underlying infrastructure changes.