topic Re: Intermittent error for /web_api/set-threat-protection in API / CLI Discussion

Intermittent error for /web_api/set-threat-protection

jmcadams — Wed, 01 Apr 2020 16:03:04 GMT

I'm hitting

/web_api/set-threat-protection

with

{
"uid": "3ca88a50-bc75-a844-9bcb-fede11222822",
"overrides": {
"profile": "Optimized",
"action": "Prevent"
}
}

And getting this on and off.

500 Server Error: Server Error for url: https://x.x.x.x:443/web_api/set-threat-protection
{
"code" : "generic_server_error",
"message" : "Failure occurred when handling overrides"
}

Any ideas what would cause that? I'm on R80.30 if that helps.

Re: Intermittent error for /web_api/set-threat-protection

PhoneBoy — Wed, 01 Apr 2020 16:25:03 GMT

Need to understand the entire flow of how you are using the API and where these commands fit in.
More specifically, how many add/set type commands are done prior to a publish action.
In general, you should periodically publish your session (every few hundred add/set actions) for performance reasons.
Depending on your API usage, the server might require tuning, which I believe the TAC can assist with.

Re: Intermittent error for /web_api/set-threat-protection

jmcadams — Wed, 01 Apr 2020 16:37:00 GMT

The flow is this:
Get a CVE (CVE-2009-1234)
Search for the CVE in all Threat Protections*
Find the relevant Threat Protection
Discard All Sessions (Optional, depends on if the server is being odd)
Logout
Get new SID
Make the above call

* I think this is likely murdering the server. I'm calling in and pulling all the Threat Protections in bunches of 100 to get their Industry Reference field. I'm on 80.30 and filter isn't available yet, and I didn't see a way to just pull the Name, UID, and industry reference in 80.30. I tried doing this with the max limit of 500 and the server straight up crashed on me.

I thought logging out and logging in might solve that problem, but it doesn't seem like it.

I'm not sure what else I could do to make this more stable. I guess I could pull the Threat Protections in smaller batches and maybe add delays to help with some of the load on the server.

Re: Intermittent error for /web_api/set-threat-protection

Timothy_Hall — Wed, 01 Apr 2020 18:10:07 GMT

You may indeed be running the api server out of memory with that call, check out sk119553: Security Management API server crashes under heavy load

Re: Intermittent error for /web_api/set-threat-protection

jmcadams — Wed, 01 Apr 2020 19:14:50 GMT

Yea, I pretty sure that's the answer. Thanks for the help!!!

Re: Intermittent error for /web_api/set-threat-protection

PhoneBoy — Wed, 01 Apr 2020 19:45:45 GMT

What I recommend you do instead is work with the results of show-threat-protections offline so you're not hitting the API server so hard.
You can then use scripting and jq to parse the results from the files, which will most likely be significantly faster.

In the below examples, I am logged into the management server in expert mode.
To create a reusable API session for my successive API calls, I did:

mgmt_cli -r true login > sid.txt

You can find out the default number of results returned by the API call and the total number you have to parse through using

mgmt_cli -s sid.txt show threat-protections --format json | jq '{limit: .to, total: .total}'

You can then generate successive calls like the following to get all the results into a single file (advancing the offset for each call):

mgmt_cli -s sid.txt show threat-protections details-level full offset 50 --format json | jq '.protections[] | {uid: .uid, name: .name, "industry-reference": ."industry-reference"}' >> ips.json

Once you've got all the results in a single file, log out:

mgmt_cli -s sid logout

Once you've got all the results in a single file, you can use jq to query against those results and get only the uid of the protection you need to modify:

cat ips.json | jq -r 'select (."industry-reference"[]? == "CVE-2020-3807") | .uid'

Obviously, all of this can be scripted.