Couple Of Errors After R77.30 to R80.10 Gateway Upgrade

[Daniel_Taney]

I just wrapped up an R80.10 upgrade of a 15400 Cluster that was previously running R77.30. The Upgrade method was doing an in-place CPUSE upgrade to R80.10. Then, we jumped to the latest HFA (Take 121) and the SegmentSmack / FragmentSmack hot fixes. 

Since the upgrade, I've been noticing a lot of these errors in /var/log/messages:

kernel: [fw4_1];[ERROR]: ida_cmi_hold_conn: idapi_fetch_identity_async failed, missing vtable information

kernel: [fw4_1];[ERROR]: ida_cmi_handle_late_contexts: error while fetching identity, notified empty CLOB
kernel: [fw4_1];[ERROR]: cmik_loader_fw_context_match_cb: match_cb for CMI APP 19 failed on context 360, executing context 366 and adding the app to apps in exception

kernel: [fw4_7];[ERROR]: ida_cmi_async_fetch_headers_done_cb: fetched src IP 10.12.0.194 from conn

kernel: [fw4_7];[ERROR]: ida_cmi_async_fetch_headers_done_cb: failed to fetch identity roles from handle

I couldn't find any relevant SK articles, or other Check Mates posts, that reference these errors. Before opening a case with TAC; Has anyone seen these before? Based on the first block of errors, I'm guessing that could have something to do with App Control or URL Filtering? I'm guessing the second one is probably related to Identity Awareness? No one has voiced any complaints or issues with things not working. I just don't like seeing logs piling up with unexplained errors!

This Gateway is running App Control / URLF, IPS, AB/AV, TE (Cloud Emulated), and Identity Awareness blades. The Identity Awareness blade is configured for Identity Sharing and collects Identity data from a separate, dedicated, PDP Gateway elsewhere on the network. 

I'm fine with opening a TAC case, but I'd rather save the time if someone here has seen these types of errors before. 

Thanks!

-Dan

[PhoneBoy · Accepted solution]

It looks like there are a couple of different errors here.

One of them is related to App Control and/or IPS signatures and it would be a good idea to do a manual update of these signatures and install policy to see if these clear.

The other errors are related to Identity Awareness, and you'll probably need to open a TAC case to troubleshoot this.

[Daniel_Taney]

It looks like the manual update / policy install may have resolved the App Control / IPS errors. I'll keep an eye on my logs and see if they come back.

I'll reach out to TAC for the other IA issue. 

Thanks!

Dan

[Liel_Shaish]

Hi Daniel,

Following IDA error messages ,IDA is waiting for XFF header on TCP traffic which is not http. I suggest to review XFF settings:

On an Identity Awareness Gateway : consider define Trust X-Forwarded-For from known proxies only.

On the Access Control Policy Layer : please make sure that ‘Detect users located behind http proxy configured with X-Forwarded-For’ is enabled for the relevant policy layer only.

For more info you can review Identity Awareness R80.10  Administration Guide , Identifying Users Behind an HTTP Proxy Server (Pages 42-43)

Thanks,

Liel Shaish | Team Leader, Identity Awareness R&D

[Daniel_Taney]

I think I spoke too soon! The errors came back a few hours later. 

Liel: I started digging into this and it looks like we have XFF enabled on a number of policy layers. We don't have any proxy servers in place that require collection of Identity data. So, I am just going to go ahead and disable this on these layers since it is not needed. 

I have opened a TAC case in parallel in case there is another issue going on.

Thanks for your input.

-Dan