Content awareness problem

[the_rock]

Hey guys,

Wondering if anyone may have some experience with this blade, more specifically, getting files to be blocked when its enabled 🙂

So, essentially, we got this working few years ago, but customer decided back then not to use the blade and they would like to do it at this point. Issue is literally the same like the post I had back in the day.

https://community.checkpoint.com/t5/Security-Gateways/Content-awareness-issue/m-p/156026/emcs_t/S2h8...

R81.20 jumbo 98

We did exact same steps esc. engineer gave us and no luck, we dont even see any logs for the blade at all when exe files is downloaded.

Before doing any debugs from below, wondering if anyone may have any other ideas/suggestions. I will also open TAC case to see what they say.

https://support.checkpoint.com/results/sk/sk119715

Thanks as always and happy weekend 🙂

Andy

[the_rock]

To add to this, when I created rule from the screenshot for content awareness exe block, it turns out that actually causes my windows update in the lab PC to fail...as soon as I disabled it, no issues.

Andy

[the_rock]

Another test I tried was add rule in content awar. layer to allow access to windows updates sites, so update now works and funny enough, after I pushed policy, exe got blocked once and then never after that, even after I rebooted.

Anyway, lets see what TAC says in remote Tuesday, March 4th.

Andy

[the_rock]

Hey guys,

Quick update, will have remote with TAC shortly, lets see if there is any progress and will update once done.

Andy

[the_rock]

We had call with TAC, but still unable to fix this in the lab. Guy said would reavh out to senior folks and let us know the next steps.

Andy

[the_rock]

Hey guys,

Just a quick update. We had remote with TAC and still no progress. They asked us to change some AV settings in TP profile, but no luck. We also ended up disabling bypass rule in ssl inspection policy and that did not work either.

Im really confused at this point what else to do. If anyone has any idea, please be free to chime in.

Tx

Andy

[the_rock]

Hey everyone,

Just something else I wanted to share, if anyone has an idea if maybe rule I have is wrong? I tried with default services, different applications, no joy no matter what I try, it NEVER hits the rule.

Andy

[the_rock]

Latest update. After doing the quick debug, I see below messages, which to me makes no logical sense, since blade is 100% enabled.

Maybe you smart guys @AkosBakos @PhoneBoy @Timothy_Hall can give some suggestions? 🙂

Best,

Andy

*************************

@;95092703.42; 7Mar2025 15:19:28.163024;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.43; 7Mar2025 15:19:28.163025;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.44; 7Mar2025 15:19:28.163027;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.45; 7Mar2025 15:19:28.163029;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.46; 7Mar2025 15:19:28.163033;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_category_sort_callback: Content Awreness not enabled;
@;95092703.47; 7Mar2025 15:19:28.163035;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.48; 7Mar2025 15:19:28.163036;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.49; 7Mar2025 15:19:28.163038;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.50; 7Mar2025 15:19:28.163040;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.51; 7Mar2025 15:19:28.163043;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_category_sort_callback: Content Awreness not enabled;
@;95092703.52; 7Mar2025 15:19:28.163046;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.53; 7Mar2025 15:19:28.163049;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.54; 7Mar2025 15:19:28.163052;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
@;95092703.55; 7Mar2025 15:19:28.163055;[kern];[tid_2];[fw4_0];1:{engine} dlpd
a_stats_get_table_rows_number_callback: Content Awreness not enabled;
[Expert@CP-FW-01:0]# enabled_blades
fw vpn urlf appi identityServer SSL_INSPECT content_awareness qos mon
[Expert@CP-FW-01:0]# cphaprob roles

ID Role

1 (local) Master
2 Non-Master

[Expert@CP-FW-01:0]#

[Timothy_Hall]

Any chance this traffic has been fact_accel'ed?  That will keep Content Awareness from working on it even though it is called for in the policy.  Next I would establish what path this problematic traffic is being handled in, please post the filtered output of fwaccel conns showing just the flags for the live connection in question that should be getting scanned by Content Awareness.  

[the_rock]

First thing I did was disable sxl, but had not tried that after installing newest jumbo, will test tomorrow.

Andy

[the_rock]

Same problem even when sxl is off. See output from command you gave.

Andy

dst ip is 142.251.40.142

[Expert@CP-FW-01:0]# fwaccel conns
Source SPort Destination DPort PR Flags TCP state C2S i/f S2C i/f Inst Policy ID (FW/UP) CPU Host Pkts Host Bytes Last Seen Duration TTL/Timeout
--------------- ----- --------------- ----- -- --------------------- ---------------- ------- ------- ---- --------------------- --- ----------- ----------- ---------- ---------- -----------
142.251.163.188 5228 172.16.10.246 10404 6 ..NA..S....L......... Established 1/1 1/1 4 3987324519/1741399108 0 13.99K 1000.10KB 42s 81h33m12s 3558/3600
9.9.9.9 53 172.16.10.246 18916 17 ..NA..S....L......... No State 1/1 1/1 4 3987324519/1741399108 0 2 214B 4s 4s 36/40
172.16.10.177 57119 142.251.163.188 5228 6 ..NA..S.............. Established 1/1 1/1 4 3987324519/1741399108 0 13.99K 1000.10KB 42s 81h33m12s 3558/3600
107.167.110.211 443 172.16.10.177 53509 6 ..NA..S....L......... Established 1/1 1/1 3 3987324519/1741399108 0 56 9.82KB 5s 27m25s 3595/3600
149.112.121.10 443 172.16.10.246 10412 6 ..NA..S....L......... Established 1/1 1/1 4 3987324519/1741399108 0 18.63K 2.14MB 26s 55h51m14s 3574/3600
107.167.110.211 443 172.16.10.246 10403 6 ..NA..S....L......... Established 1/1 1/1 3 3987324519/1741399108 0 56 9.82KB 5s 27m25s 3595/3600
172.16.10.177 61257 9.9.9.9 53 17 ..NA..S.............. No State 1/1 1/1 5 3987324519/1741399108 0 2 255B 4s 4s 36/40
107.167.110.216 443 172.16.10.246 10400 6 ..NA..S....L......... Established 1/1 1/1 0 3987324519/1741399108 0 81 40.20KB 21s 27m42s 3579/3600
107.167.96.30 443 172.16.10.246 10400 6 ..NA..S....L......... Established 1/1 1/1 5 3987324519/1741399108 0 30 8.14KB 34s 6m55s 3566/3600
9.9.9.9 53 172.16.10.246 25933 17 ..NA..S....L......... No State 1/1 1/1 5 3987324519/1741399108 0 2 255B 4s 4s 36/40
9.9.9.9 53 172.16.10.177 61257 17 ..NA..S....L......... No State 1/1 1/1 5 3987324519/1741399108 0 2 255B 4s 4s 36/40
142.251.167.188 5228 172.16.10.177 58626 6 ..NA..S....L......... Established 1/1 1/1 4 3987324519/1741399108 0 9.96K 755.14KB 21s 57h27m28s 3579/3600
172.16.10.177 57101 107.167.96.30 443 6 ..NA..S.............. Established 1/1 1/1 5 3987324519/1741399108 0 30 8.14KB 34s 6m55s 3566/3600
107.167.110.216 443 172.16.10.177 53456 6 ..NA..S....L......... Established 1/1 1/1 0 3987324519/1741399108 0 81 40.20KB 21s 27m42s 3579/3600
149.112.121.10 443 172.16.10.177 59207 6 ..NA..S....L......... Established 1/1 1/1 4 3987324519/1741399108 0 18.63K 2.14MB 26s 55h51m14s 3574/3600
142.251.163.188 5228 172.16.10.177 57119 6 ..NA..S....L......... Established 1/1 1/1 4 3987324519/1741399108 0 13.99K 1000.10KB 42s 81h33m12s 3558/3600
172.16.10.177 59095 31.209.137.47 61613 6 ..NA..S.............. Established 1/1 1/1 5 3987324519/1741399108 0 5.58K 386.79KB 0s 7h42m26s 3600/3600
172.16.10.177 58626 142.251.167.188 5228 6 ..NA..S.............. Established 1/1 1/1 4 3987324519/1741399108 0 9.96K 755.14KB 21s 57h27m28s 3579/3600
107.167.96.30 443 172.16.10.177 57101 6 ..NA..S....L......... Established 1/1 1/1 5 3987324519/1741399108 0 30 8.14KB 34s 6m55s 3566/3600
31.209.137.47 61613 172.16.10.246 10400 6 ..NA..S....L......... Established 1/1 1/1 5 3987324519/1741399108 0 5.58K 386.79KB 0s 7h42m26s 3600/3600
31.209.137.47 61613 172.16.10.177 59095 6 ..NA..S....L......... Established 1/1 1/1 5 3987324519/1741399108 0 5.58K 386.79KB 0s 7h42m26s 3600/3600
172.16.10.177 63193 9.9.9.9 53 17 ..NA..S.............. No State 1/1 1/1 4 3987324519/1741399108 0 2 214B 4s 4s 36/40
172.16.10.177 53456 107.167.110.216 443 6 ..NA..S.............. Established 1/1 1/1 0 3987324519/1741399108 0 81 40.20KB 21s 27m42s 3579/3600
172.16.10.177 59207 149.112.121.10 443 6 ..NA..S.............. Established 1/1 1/1 4 3987324519/1741399108 0 18.63K 2.14MB 26s 55h51m14s 3574/3600
9.9.9.9 53 172.16.10.177 63193 17 ..NA..S....L......... No State 1/1 1/1 4 3987324519/1741399108 0 2 214B 4s 4s 36/40
172.16.10.177 53509 107.167.110.211 443 6 ..NA..S.............. Established 1/1 1/1 3 3987324519/1741399108 0 56 9.82KB 5s 27m25s 3595/3600
142.251.167.188 5228 172.16.10.246 10400 6 ..NA..S....L......... Established 1/1 1/1 4 3987324519/1741399108 0 9.96K 755.14KB 21s 57h27m28s 3579/3600

Idx Interface
--- ---------
0 lo
1 eth0
2 eth1
3 eth2
4 eth3

Total number of connections: 9
Total number of links: 18

[emmap]

What do you see in the allow logs for the connections you are trying to block? It would help to enabled Extended Logging on the rule the connection is being accepted on so we can see more detail. 

You're blocking QUIC traffic while testing this, yes?

[the_rock]

Hey Emma,

Yes, quic is blocked. We do have extended logging enabled, but it gives us exact same log info. When ssl bypass rule is active, shows its bypassed, but when we disable it, content awareness block rule never gets hit. I asked TAC if they can replicate this in their lab, because I feel like customer and I tried literally everything we can think of 🙂

Andy

[emmap]

OK but you're seeing an SSL inspected log of the traffic with the exe file being accepted that's matching a lower rule than the block rule?

[the_rock]

Well, I see it being accepted on allow rule at the bottom of that layer.

Andy

[emmap]

Do you have Extended Logging on that rule? That will enable appc/urlf/content logging on that traffic which may open more useful detail here.

[the_rock]

I do, but see, the problem is that sadly does not do much (if anything), since the rule never gets hit : - (

Andy

[emmap]

The rule that the connection is being accepted on isn't being hit?

[the_rock]

No no, the rule thats supposed to block exe files is not being hit at all. I enabled extended logging on last rule in that layer, also rule in what I call "final layer", one below content awareness layer, but I dont get any additional info.

Here is what I will try shortly. Since it does let me copy ssl inspection bypass rule to another layer, let me take a screenshot and manually create a rule like that in app/urlf layer and test. Thats how we made it work 3 years ago when we worked with escalation engineer, but its possible I missed something 2 weeks ago when I tried it the first time.

Andy

[the_rock]

@emmap Just copied the bypass rule to url layer, which is how we fixed it 3 years ago, no dice. 

Andy

[Timothy_Hall]

The connections involving 142.251.40.142 are not shown at all in that output, so those connections are slowpath.  Content Awareness should be able to be enforced in that path and not just the Medium Path.

[Timothy_Hall]

Also looks like you can't match applications and Content Awareness types simultaneously in the same first ordered layer (or in the top/parent inline layer), see here: sk180116: Rule with Application Control and Content Awareness is not matched

[the_rock]

We tried layer with only content awareness on, same result.

Andy

[Timothy_Hall]

Two more things:

1) Verify default settings are configured on the Blade properties for Content Awareness, then try setting "fail-close" and see what happens, perhaps the blade is having some kind of problem and just letting it through, fail-close will block everything subject to Content Awareness if this is the case:

2) Try setting an explicit override for the file type you are trying to detect via the dlpda_file_family_mapping_override.C file, maybe that will give it the kick it needs to work properly:

sk114954: How to configure actions for a specific file type in R80.10 Content Awareness blade

[the_rock]

Yup, tried both last week, same problem.

Andy

[Timothy_Hall]

Strange, sounds like Content Awareness isn't working at all.  Checking that the Unified Policy was generated correctly will be needed, then a kernel debug on the gateway to figure out what is going on.

[the_rock]

Yep, that sounds right. Funny enough, when I ran debug Friday, it was full of messages that content awareness blade is not enabled. I think at this point, lets wait for TAC to provide next steps 🙂

Thanks Tim!

Andy

[the_rock]

Hey Tim,

Just FYI, I chatted with person online on support site and they confirmed that sk114954 has not been applicable since R80.10 anyway, though I tried it last week, but did not change the situation.

I will update the post once we hear back from TAC on next steps.

Tx again for everything.

Andy

[the_rock]

For what its worth, here is the video I took from the lab. Just FYI, its EXACT same result even if bypass ssl inspection rule is deleted/disabled and only any any inspect is on. My gut feeling is that this issue is caused by https inspecdtion, I just cant figure out which part exactly...

Andy

[the_rock]

We have a call March 14th with T3 TAC engineer, so will update afterwards.

Andy