Custom Intelligence Feed Exceptions Not Working

[Mike_Jensen]

I am running R81.10 with GA JHA take 130 installed on my gateways and SMS.

I have a External Custom Intelligence Feed named Talos_blacklist configured on my gateway cluster via the CLI.

I have a IP that is on that blacklist that gets by design of the feature.

However I need to make an exception for this IP address and everything I have tried in SmartConsole does not work, traffic to the IP in question is still dropped due to my Talos_blacklist.

In the example with my screen shots the source is 10.1.1.10 > 185.242.113.224 (black listed IP).  From the log card I have selected add an exception, select the defaults, the exception is created (see screen shot), I install threat prevention policy and traffic is still blocked from my source to the destination due to the Talos_blacklist.

I have also tried creating my own threat prevention rule and assigning the source and/or destination to a dummy no threat prevention policy that doesn't have any TP enabled and that does not work as well.

Is it possible to make exceptions for IP's on External Custom Intelligence Feed's and if so how can I create one that will work?

Thank you in advance.

[the_rock]

I think protected scope should be any and source should be 10.x.x.x

Andy

[Mike_Jensen]

The exception doesn't work that way either.  It's almost as if exceptions through SmartConsole aren't support with Custom Intelligence Feed's but I can't find any documentation stating that or showing a different way to do an them.

[the_rock]

I see what you mean. Any difference if you leave protection/site as n/a?

Andy

[Mike_Jensen]

Good idea.  I tried with the protection / site set to N/A.  When I test traffic again no logs are generated but running zdebug I see the following:

[Expert@SG-1:0]# fw ctl zdebug + drop | grep 185.242.113.224
@;4184305;[kern];[tid_0];[SIM-241965305];pkt_handle_stateless_checks: dropping d ue to dos_pkt_should_drop(), conn: <10.1.1.10,1,185.242.113.224,0,1>;
@;4184305;[kern];[tid_0];[SIM-241965305];do_packet_finish: SIMPKT_IN_DROP vsid=0 , conn:<10.1.1.10,1,185.242.113.224,0,1>;
@;4185381;[kern];[tid_0];[SIM-241965305];pkt_handle_stateless_checks: dropping due to dos_pkt_should_drop(), conn: <10.1.1.10,1,185.242.113.224,0,1>;
@;4185381;[kern];[tid_0];[SIM-241965305];do_packet_finish: SIMPKT_IN_DROP vsid=0, conn:<10.1.1.10,1,185.242.113.224,0,1>;
@;4186459;[kern];[tid_0];[SIM-241965305];pkt_handle_stateless_checks: dropping due to dos_pkt_should_drop(), conn: <10.1.1.10,1,185.242.113.224,0,1>;
@;4186459;[kern];[tid_0];[SIM-241965305];do_packet_finish: SIMPKT_IN_DROP vsid=0, conn:<10.1.1.10,1,185.242.113.224,0,1>;
@;4187515;[kern];[tid_0];[SIM-241965305];pkt_handle_stateless_checks: dropping due to dos_pkt_should_drop(), conn: <10.1.1.10,1,185.242.113.224,0,1>;
@;4187515;[kern];[tid_0];[SIM-241965305];do_packet_finish: SIMPKT_IN_DROP vsid=0, conn:<10.1.1.10,1,185.242.113.224,0,1>;

[the_rock]

Seems to me that pings are dropped based on that...

Andy

[Mike_Jensen]

If I delete the exception the pings are then dropped by the blacklist again.  If I turn of the custom intelligence feed the pings work as expected. 

[the_rock]

Not sure, sorry...maybe worth TAC case to verify. Please share the solution once its fixed.

Best,

Andy

[Mike_Jensen]

Will do.  In the meantime if someone from Check Point can chime in and let me know if this is even supported or not I would really appreciate it.

[the_rock]

Im curious too, for sure. In the meantime, I will keep digging to see if there is a way...

Andy

[Timothy_Hall]

The zdebug shows that the drop is occurring in the sim/SecureXL module even though Threat Prevention is responsible, which is normally enforced by a Firewall Worker Instance in the Medium Path at least.  This is a little-known optimization to drop traffic solely by IP address at a very early stage of processing inside sim, even though Threat Prevention is normally enforced after Access Control.  Custom Observables can be complex and based on many things other than header values such as IP addresses and ports (which is all SecureXL really looks at for the most part), and my guess is that the exception is not being applied correctly down into this sim optimization.  If you selectively disable SecureXL for either IP address (sk104468: How to exclude traffic from SecureXL) does the exception start working?  My guess is it might.

See here for more detail about this optimization:  Are IoC feeds processed before Access Control policy?

Also hopefully goes without saying, but once you try selectively disabling SecureXL make sure to start a new ping (or whatever test traffic) after waiting at least 30 seconds with no test traffic at all.

[Mike_Jensen]

Hi Tim,

Thank you for the great info!

I tried with selectively disabling SecureXL for the destination IP and had the same results, assuming I edited the table.def file correct:

[Expert@SMS-1:0]# vi $FWDIR/lib/table.def

f2f_addresses={<185.242.113.224>};

If I use the exception the way SmartConsole creates it from the log card option with the source being the protected scope traffic to the destination continues to be dropped by the threat prevention blacklist.

If I change the exception that SmartConsole created in anyway such as moving the host object in the protected scope to the source or adding the destination IP to the destination field I then see the drops in zdebug.

In my lab environment I turned off SecureXL on the gateway with fwaccel off just incase my addition to the table.def file wasn't correct and had the same results.

[Timothy_Hall]

Thinking about it more, disabling SecureXL won't matter as the optimal drop occurs long before it even figures out what path the connection should be handled in.  It still sounds like the exception you created is not being integrated into the optimal drop calculation inside sim, but this may be the case on a Firewall Worker Core as well if exceptions cannot be applied against custom indicators at all as you suspected initially.  Let me tag @Peter_Elmer and hopefully have him weigh in.

[Tobias_Moritz]

Very fresh from R81.20 JHF T43 from today:

PRJ-43434, PRHF-26673

Threat Prevention

UPDATE: It is now possible to add exceptions to external Custom Intelligence feeds.

Would be nice to know, how to do it. Just an exception for the relevant TP policy rule, like you tried?

[the_rock]

Im installing it in my lab atm, lets see : - )

Andy

[Mike_Jensen]

I wonder if this will be added to 81.10....

[the_rock]

I just innstalled jumbo 43 in the lab, pushed policy, literally no new options for TP exceptions for IOC, at least that I can see.

Best,

Andy

[PhoneBoy]

Try creating a regular Threat Prevention exception in the policy.

[Mike_Jensen]

Hi PhoneBoy, I have tried and it doesn't work either.

[the_rock]

No dice either...

Andy

[Mike_Jensen]

So even in 81.20 with jha take 43 installed any created exceptions for the ioc feeds simply don't work and the traffic is still prevented?

[the_rock]

correct

[Mike_Jensen]

Very interesting.  Thank you for sharing.

[the_rock]

Any time.

Andy

[Mike_Jensen]

I just tried in 81.10 with ga jumbo take 150 installed and it did not work.  I created the exception with the link right in the log card.

[Peter_Elmer]

Hello @Timothy_Hall , @Mike_Jensen ,

reading the all updates on 9-Jan here are my thoughts:

• I contacted @TPExpert offline to understand how we can achieve the required documentation of the whitelisting feature introduced in R81.20 JHF T43.
• The R81.20 is the recommended release for now and backporting the feature to R81.10 would require a formal RFE process. Don't expect this to be in a JHF for R81.10 without contacting your local sales contact and initiating the RFE process. I personally don't see such an RFE being done, as the R81.20 packet processing for IoC feeds has improved in general. I strongly suggest following the recommended release train.
• I searched for a flow diagram showing that IoC feed content is inserted into SecureXL. I recall such a diagram was in the sk92264 Anti-Virus and Anti-Bot ATRG but I can't find it anymore for now. I know from testing IoC feeds in R81.10 that observables where injected in SecureXL.
• The zdebug indicate 'dos' ...that made me searching SecureXL Penalty Box sk112454 . I don't have the reverse engineering proof that Penalty Box is 'feeded' by IoC. 

I am sorry, that by the time of this writing above is the best I can share as current business travels are limiting access to test lab.

From a design point of view, the IoC feed should be updated not to require exceptions - in other words: can we improve the source of the IoC? Exceptions are making 'life' complicated and 'complexity' is not a friend of 'security'.

If exceptions are required, the Threat Prevention rule bases is best using src/dest logic - not the the 'protected scope' logic. The src/dst logic makes it easier mapping Access Control rules and help 'reading' the Access Control and Threat Prevention policies.

Please let me know if more help is needed here.

greetings

pelmer 

[Mike_Jensen]

I submitted a RFE for this to be added to 81.10, feedback reference # 33a5qvL18.  It would be nice if Check Point can add this feature to R81.10 considering the OS is still supported until July 31 2025.

[the_rock]

Lets hope so.

[PhoneBoy]

If you saw the message from @TPExpert below, we are already planning to add this in the R81.10 JHF.
However, if you're using large IoC feeds, I strongly recommend upgrading your gateways (and management) to R81.20 which supports more than 2 million IoCs per feed and parses the feeds much more efficiently.

[Alex-]

Could you make this work? I've seen it was added in Take 131.

PRJ-43433, PRHF-26673

Threat Prevention

UPDATE: It is now possible to add exceptions to external IoC feeds.

However, editing $FWDIR/conf/ip_whitelist.eng doesn't change anything.

Gateways are R81.10 T139 + CVE fix, planned to be upgraded to R81.20 in the coming weeks.