Rulabese matching mechanism column/row

[Martin_Raska]

Hello,

I would like to ask for clarification regarding rulebase matching.

Some materials (including SKs and the CCSA R82 course) state that the rulebase is evaluated row by row and its best practise to put most hit rules to the top.

However, when R80 was introduced, there were materials (CCSA or CCSE) explaining that evaluation starts with the Destination column, then checks all Source columns, and finally the Service column. So in this mechanism there is irelevant to put most hit rules to the top, because first you verify all rules but only destionation column, then source etc.

Could someone please clarify which behavior is correct?

[simonemantovani]

Hello

from what I know, every traffic is evaluated based on source/destination/port ... so this confirm what best practices recommend, that is to put most used rules on top.

[Martin_Raska · Accepted solution]

I am sharing old thread describing column-based search

https://community.checkpoint.com/t5/Firewall-and-Security-Management/Unified-Policy-Column-based-Rul...

[simonemantovani]

mmm ... I don't think that it refers to the order of the policy, but how the firewall matches a rule and what criteria are used to match the correct rule.

For what I understand it explains the process to match the rules

[Vincent_Bacher]

I cannot agree with your assumption
"So in this mechanism there is irelevant to put most hit rules to the top, because first you verify all rules but only destionation column, then source etc."

Why do I think so:

1. 1. First-match principle still applies
   The column-based optimization doesn't change WHICH rule matches - it's still the first matching rule from top to bottom. If you have overlapping rules, order matters:
   Rule 10: Allow 192.168.1.0/24 → Any → SMTP
   Rule 50: Allow Any → Any → Any
   A packet from 192.168.1.10 to SMTP will match Rule 10, not Rule 50, because rule order matters.
2. 2. Performance benefits remain
   Even with column-based optimization, putting most-hit rules at the top improves performance:
   Each column pass still processes rules top to bottom
   A rule at position 5 requires checking 5 rules per column pass
   A rule at position 500 requires checking 500 rules per column pass
   Most-hit rules at the top = fewer lookups = better performance

[Martin_Raska]

Hi Vincent,

I dont agree with point 2. If you have 500 rules, first you are looking match to all 500rules at destination column, so its does not matter if rule is at the top or bottom. In this cycle you find 20 matches on destination column and now you search hit at the source column of those 20 rules, you hit 5, so you have 5 rules where you have hit on source and destination column, now you need to do match on source. You hit two rules and first one is your match and considered full match rule.

[Vincent_Bacher]

Thanks for your explanation. Now it depends on the details of the rule search. If all 500 rules are searched through the destination column first, then all (remaining) through the source column, and so on, then yes, you are right. I understood it a bit differently, but if your explained approach is the one that's now used, you are correct. Then from a performance point of view, the position has minimal impact. Other principles (correctness, maintainability) then determine the order.

[Martin_Raska]

I am trying to figure out if this mechanism is still applied, its described here.

https://community.checkpoint.com/t5/Firewall-and-Security-Management/Unified-Policy-Column-based-Rul...

It was also part of CCSE trainings in the past.

[Vincent_Bacher]

Out of curiosity I had a look at a kernel debug trace I did for a single echo request to see how the column-based matching works in practice.
Here's what I observe:

When the rulebase matching starts for the actual policy (not implied rules), you can clearly see the column-by-column approach:

1. Destination Column Pass:

{rulebase} up_column_ip_opm_match: matching column Destination
{match} up_policy_tables_get_rulenum_list_from_range_tbl: found 3 matching rule ranges
matching rule ranges list: [1 - 2] [4 - 5] [16777215 - 16777215]
{match} up_handle_intersect_rules: adding 1
{match} up_handle_intersect_rules: adding 2
{match} up_handle_intersect_rules: adding 4
{match} up_handle_intersect_rules: adding 5
{match} up_handle_intersect_rules: adding 16777215
{match} up_handle_intersect_rules: 5 elements in matched rules array

I guess 16777215 is the implied cleanup rule. 

2. Then Source Column Pass:

{rulebase} up_column_ip_opm_match: matching column Source
{match} up_policy_tables_get_rulenum_list_from_range_tbl: found 2 matching rule ranges
matching rule ranges list: [4 - 5] [16777215 - 16777215]
{match} up_handle_intersect_rules: remove rulenum 1
{match} up_handle_intersect_rules: remove rulenum 2
{rulebase} up_handle_intersect_rules: keep rulenum 4
{rulebase} up_handle_intersect_rules: keep rulenum 5
{rulebase} up_handle_intersect_rules: keep rulenum 16777215
{match} up_handle_intersect_rules: after join - num matched rules 3

3. Then Service Application Column Pass:

{rulebase} up_compound_column_parent_default_opm_match: matching column parent 'Service Application'
{match} up_policy_tables_get_clob_rulenum_list: found 1 matching rule ranges
{match} up_policy_tables_get_clob_rulenum_list: range 0 [5 - 5]
{rulebase} up_handle_intersect_rules: keep rulenum 4
{rulebase} up_handle_intersect_rules: keep rulenum 5
{rulebase} up_handle_intersect_rules: keep rulenum 16777215
{match} up_handle_intersect_rules: after join - num matched rules 3

4. Final match determination:

{rulebase} up_sub_policy_post_columns_calc: we have a Match(4)!
{rulebase} up_sub_policy_match: return match status 'MATCH' action=Accept
'GlobalerKrempel Network', 4, dir 0, 10.17.71.196, 63239, 10.17.71.192, 0, IPP 1, -1, Accept
'GlobalerKrempel Network' (0): MATCH 4(1.4) Accept

The system first checks all rules against the destination column, builds a matched rules array (5 rules in this case), and then proceeds to check those remaining rules against the source column. This is the column-based matching optimization in action.

The function names like up_column_ip_opm_match (opm = optimized policy matching) and up_handle_intersect_rules clearly show this mechanism.

edith adds: I did this kernel debug on a R82.10 device.

[Martin_Raska]

So we agreen on matching method now. Do we also agree that put most hit rules to the top of rulebase is not necessary and statement that it makes performance benefit is obsolete?

[Vincent_Bacher]

This assumption is too complex to be answered in the affirmative or negative. I am not sufficiently familiar with the subject to provide an informed opinion.

[PhoneBoy]

For optimizing rulebase lookup, the location of a "most used" rule no longer makes a significant difference.
There may be other reasons you might have a "most used" rule early or late in the rulebase, depending on the rules in question and their impact on SecureXL.
Consider the following example:

If the order of the rules were reversed, backup server traffic would be subject to Medium Path inspection, creating a performance bottleneck.
Technical reason for this: the service "https" can be matched on the first packet (TCP SYN), thus the connection can be fully accelerated with SecureXL. 
That would apply even if the second rule (with Web Browsing) is used more often.

[Martin_Raska]

Yes this makes a sense if you dont use threat prevention.

[_Val_]

The new match process: Destination (array) -> Source (array) -> Service / Application provides a performance boost for non-optimized rulebases, if compared with the previous logic: rule by rule match for source, destination, and service.