USFW on appliances with less than 40 cores

[HristoGrigorov]

So, as it became evident starting from R80.40 USFW is now automagically enabled on some appliances even if they have lower than 40 amount of CPU cores - 4,8,16. Also, few people reported increased amount of CPU usage on such systems.

Bug or a feature ?

Share your thoughts, expectations, observations, curses, etc...

[HristoGrigorov]

Let me be the first to report...

Mine is 4-core 3600 appliance. It came from CheckPoint with R80.30 and USFW was enabled by default. Recently I upgrade it to R80.40. Because most of our users are working from home now load on appliance is really low so can't say about CPU usage...

[HeikoAnkenbrand · Accepted solution]

On 3.10 kernel (R80.40) UMFW is enabled by default.

I can confirm similar behavior on some firewalls. What surprises me is that the basic process is already producing about 10%-20% CPU load (without firewall traffic).

In UMFW the fw instances are threads of the fwk0_dev_0 so by default the top shows all the threads cpu utilization under the main thread. Top has the option to present the utilization per thread as well.

A small calculation sample for the utilization of process fwk0_dev_0:

                                 max_CoreXL_number            max_CoreXL_number
fwk0_dev_0      =      ∑       fwk0_x                    +       ∑         fwk0_dev_x          +        fwk0_kissd        +          fwk0_hp
                                 x=0                                              x=0

Thread from process fwk0_dev_0:

- fwk0_X              ->  fw instance thread that takes care for the packet processing
- fwk0_dev_X      -> the thread that takes care for communication between fw instances and other CP daemons 
- fwk0_kissd       -> legacy Kernel Infrastructure (obsolete)
- fwk0_hp            ->  (high priority) cluster thread

More read here:
R80.x - Performance Tuning Tip – User Mode Firewall vs. Kernel Mode Firewall 

[Logan_Thomas]

I just wanted to share my quick experience with USFW, as this is something to be aware of since it is automatically enabled in R80.40.

If you have high F2F traffic, be careful enabling USFW. We were running around 40% F2F traffic and probably an average CPU load of around 60% during peak usage. We got a new firewall that had USFW enabled by default. When we hit peak loads, our firewall started dropping at least 1/3 of the packets, if not more. This happened even though our overall CPU usage was lower. As soon as we disabled USFW, everything worked great. So make sure you fix any F2F traffic issues before enabling USFW and disable USFW on any new firewalls if needed. Also note that even though TAC confirmed the process for us to disable it, it kept turning back on after reboot and required TAC to edit some files for us.

We eventually figured out our high F2F issue and we should be able to run USFW just fine now.

[HeikoAnkenbrand]

GAIA version/ Kernel/ Cores	Firewall mode	Check
R80.30 kernel 3.10 more then 35* cores	UMFW is enabled	checked on HP DL 380 G10 2 * Platinum 8180MProcessor 28 cores = 56 cores
R80.30 kernel 3.10 less then 35* cores	KMFW is enabled	checked on HP DL 380 G10 1 * Platinum 8180MProcessor 28 cores
R80.30 kernel 2.6	KMFW is enabled	checked on VMWare with 30 cores and with 46 cores
R80.40 (default 3.10 kernel)	UMFW is enabled by default	checked on VMWare with 4 cores

[HristoGrigorov]

I came across this article that gives a clue why is USFW efficient even on small amount of CPU cores:

https://netdevconf.info/2.1/papers/netdev.pdf

Btw, cpview calls this "zeco" 😀

[Bruno_Duarte]

Let me share my experience..

I have customers with 6200 appliances running R80.30 and USFW was enabled by default..

[HeikoAnkenbrand]

On R80.30 with 3.10 kernel it is enabled by default.

[JozkoMrkvicka]

but only in case more than 40 cores, right?
16000 appliance has 32 cores, is based on R80.30 3.10 kernel and USFW is disabled.

[Timothy_Hall]

In preparation for release of the R80.40 addendum for my book, I had an very enlightening discussion with a member of R&D about USFW.  I will alert him to this thread privately so he can correct anything that I missed.

Whether USFW will be enabled by default is actually much more dependent on specific hardware/appliance type than Gaia kernel or number of cores.  So assuming at least version R80.30 here we go...

• USFW is enabled by default on Check Point Appliance 2019 series (3600 [4 core], 3800 [8 core], 6XXX, 7XXX, 16XXX, 26XXX, 28XXX)
• USFW enabled by default in any kind of virtualized environment like VMWare, regardless of the number of cores.  Only 2 cores present in VMWare?  USFW enabled.
• Open Hardware Server (not VMWare) - Depends on number of cores as Heiko said, less than 35 cores USFW disabled by default, more than 35 cores USFW enabled by default.
• USFW not enabled by default on Check Point appliance 2016 series (3100, 3200, 5XXX, 15XXX, 23XXX)  except for model 23900 which has USFW enabled by default.
• USFW not enabled by default on Check Point appliance 2012 series (2200, 4XXX, 12XXX, 13XXX, 21XXX).  Note that most if not all of these 2012 series appliances reach end of support in 2022.

This criteria for whether USFW is enabled by default seems to have changed over time, which may explain some early 16000's that don't have USFW enabled by default that were mentioned in this thread.

[HristoGrigorov]

The explanation that USFW depends mostly on hardware type (I assume here processor family) and at the same time is enabled on any virtualized environment kind of contradicts by itself.

I think USFW actually depends mostly on whether hardware architecture is suitable for running efficiently hypervisor or not. 

Otherwise, our 3600 seems to cope very well with USFW on R80.40.  

[shais]

Hi all,

I understand there is a bit of confusion regarding USFW status on R80.30 3.10 and R80.40.

I will make sure to post in the upcoming days a clear information about this.

In the meantime i would like to clarify regarding the question above 

R80.40 is not USFW by default, USFW status is depend on  

• Hardware type, for example 6900 appliance or VMs will run in USFW by default (since R80.30 3.10)
• Number of cores - Kernel is limited to 40 instances, above can only run in USFW

I'm currently collecting all the USFW questions and will answer all of then in a single post

Thanks,

Shai Shabat - Framework group manager ,CheckPoint

[HristoGrigorov]

This USFW is really cool!

I am on R80.40 and today one of the firewall processes went nuts on policy install and crashed. It was quickly restarted and there was no reboot, only minor loss of connectivity to here and there.

I am two Takes behind and I know at least one of them fixes something like that but that's not the point. 

[Rob_Bush]

---

@shais wrote:

R80.40 is not USFW by default, USFW status is depend on  

• Hardware type, for example 6900 appliance or VMs will run in USFW by default (since R80.30 3.10)
• Number of cores - Kernel is limited to 40 instances, above can only run in USFW

Just an FYI... I installed R80.40 on a 4400 and 4800 and USFW turned on by default. I noticed my CPU load was twice as high as on R80.40 as it was on R80.30.  I had a ticket with CP here recently on an unrelated issue and the tech noticed USFW was on and disabled it.  When they did that, my CPU load dropped by 50% immediately.

There is clearly something wrong in the code in R80.40 install that is causing it to turn on when it shouldn't be.  I installed R80.40 using the latest BLINK image on March 20th, 2020.

[shais]

Hi,

I'm sorry to hear that USFW resulted in CPU spike on your system, may i please get the ticket you had with support? i would like to see the information collected.

As for the 4400/4800 appliance running by default in USFW - I will verify this in our lab as this appliance should not run in USFW

[Rob_Bush]

6-0001980814

[_Val_]

All, there is a new SK available for the matter:

https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...

[HristoGrigorov]

Linking thread with answers from CheckPoint:

https://community.checkpoint.com/t5/General-Topics/User-Space-firewall-support-for-R80-30-3-10-and-a...