R80.x Performance Tuning Tip – SMT (Hyper Threading)

Document created by Heiko Ankenbrand Champion on Nov 9, 2018Last modified by Heiko Ankenbrand Champion on Dec 3, 2018
Version 42Show Document
  • View in full screen mode
SMT (Hyper Threading)

Hyper Threading Technology is a form of Simultaneous Multithreading Technology (SMT) introduced by Intel. Architecturally, a processor with Hyper-Threading technology consists of two logical processors per core, each of which has its own processor architectural state.Each logical processor can be individually halted, interrupted or directed to execute a specified thread, independently from the other logical processor sharing the same physical core. Unlike a traditional dual-processor configuration that uses two separate physical processors, the logical processors in a hyper-threaded core share the execution resources. These resources include the execution engine, caches, and system bus interface; the sharing of resources allows two logical processors to work with each other more efficiently, and allows a logical processor to borrow resources from a stalled logical core (assuming both logical cores are associated with the same physical core).

A processor stalls when it is waiting for data it has sent for so it can finish processing the present thread. The degree of benefit seen when using a hyper-threaded or multi core processor depends on the needs of the software, and how well it and the firewall.

 

Hyper-Threading works by duplicating certain sections of the processor - those that store the architectural state - but not duplicating the main execution resources. This allows a hyper-threading processor to appear as the usual "physical" processor and an extra "logical" processor to the firewall.

 

The number of concurrent threads can be decided by the chip designers. Two concurrent threads per CPU core are common. Because it is really an efficiency technique that inevitably increases conflict on shared resources, measuring or agreeing on its effectiveness can be difficult.

 

Chapter

 

Architecture:

R80.x Security Gateway Architecture (Logical Packet Flow)

R80.x Security Gateway Architecture (Content Inspection) 

R80.x Security Gateway Architecture (Acceleration Card Offloading) 

R80.x Ports Used for Communication by Various Check Point Modules 

Performance Tuning:

R80.x Performance Tuning Tip - AES-NI 

R80.x Performance Tuning Tip - SMT (Hyper Threading) 

R80.x Performance Tuning Tip - Multi Queue 

R80.x Performance Tuning Tip - Connection Table 

R80.x Performance Tuning Tip - fw monitor

R80.x Performance Tuning Tip - TCPDUMP vs. CPPCAP 

R80.x Performance Tuning Tip – DDoS „fw sam“ vs. „fwaccel dos“ 

 

Preview to Intel Architecture

 

The following statement is also often discussed on the Internet:

SMT can increase message rate for multi process applications by having more logical cores. This increases the latency of a single process due to lower frequency of a single logical core when hyper-threading is enabled. This means interrupt processing of the NICs will be slower, load will be higher and packet rate will decrease. I think that's why Check Point doesn't recommend SMT in pure firewall and VPN mode. From my point of view, it only accelerates software balades. Therefore I use it if necessary, if many blades are activated. I'd like to discuss that with Check Point.

 

Small example with basic viewing:

This presentation is very simplified and should illustrate the issues. If SMT channel 2 uses all core resources with I/O operations, channel 1 must wait for the core resources. This can reduce the performance with enabled SMT. The same effect can occur with multi-queue and enabled SMT. The problem can be fixed by adjusting the Check Point affinity or disable SMT. What we see here, many Intel architecture issues can affect SMT and therefore the firewall performance.

 

Check Point and SMT

 

SMT is a feature that is supported on Check Point appliances running Gaia OS. When enabled, SMT doubles the number of logical CPUs on the security gateway, which enhances physical processor utilization. When SMT is disabled, the number of logical CPUs equals the number of physical cores. SMT improves performance up to 30% on software blades such as IPS, Application & URL Filtering and Threat Prevention by increasing the number of CoreXL FW instances based on the number of logical CPUs.

 

Turning on SMT can have some side effects in terms of multi-queue and affinity. After turning on SMT the affinity should normally be adjusted.

 

There are also some cases in which SMT should not be used:

  •         if only firewall/VPN blades are used
  •         if these blades are enabled:
    •        DLP
    •        Anti Virus
  •         if these features are enabled
    •        Using services with resources in firewall policy
  •         use hide NAT extensively -> Refer to sk88160 - Dynamic NAT port allocation feature to mitigate this limitation.
  •         more see SK93000

 

Following information must also be observed before turning on SMT:

  •         Changing SMT state requires reboot.
  •   For supported appliance refer to "Supported Appliances".
  •         If you have two ports on any of the aforementioned appliances to handle most of the traffic, it is recommended to enable the multi-queue feature to increase SMT performance.
  •         For R77.30 versions only!
      If the VPN blade is enabled and a significant amount of IPSec VPN traffic passes through the security gateway, then follow the steps in SK93000.
  •         For the best performance, it is recommended that user space processes not be affined to the same physical CPU core as the secure network distributors (SNDs), but rather that they be 
      affined to the firewall workers CPUs (more see SK93000, SK98737).
  •         If necessary, the affine has to be adjusted here (more see SK93000).
  •        Enabling SMT will load additional CoreXL FW instances. These instances consume memory and the maximal connection capacity may decrease by up to 10%.
     (info from Check Point Ofir S. )
  •         more limitations see SK93000

 

Supported configurations for SMT:

  •   only on Check Point appliances
  •   only on security gateways running Gaia OS with 64-bit kernel
  •   supported for R77 release and later

 

Tip!

On some open servers SMT (HT) must be disabled in the BIOS for the gateway installation. This is documented in the HCL for Server see link HCL.

For example for "HP ProLitan DL360 Gen9" refert to SK Required steps before installing Gaia OS on HP ProLiant Gen9 servers.

 

Supported Appliances

 

Attention!

SMT is supported only on following Check Point appliances not on Open Server.

 

ApplianceComment

3100

3200

No hardware support for SMT.

5100

5200

5400

5600

No hardware support for SMT.
5800
5900

Is already shipped with enabled SMT feature in the BIOS.

SMT is recommended with all blades.

  1. Installation of the hotfix from sk109772 - R77.30 NGTP, NGTX and HTTPS Inspection performance and memory consumption optimization.
  2. Enabling of SMT feature in 'cpconfig' (refer to "Enable SMT" section).
12400Requires 8 GB of RAM.
Refer to these solutions:
  • sk68700 for instructions for "12400 Appliance Installing and Removing Memory"
  • sk71001 for instructions on how to upgrade to 64-bit Gaia computers
Requires enabling of SMT feature both in the BIOS (1) and in 'cpconfig' (refer to "Enable SMT" section).
12600Requires enabling of SMT feature both in the BIOS (1) and in 'cpconfig' (refer to "Enable SMT" section).
13500
13800

Is already shipped with enabled SMT feature in the BIOS.

Requires enabling of SMT feature only in 'cpconfig' (refer to "Enable SMT" section).
15400
15600

Is already shipped with enabled SMT feature in the BIOS.

SMT is recommended with all blades.

  1. Installation of the hotfix from sk109772 - R77.30 NGTP, NGTX and HTTPS Inspection performance and memory consumption optimization.
  2. Enabling of SMT feature in 'cpconfig' (refer to "Enable SMT" section).
21400
21600
21700

SAM Acceleration card is not supported with SMT

Requires enabling of SMT feature both in the BIOS (1) and in 'cpconfig' (refer to "Enable SMT" section).
21800

Is already shipped with enabled SMT feature in the BIOS.

Requires enabling of SMT feature only in 'cpconfig' (refer to "Enable SMT" section).
23500
23800

Is already shipped with enabled SMT feature in the BIOS.

SMT is recommended with all blades.

  1. Installation of the hotfix from sk109772 - R77.30 NGTP, NGTX and HTTPS Inspection performance and memory consumption optimization.
  2. Enabling of SMT feature in 'cpconfig' (refer to "Enable SMT" section).
23900

Hyper Threading is hard-coded to be disabled on R77.30 and R80.10, with no impact on performance

TE250X
TE1000X
TE2000X

Is already shipped with enabled SMT feature in the BIOS.

Requires enabling of SMT feature only in 'cpconfig' (refer to "Enable SMT" section).

On these appliances, SMT is recommended with all blades.

40K 

60K

SGM220      no SMT support

SGM200T    no SMT support

SGM260      SMT support  (20 physical cores / 40 with enabled SMT)

SGM400      SMT support  (28 physical cores / 56 with enabled SMT)

(1) Note: To enable SMT in the BIOS, contact Check Point Support or contact Check Point Professional Services to get confirmation and approval beforehand.

 

 

Check if SMT is activated

 

Check if SMT is activated and check current SMT status on security gateway:

 

# cat /proc/smt_status

 

Either SMT is not supported on this machine or SMT is disabled in the BIOS.

SMT is enabled in the BIOS, but disabled in 'cpconfig'

SMT is enabled in BIOS and in 'cpconfig'

 

 

If SMT is activated this command shows the nummbers of CPUs, cores and SMTs:

 

# grep -E "cpu cores|siblings|physical id" /proc/cpuinfo | xargs -n 11 echo |sort |uniq

 

Enable SMT

 

1) Check the number of cores

# fw ctl multik stat

 

2) Check the number of cores in the gateway license.
# cplic print

 

3) Enable SMT in the BIOS + Reboot

 

4) Enable SMT in Check Point software and reboot the gateway.

 

Attention!
  1) If multi-queue and affinity are not adjusted or used, this can lead to performance problems in combination with SMT.

  2) Before enabling SMT, follow the instructions in SK93000 to verify that the Security Gateway can support SMT safely and check if there is enough memory available for the FW_Worker.

 

# cpconfig

  •        Choose 'Configure Hyper-Threading'
  •        Select 'yes'

# reboot

 

5) Check the number of CoreXL FW instances

# fw ctl multik stat

If the number of CoreXL FW instances did not increase automatically, then configure the CoreXL.

 

# cpconfig

  •         Choose 'Configure Check Point CoreXL'
  •         Set the desired number of CoreXL FW instances
      (for example 3 CoreXL FW instances)

Attention!
With a ClusterXL, the core number must be the same on all gateways. Otherwise ClusterXL problems will occur.

# reboot

 

6) Check again the number of CoreXL FW instances

# fw ctl multik stat

 

Disable SMT

 

1) Disable SMT in Check Point software and reboot the gateway.

# cpconfig

  •         Choose 'Configure Hyper-Threading'
  •         Select 'No'

# reboot

 

2) Check the number of CoreXL FW instances

# fw ctl multik stat

 

Q&A


Q: Which appliances were tested with SMT enabled.

A: 5800 / 5900 / 15400 / 15600 / 23500 / 23800 / TE250X / TE1000X / TE2000X

 

Q: I have an Open Server. Can I enable SMT in the BIOS and use it?

A: SMT is supported only on Check Point appliances.

 

Q: Is any degradation expected when enabling SMT?
A: Enabling SMT will load additional CoreXL FW instances. These instances consume memory and the maximal connection capacity may decrease by up to 10%.

 

Q: On which appliances is SMT enabled by default in the BIOS?

A: Only on 13500 / 13800 / 15400 / 15600 / 21800 / 23500 / 23800 / TE250X / TE1000X / TE2000X appliances

 

Q: What to do if multi-queue is enabled?

A: If using Multi-Queue, once the final CoreXL split has been set be sure to run cpmq reconfigure and reboot again, this will help ensure the new allocation of SND/IRQ cores are properly deployed for SoftIRQ processing on the Multi-Queue-enabled interfaces.

 

References

 

SMT (HyperThreading) Feature Guide

Best Practices - Security Gateway Performance

ATRG: CoreXL 

Dynamic NAT port allocation feature 

75 people found this helpful

Attachments

    Outcomes