R80.x Performance Tuning Tip - AES-NI

[HeikoAnkenbrand]

What is AES-NI

Intel‘s AES New Instructions AES-NI is a encryption instruction set that improves on the Advanced Encryption Standard (AES) algorithm and accelerates the encryption of data in many processor familys.

Comprised of seven new instructions, AES-NI gives your environment faster, more affordable data protection and greater security.

Chapter

More interesting articles:

- R80.x Architecture and Performance Tuning - Link Collection

Appliances and Open Servers with AES-NI

Better throughput can be achieved by selecting a faster encryption algorithm. For a comparison of encryption algorithm speeds, refer to sk73980 - Relative speeds of algorithms for IPsec and SSL.

AES-NI is Intel's dedicated instruction set, which significantly improves the speed of Encrypt-Decrypt actions and allows one to increase AES throughput for:

•       Site-to-Site VPN
•       Remote Access VPN
•       Mobile Access
•       HTTPS Interception

The general speed of the system depends on additional parameters.

Check Point supports AES-NI on many appliances, only when running Gaia OS with 64-bit kernel. On these appliances AES-NI is enabled by default. AES-NI is also supported on Open Servers.

Affected encryption algorithms include:

• AES-CBC (128-bit and 256-bit)
• AES-GCM (128-bit and 256-bit), which shows the most significant improvement - with AES-NI, it is faster than AES-CBC, when both sides support AES-NI. Without AES-NI support, it is slightly slower than AES-CBC + HMAC-SHA1

Check Point supports AES-NI on the most appliances (only when running Gaia OS with 64-bit kernel).
AES-NI is also supported on Open Servers. Make sure that Gaia OS is running in 64-bit mode.

Check if AES-NI is activated

R80.10 - R80.30

Old AES-NI commands with "dmesg" no longer work in R80.40 and R81 (sk170779).

# dmesg | grep "AES-NI"

If it is not available, the following message is displayed:

R80.40 Jumbo HFA 100+ and R81 Jumbo HFA 13+

# fw ctl get int AESNI_is_supporte

0 = not supported
1 = supported

Check AESNI CPU support

It can also be checked if the CPU provides AES-NI. For this the following command should be executed. Here "aes" should now be displayed.

# grep -m1 -o aes /proc/cpuinfo

If AES-NI is not enabled, it must be turned on in the BIOS (if available). Typical way for Open Servers.

AES-NI performance measurement

A little bit of reverse engineering.

Check Point uses OpenSSL as library. Therefore the command "openssl" is provided as "cpopenssl". This gives us the possibility to execute all openssl commands. With this I tested a little bit and came to the conclusion that performance measurements are possible with the following command. So you can test the performance differences with enabled and disabled AES-NI.

Warning notice: If you execute this command you have 100% CPU usage on the firewall for 20 sec.

# cpopenssl speed aes-256-cbc

Enabled AES-NI:

Disabled AES-NI:

After these results I would always recommend to activate AES-NI and AES is preferred to 3DES because it offers many performance advantages through the hardware acceleration.

With the following command you can test and compare all encryption methods. After these results I would always recommend to activate AES-NI and AES is preferred to 3DES because it offers many performance advantages through the hardware acceleration. 

Warning notice: If you execute this command you have 100% CPU usage for a long time!

# cpopenssl speed

This makes it possible to compare encryption algorithms. It shows that e.g. AES 256 is more performant than DES. Therefore AES 256 should rather be used for VPN connections than DES or 3DES. This is also well described in the following SK Relative speeds of algorithms for IPsec and SSL.

References

Relative speeds of algorithms for IPsec and SSL 
Best Practices - VPN Performance 
vSEC Virtual Edition (VE) Gateway support for AES-NI on VMware ESX 
Best Practices - VPN Performance 
MultiCore Support for IPsec VPN in R80.10 and above 

[HeikoAnkenbrand]

What I noticed was, that R80.20 uses a 1 year old openssl version. I think R&D should look at that.

# cpopenssl version

[Vladimir]

Heiko,

Great write-up! Thank you.

Do you happen to know if AES-NI is working in VSX?

If it does, is it a global setting for the parent host or is it working on per-VS basis?

[Sandro_Gerdel]

Hi ,

Am I getting this right?

From your point of view it makes more sense to use AES256 for VPN than 3DES. Is that correct?

Cheers

Sandro

[HeikoAnkenbrand]

Hi Sandro,

Yes, according to SK and my tests, AES256 with AES-NI should be more effective.

More see here: Relative speeds of algorithms for IPsec and SSL 

Regards

Heiko

[HeikoAnkenbrand]

Hi Vladimir.

I can't say that 100% because the R&D guys should say something. But I think so, because this is actually on the operating system level. Here the linux crypto API is provided or the openssl crypto library.

Regards

Heiko

[Danny]

Note: Tim wrote on page 44 of his Security Gateway Performance Optimization presentation that large amounts of IPSec VPN traffic using algorithms SHA-384, AES-GCM-128 or AES-GCM-256 can cause an elevated percentage of traffic to be handled by SecureXL in F2F (Slow path).

[Timothy_Hall]

Update: AES-GCM-128 or AES-GCM-256 (Galois/Counter Mode algorithms) can be fully handled inside SecureXL/Accelerated Path starting in R80.30.  SHA-384 is still not implemented in SecureXL as of R80.30, so VPN traffic utilizing SHA-384 still cannot be accelerated by SecureXL.

[HeikoAnkenbrand]

Hi Danny,

I agree with you and Timothy.

IPSec VPN trafic utlizing the AES-128 or AES-256 algorithms does not require handling in F2F (Slow path). I always use AES-256 for site to site VPN. Here I get the best performance (AES-NI) with a good security level.

.

[HeikoAnkenbrand · Accepted solution]

When SecureXL is enabled, Encrypt-Decrypt actions usually take place on SecureXL level (on CPU cores running as CoreXL SND). All VPN traffic will be handled on the CPU cores running as CoreXL SND under the following conditions:

• Only "Firewall" and "IPSec VPN" software blades are enabled
• There are no fragmented packets
• SecureXL acceleration is not disabled by any of the security rules (refer to sk32578)
• VPN features that are disqualified from SecureXL (see below) are disabled

If all the above conditions are met, all VPN traffic will be handled on CPU cores running as CoreXL SND with minimum traffic being forwarded to the CoreXL FW instances, resulting in multi-core processing of VPN traffic (depending on the number of CPU cores running as CoreXL SND).

The following VPN features are handled by CPU cores running as CoreXL FW instances:

• Fragmented VPN packets
• Any compression algorithms (go to IPSec VPN Community properties - "Advanced Settings" pane - "Advanced VPN Properties")
• Using HMAC-SHA384 for data integrity and authentication (refer to sk104578)
• Any transport mode SA (used in L2TP clients and GRE tunnels)
• Multicast IPsec (GDOI)
• Monitoring Software Blade - if in addition to "System Counters", also "Traffic" counters are enabled in Security Gateway object (in such a case, connections are flagged with "Accounting" flag in the output of "fwaccel conns" command)
• Any Software Blades other than "Firewall" are used

Note:

With R80.20 fragmented packets do not necessarily have to run over the F2F path. With fragmented VPN packets under R80.20 I'm not sure which way they go.

More see here: Best Practices - VPN Performance 

[Alexander_Wilke]

Hi,

I am really shocked and worried that you are still talking and comparing Speed with DES and 3DES. No matter what Speed DES and 3DES has it is insecure and should never be used anymore - independent if it is faster or not. My opinion!

[Alexander_Wilke]

You can check OpenSSH version and you will be surprised ... 😉

[HeikoAnkenbrand]

Hi Alexander,

The statement I find great.

Point 1:
I totally agree with you. DES, 3DES, SHA1, RC4,…  should not be used anymore. That is also a clear recommendation in all my concepts. Now reality is catching up with us. Many firewall environments still use 3DES because the administrators have work with the change to AES-256. I think about 40% of the firewalls I see are still using 3DES. Secure Client connection is also used 3DES as default for phase 2 (see picture) in R80.20. I think it's a security issue.

Point 2:

Performance and security is a difficult issue in firewall tuning. There is a fine line between more security and high performance. This article is about performance and in this case the more powerful AES-256 encryption is also the safer one. It couldn't be better:-)

Point 3:

Please have a look at what Check Point offers with IPSec VPN in phase 1 and phase 2 in R80.20. Here DES and 3DES are still provided.

From my point of view, DES and 3DES will continue to exist for a long time. Why? Many other manufacturers do not manage to implement high encryption standards. These are often seen in medical sector or in industry sector. It's even worse for industrial systems. Often no encryption is used here.

And I agree with you, away with DES, 3DES, SHA1, RC4.

Regards

Heiko

[Dmitry_Krupnik]

Hello Heiko, Alexander,

Thank you for your notices, I have talked with RnD owners about these questions.

The OpenSSH and OpenSSL libraries are being patched with every relevant CVE. Open source packages on Check Point are being hardened and modified to answer our strict security standards. We also port many features from newer versions.

In other words, for example, the "OpenSSH 4.3p2" version in the R80.20 isn't equal to the OpenSSH 4.3p2, which we can download from the openssh.com. This is "OpenSSH 4.3p2 Checkpoint Edition".

[Alex-]

I was explaining to a customer the AES-NI thing and bumped on this interesting thread. It is in my understanding that there is a consensus that AES-GCM is preferable to AES-CBC because of built-in authentication and better general performance. Also, on 64-bit systems a subset of SHA-512 like SHA-384 is supposed to be more CPU efficient than SHA-256.

However Timothy Hall's presentation seems to show that using GCM and SHA-384 is actually worse than CBC and SHA256 on Checkpoint. Is it correct?

[Timothy_Hall]

"Worse" is a relative term.  VPN traffic that uses any form of GCM or SHA-384 cannot be handled by SecureXL in R80.10 and earlier and will be forced F2F which is a much less efficient processing path.  This assumes of course that there is not some other "Deep Inspection" blade such as APCL or Threat Prevention needing to inspect the traffic coming out of the VPN and forcing it into a higher path anyway.  So in a situation where only the most basic blades are enabled on a gateway (Firewall, IPSec VPN, and IPS w/ profile Default_Protection/Optimized) the vast majority of traffic can typically be handled in the most efficient SXL path at ludicrous speed, even if it is encrypting/decrypting to/from a VPN.  However use of GCM or SHA-384 on a VPN tunnel will force all of that tunnel's traffic F2F in R80.10 and earlier regardless of what blades are enabled.  

However on most modern firewalls, at least one "Deep Inspection" blade is enabled so most traffic won't be able to be handled in the SXL path anyway, and will end up in the Medium Path (PXL) or F2F.  An entire 16-page chapter of the second edition of my book (chapter 😎 was dedicated to this topic of VPN optimization.

However this limitation can be turned to your advantage.  If you suspect SecureXL acceleration is mishandling a certain VPN, simply enable a GCM-based algorithm or SHA-384 for that VPN which will keep SecureXL from trying to handle it and make it all go F2F in R80.10 or earlier.  This technique is much more efficient than disabling SecureXL completely, or alternatively switching off all SecureXL VPN acceleration with sim vpn off.

Edit: I forgot to mention that AES-NI can be successfully leveraged in all processing paths including SXL and F2F.

Update: AES-GCM-128 or AES-GCM-256 (Galois/Counter Mode algorithms) can be fully handled inside SecureXL/Accelerated Path starting in R80.30.  SHA-384 is still not implemented in SecureXL as of R80.30, so VPN traffic utilizing SHA-384 still cannot be accelerated by SecureXL.

--

CheckMates Break Out Sessions Speaker

CPX 2019 Las Vegas & Vienna - Tuesday@13:30

[Alex-]

Thank you Timothy Hall‌, very interesting. I will review that chapter of your book.

[HeikoAnkenbrand]

THX for this info.

Regards

Heiko

[HeikoAnkenbrand]

I also think that's a better way than disabling SecureXL completely.

I'd rather use:

sim vpn off

Regards

Heiko

[Tim_Maurer]

nice nice nice

[PhoneBoy]

Small addition to this thread from R&D:
AES-NI is supported and enabled out-of-the-box for 2.6.18 kernels.
In 3.10 kernels it’s supported and enabled out-of-the-box since R80.30 GA.

[Jan_Kleinhans]

Hello,

in the description there is mentioned, that AES-NI is enabled by default on 15600 and 23800 Appliances.

I cannot see this in /proc/cpuinfo or dmesg.  Is this the normal behaviour? We are using R80.20 64bit VSX

Best regards,

Jan

[PhoneBoy]

Which kernel?

[Jan_Kleinhans]

Hello,

2.6.18-92cpx86_64

[HkeNkd]

I ran the commands and got the following output:

So, how do you activate AES-NI if it is not turned on? I see the mention of enabling in the BIOS, but is that the case on a 15400 appliance? I haven't found any support posts or instructions in the admin guides on how to activate AES-NI.

[Timothy_Hall]

AES-NI will be utilized even if it is not showing up in the output of /proc/cpuinfo, please see my investigation of this here:

https://community.checkpoint.com/t5/SecureKnowledge/Tip-of-the-Week-VPN-Performance-Best-Practices/b...

[Timothy_Hall]

Updates: 

AES-NI may not show up as being enabled starting in R80.40 even though it is enabled and the firewall is actively utilizing it, see sk170779: AES-NI commands no longer work in R80.40

VPN traffic utilizing SHA-384 can now be handled by SecureXL and will not always be forced F2F starting in R80.30 Jumbo HFA Take 221+, R80.40 Jumbo HFA 87+, and R81.  sk168336: VPN traffic (after encryption) is not visible via tcpdump and does not arrive at the remot...

[Timothy_Hall]

sk170779: AES-NI commands no longer work in R80.40 and R81 has been updated since my post above, the new command to check for the presence of AES-NI support under R80.40 Jumbo HFA 100+ and R81 Jumbo HFA 13+ is now fw ctl get int AESNI_is_supported, 0=not supported, 1=supported.  @HeikoAnkenbrand please update the main post at the start of this thread.

[HeikoAnkenbrand]

Hi @Timothy_Hall,

Thanks for the tip. I have modified it.

[RamGuy239]

Interesting thread. When running R80.40+ with AES-NI supported I normally push for:

Phase 1:

IKEv2
AES-256-CBC
SHA-256
DH-Group 19

Phase 2:

AES-256-GCM
NULL
PFS Group 19


This seems to provide the best combination of secure encryption and good acceleration. There are always issues when going with AES-256-GCM when you are dealing with third parties. A lot of third parties like Microsoft Azure, Palo Alto etc. Allows you to utilise GCM for both Phase-1 and Phase-2. But the RFC for Suite-B which is the one Check Point follows utilises AES-256-CBC + SHA-256 for phase 1 when utilising AES-256-GCM for phase 2. This becomes messy when third parties start utilising GCM for both phase 1 and phase 2. Doesn't seem to be any way to manually override this on Check Point, there exists no option to enforce the use of GCM for phase 1 regardless of selecting Suite-B or doing the settings manually.


Where things become messy is when you add Gaia Embedded / SMB into the mix. There is a lack of information on what Gaia Embedded supports in terms of hardware-accelerated encryption algorithms. The end-of-sales, but not yet end-of-support 700 and 1400 series is still running R77.20.XX so I wouldn't expect them to really support much at all? Considering how this thread shows how you should be running R80.20+ in order for SecureXL to become handy.

For the new 1500, 1600 and 1800 series there isn't much information to be found either. According to the R81.10.00 release notes it mentions:

Supported Hardware Acceleration

Hardware acceleration only supports:

AES-128, AES-256, DES, and 3DES encryption methods
MD5 and SHA1 integrity check types

Does this mean that even the latest SMB appliances running the latest R81.10.00 software are incapable of hardware-accelerated AES-256-CBC and SHA-256? Let alone any AES-GCM at all? This seems rough considering most people would recommend companies to at least opt for SHA-256 as the bare minimum in 2022. Do to the weak CPU in these SMB appliances throughput seems to get really slow once you step outside of accelerated traffic making it best practice to not opt for anything beyond AES-256-CBC + SHA-1?

None of the AES-NI commands works on SMB so there is no way to verify anything regarding AES-NI on these. They don't seem to run an X86-based CPU, to begin with, and when it comes to MIPS or ARM it all depended on the SoC whether it has added AES-NI instruction sets or not. This makes it really hard to wrap one's head around whether it supports AES-NI and if doesn't, what encryption algorithms do is supported to be offloaded to hardware and not.


The same goes for questions such as will VTI affect acceleration on SMB? On regular Gaia, VTI would affect SecureXL when running anything prior to R80.20. But with the rewritten SecureXL engine that came with R80.20, we are now able to fully utilise VTI without any downsides to performance. How does this affect Gaia Embedded / SMB? Feels like guesswork at this point, but I have just concluded based on logic that 700 and 1400 series won't support the use of VTI without issues with SecureXL as it's still on R77.20 code. While 1500, 1600 and 1800 might support it as it's based on R80.20 code?