Quantum - HyperFlow, Now in EA!

[Chen_Muchtar]

We are very excited to share that HyperFlow Early Available program is now open for registration.

In today’s fast changing environment, with a growing demand to address different traffic volumes per connection, HyperFlow is designed to automatically tackle such challenges.

Integrated with additional existing gateway performance features like Dynamic Balancing, HyperFlow allows seamless gateway tuning and optimization in the way we utilize the hardware providing Check Point unprecedented security.

See it in action - Demo & Overview of the new advancements:

HyperFlow EA is mainly for NGTP customers with appliance models of 8 cores and above.
Customer’s benefits:

• Influence – Ability to shape Check Point’s cyber security upcoming products
• Full technical support provided by EA & RnD through the entire process
• Upgrade from EA to GA once available

[the_rock]

Thats exciting news!

[genisis__]

Awesome!

But I see some caveats here.

- to resolve elephant flow traffic hyper flow requires more cores, so realistically your looks at a minimum appliance spec of a 66xx (Assuming here).

- Does hyperflow work on openserver?

[the_rock]

Thats good point, for sure...sometimes, as crazy as it sounds, but 8 cores might not be enough to solve elephant flows problem.

[Duane_Toler]

This is a solution in search of a problem (that doesn't exist).

If they honor DSCP PHB like they should, as per RFC-4594, then downstream devices can mark exceeding packets as CS1 and then FloodGate can just rate that at 10mbps.  Deal with the problem at the source, like RFC-4594 says.  "Mark as close to the source as possible".  Likewise, if a clever user tries to mark their packets as EF, that's a problem best-handled by the downstream switches.  They should be remarking incoming frames absolutely and not trust DSCP or CoS.

If the topology table already considers non-external interfaces as "trusted", then what's the problem?  Are those interfaces trusted or not?  "External" topology interfaces should have the option to trust DSCP or not (default to untrusted, but give the option to trust; there's a LOT more DSCP on the network than one may realize, including ECN, and it works).  Seems like Check Point is not honoring RFC-4594.

[_Val_]

Are you sure you are responding to the right thread? Does not look like it.

[Chen_Muchtar]

HyperFlow boosts elephant flow by processing it on multiple cores in parallel, a minimum of 8 cores is required to allow optimal resource management of the system.

Openserver support will be addressed in future JHFs

[Magnus-Holmberg]

Is / will VSX be supported aswell?

regards,
Magnus

[Chen_Muchtar]

Yes, VSX is supported 

[Magnus-Holmberg]

awesome! will it require 8 vs instances or how is it "enabled" when it comes to VSX?

[Chen_Muchtar]

The requirement in VSX is 6 FWK cores, not instances

[genisis__]

Question:

When you say 8 cores, do you mean 8 worker cores?  If so, this in affect would mean an appliance/openserver with more then 8 cores is required as clearly SND cores would need to be factored in.  This would then imply that we would require a minimum 10 cores, including SNDs.

So absolutely this is great milestone, but we are talking highend appliances could realistically benefit from this.

Hyperflow 2.0 ... objective, get this down to a minimum of 3 worker cores, and 1 SND 😉

[Chen_Muchtar]

8 logical cores, in this case:
w/o HyperFlow: 2 SNDs, 6 FW instances
w/ HyperFlow: 2 SNDs, 2 PPE MGR, 2 PPEs, 2 FW instances

[_Val_]

The assumption is way too liberal. 😀

Similar to Dynamic Balancing, HyperFlow can be available with 8 or more cores, as @Chen_Muchtar already mentioned

[genisis__]

Will checkpoint plan to address this on lower end appliances?  In most cases people would have 52, 62, 64 or even 6600 appliances where elephant flows are more likely to be seen (again making an assumption here).

[_Val_]

All with 8 or more CPUs. I actually misread your statement and thought you were talking about SP series.

[the_rock]

Not an assumption, seen it happen before. Though, not too often, but does happen.

[_Val_]

Regardless of amount of CPUs, we consider an elephant flow a.k.a. a heavy connection when:

• Specific instance CPU is over 60%
• Suspected connection lasts more than 10 seconds
• Suspected connection utilises more than 50% of the total work the instance does

As an example, for a single FWK instance, if it runs 60% of utilization, one single connection should be taking 30% or more. The "classic" cases are: DB replication, backups, VM migrations, and other data transfers between one source and one destination IPs.

The first condition of overall 60% CPU utilization is indeed more common to the situations where your appliance already is running high CPUs. However, you can also have that situation happening on a machine with many cores, where just one of them is too busy, and others are mostly idle. The recent example from today's community thread is here.

Just a few cores, or many of them, one specific FWK is suffering from a massive data transfer all the same. Before HyperFlow, we had Priority Queues and fast_access to work those issues around, now can leverage multiple CPUs for a single connection. With is, in my book, HUGE.

[Timothy_Hall]

Very nice, been a long road of development for this one!

[Martijn]

Hi All,

This is great news and a welcome feature for some of our customers.

Does Maestro and Scalable Platform support HyperFlow?

Regards,
Martijn

[Chen_Muchtar]

Maestro is supported 

[HeikoAnkenbrand]

We have been waiting for this for a long time;-)

Because the connection works over several cores, I have few questions:
1) Each core has a connection table. Will be the tabels are shared?
2) How does monitoring work via fw monitor. Are all cores that are used for the connection displayed here?
3) How is IPS, Anti Bot,... supported if the connection works over several cores? How does the PSLXL path work here?
4) Is there a design description for the packet flow?

[Tom_Loeber]

Agreed and great questions!

This leads me to ask if CPAS (Active Streaming) and security blade/malware detection is handled with Hyperflow?

I currently have a customer that shows zdebug drops when trying to download very large files. TAC case in progress.

[Chen_Muchtar]

PPE worker is capable to process hash of large files for AV blade in order to detect malware.

If CPAS is active over heavy connection supported by HyperFlow, and AV blade is enabled, the Hash calculation will be performed by PPE.

Tom - feel free to contact me offline @ chenmu@checkpoint.com to review your customer env. for this matter  

[Chen_Muchtar]

We should first separate between the FW instance handling the connection, and HyperFlow cores doing DPI processing. The only thing that is being shared between each FW instance and HyperFlow cores is relevant data for the DPI jobs to be processed in parallel.

Streaming and blade logic layer is still being handled by the FW instance owning the connection.

Packet flow design description:

-      This example showcases a single data packet flow

-      In this case, FW instance 1 is the connection owner

-      PPE Manager dispatches DPI jobs to PPE workers

-      Once the last job is done, a message is sent to FW, notifying that the DPI processing has been completed, allowing it to continue to outbound processing

-      PPE Manager can dispatch jobs to any PPE worker, even for the same connection, allowing multiple buffer’s jobs of the same connection to be processed concurrently

o    For example, multiple PM jobs of different buffers of the same connection can run concurrently on different PPE workers

Regarding FW monitor, it is a networking focused tool (output can be parsed by wireshark, for example), as such, it is built to work with packets as a whole.
Since HyperFlow works at the parser level, which dissects the TCP data into different segments, it uses other monitoring tools, such as connection tracker.

[genisis__]

It would be really nice to release a video demoing this from setup

[_Val_]

What setup? HyperFlow, similar to Dynamic Balancing, is automated, and works out of the box with 8 or more cores.

[HeikoAnkenbrand]

Hi @Chen_Muchtar,

Thanks for the detailed description.
👍

[Andrey_Kretsul]

It seems very interesting!

Does this feature support Jumbo Frame?

[Chen_Muchtar]

Yes