Introduction This document describes the content inspection in a Check Point R80.10 and above gateways. Context Management Infrastructure (CMI) is the "brain" of the content inspection and use more different modules (CMI Loader, PSL vs. PXL, Protocol Parsers, Pattern Matcher, Protections and new in R80.10 NGTP Architecture) for content inspection. Chapter More interesting articles: - R80.x Architecture and Performance Tuning - Link Collection- Article list (Heiko Ankenbrand) What’s new in R80.10 UP Manager Unified policys NGTP Architecture Content Awareness (CTNT) Context Management Infrastructure This picture should show a block overview and not all components are included. It is intended for explanatory purposes only. Context Management Infrastructure - The Context Management Infrastructure (CMI) is the "brain" of the content inspection. It coordinates different components, decides which protections should run on a certain packet, decides the final action to be performed on the packet and issues an event log.CMI separates parsers and protections. Protection is a set of signatures or/and handlers, where Signature - a malicious pattern that is searched for Handler - INSPECT code that performs more complex inspection CMI is a way to connect and manage parsers and protections. Since they are separated, protections can be added in updates, while performance does not depend on the number of active protections. Protections are usually written per protocol contexts - they get the data from the contexts and validate it against relevant signatures. Based on the policy, the CMI determines which protections should be activated on every context discovered by a protocol parser. If policy dictates that no protections should run, then the relevant parsers on this traffic are bypassed in order to improve performance and reduce potential false positives. CMI Loader - collects signatures from multiple sources (e.g. IPS, Application Control,...) and compiles them together into unified Pattern Matchers (PM). Stream Assembly - In order to identify more sophisticated threats we need to do additional inspection of the initial and subsequent packets within a session. The Streaming Engine processes the individual packet chains, creates an ordered packet stream and directly performs a number of security functions on the stream. One is to validate the TCP stream, e.g. looking for retransmissions with modified data or TCP connection packets with no handshake. The streaming engine passes assembled stream to the protocol parsers. The engine has two modes depending on the nature of the traffic. In the passive mode there is limited opportunity to modify the traffic stream itself. The active mode however, is able to modify the connection. This mode essentially proxies the TCP connection and is necessary when performing HTTPS man-in-the-middle inspection. Active streaming also facilitates timing and buffering when it’s necessary to inspect content such as large files. Streaming serves - an important function in the NGTP software architecture. The streaming process creates an ordered packet stream and directly performs a number of security functions on the stream. NGTP can assemble packets into a stream in two ways, passive or active, depending on the nature of the traffic. Each has advantages and disadvantages. The passive mode gives little opportunity for modifying the traffic stream. In contrast, the active mode can modify the connection. Active mode essentially proxies the TCP connection and is necessary when performing HTTPS inspection. Active streaming also facilitates timing and buffering when inspecting content such as large files. Keeping the goals of integrated, high security effectiveness and performance in mind, Check Point chose to provide the option to do passive and active streaming, which provides the best balance of security and optimized performance based on actual usage conditions. Passive Streaming Library (PSL) PSL - Packets may arrive out of order or may be legitimate retransmissions of packets that have not yet received an acknowledgment. In some cases a retransmission may also be a deliberate attempt to evade IPS detection by sending the malicious payload in the retransmission. Security Gateway ensures that only valid packets are allowed to proceed to destinations. It does this with Passive Streaming Library (PSL) technology. PSL is an infrastructure layer, which provides stream reassembly for TCP connections. The gateway makes sure that TCP data seen by the destination system is the same as seen by code above PSL.This layer handles packet reordering, congestion handling and is responsible for various security aspects of the TCP layer such as handling payload overlaps, some DoS attacks and others. The PSL layer is capable of receiving packets from the firewall chain and from SecureXL module. The PSL layer serves as a middleman between the various security applications and the network packets. It provides the applications with a coherent stream of data to work with, free of various network problems or attacks The PSL infrastructure is wrapped with well defined APIs called the Unified Streaming APIs which are used by the applications to register and access streamed data For more details - how a connection comes into the PSL modul - see article:R80.x Security Gateway Architecture (Logical Packet Flow) PSL is visible in the input and output fw monitor chain as "passive streaming (pass_str)". PXL vs. PSLXL - Technology name for combination of SecureXL and PSL. PXL was renamed to PSLXL in R80.20. This is from my point of view the politically correct better term. QXL - Technology name for combination of SecureXL and QoS (R77.10 and above). This has no direct association with PXL. It is used exclusively for QoS. But also here it is possible to use the QoS path in combination with PSL. This picture shows an example of http. PSL/ Protocol Parsers and Pattern Matcher include http, SMTP, DNS, IMAP, Citrix, and many others protocols. Protocol Parsers - Protocols include HTTP, SMTP, DNS, IMAP, Citrix, and many others. Protocol parser instances register with the streaming engine in order to receive ordered streams of data, both client-to-server (C2S) as well as server-to-client (S2C) streams. They test for conformance to RFCs and look for anomalies. An example of a protocol anomaly would be where, for example, there is a non-existing HTTP method instead of GET, POST etc. in the HTTP header. Another job of the parsers is to normalize content. The parser will normalize this content before passing the URL to the protections for inspection. The parsers main purpose is to extract ‘contexts’ from the streams to prepare for the next level of inspection. The content is normalized before inspection to thwart attempts to evade detection. The protocol parser detects and prevents protocol anomalies. The parsers main purpose is to extract ‘contexts’ from the streams to prepare for the next level of inspection. Pattern Matcher - Security modules each have a distinct function and they register with the CMI to receive particular context types. The CMI Loader maintains a register table of all security modules and the contexts they want to inspect. This registration process occurs at policy install time. During policy installation, the CMI collects signatures from multiple sources (e.g. IPS and Application Control) and compiles them together into Pattern Matchers (PM) (one for each context - such as URL, Host header etc.). The CMI Pattern Matcher quickly identifies harmless packets, common signatures in malicious packets, and does a second level analysis when needed to reduce false positives. The engine uses a two tiered inspection process. The first tier quickly filters out the vast majority of traffic which is clearly harmless by looking for signatures that are simple to find at a low CPU cost. If the first tier identifies a common attack signature a second level analysis is done, thus increasing the confidence that there is indeed an attack. The first tier will never decide on its own that a packet is malicious. It can only decide that a packet is clearly harmless. The second tier can also be instructed to activate further inspection using INSPECT technology when some patterns are matched. The Pattern Matcher is a fundamental engine within the enforcement architecture. Pattern Matcher quickly identifies harmless packets, common signatures inmalicious packets, and does a second level analysis to reduce false positives. Pattern Matcher engine provides the ability to find regular expressions on a stream of data using a two tiered inspection process: The first tier quickly filters out the vast majority of traffic which is clearly harmless by looking for signatures that are simple to find at a low CPU cost. If the first tier identifies a common attack signature it passes the connection to the second tier to do a second level analysis, thus increasing the confidence that there is indeed an attack. The first tier will never decide on it’s own that a packet is malicious. It can only decide that a packet is clearly harmless. The second tier can also be instructed to activate further inspection using INSPECTv2 technology when some patterns are matched. Protections - are composed of signatures that identify malicious activity. There are three types of signatures; Regular expression like patterns INSPECT code Compound Signature Identification (CSI) Regex signatures are done first followed by INSPECT and CSI if needed. CSI can address signatures on multiple parts of a packet, multiple parts of the protocols such as URL and an HTTP header, multiple parts of a connection, such as CIFS request and response, or multiple connections, such as VoIP control and data connections. In its operation, CSI constructs complex signatures that are triggered only if a certain logical condition over multiple contexts is matched. The logical expression can use AND, OR, NOT or ORDERED-AND to construct the logical expression. Content Security Policy and Inspection - When enabling advanced security features like Application Control, URL Filtering, Content Awareness, IPS, Anti-Bot, Antivirus and SandBlast Threat Emulation (sandboxing); the security gateway inspects content using the contexts derived from the protocol parser. The gateway’s local cache is augmented with real-time cloud updates and zero-day threat protection. During policy installation, signatures are compiled into Pattern Matchers (PM) with one PM for each context such as URL, Host header etc. The PM quickly identifies harmless packets, common signatures in malicious packets, and if needed, does a second-level analysis to reduce false positives. The engine uses a two-tiered inspection process. The first tier quickly filters out the vast majority of traffic, which is clearly harmless, by looking for signatures that are simple to find at a low CPU cost. If the first tier identifies a common attack signature, it passes the connection to the second tier to do a second level analysis. This increases confidence there is indeed an attack. It is worth noting that in some cases, the determination is a simple comparison. For instance, minimal processing time is needed to check an IP, a URL, a domain, or a file checksum against a known list. These comparisons are done first. A Context Management Infrastructure (CMI) connects parsers with pattern matchers. CMI determines which protections should be activated on every context discovered by a protocol parser. If policy dictates that no protections should run, then the relevant parsers on this traffic are bypassed in order to improve performance and reduce potential false positives. For instance, if the IPS policy is activating server protections only, then the HTTP parser will not analyze the server to client (S2C) traffic. After the security module analysis is done, their verdict is passed to the CMI to perform the final action to be done on the various contexts. Cache with Cloud Backup- After a match by the Pattern Matcher, the CMI is notified by the relevant module like Application Control, URL Filtering, Antivirus, Anti-Bot and Threat Emulation that its signature was matched. Additional processing is done as needed by the security module. For instance Application Control retrieves all the data regarding the application (its categories, priority, etc.) from a cache based on the signature that was matched. If a match is not found or social network widget detection is needed, then the cloud service is checked. URL Filtering is another example where if the context is not found in cache, then the cloud service is checked. URL categorization occurs mostly in the kernel. When not found in the local cache, the cloud service is checked and the result is then cached in the kernel.For instance: Application Control retrieves all the data regarding the application (its categories, priority, etc.) from a cache based on the signature that was matched. If a match is not found or social-network widget detection is needed, then the cloud service is checked and the cache is updated with the result. URL Filtering checks for the URL category in the local cache. If not found, then the cloud service is checked. The cache is then updated with the result. Threat Emulation is a special case where the Pattern Matcher can be bypassed. If the checksum of a file is not cached, then the sandbox needs to receive a complete file. The protocol parser extracts the file and sends the file to our cloud service for processing where compute, disk and memory are virtually unlimited. If the protocol requires immediate delivery as in the case of HTTP/S, we offer user agents to notify the user and deliver only safe content to the user while the emulation happens in the background. Newly discovered threats are added to the local cache and sent to the cloud database to protect other Check Point platforms. File Handling - In some situations we need to inspect an entire file for malicious behavior. In these cases the file is sent directly to Threat Emulation, Threat Extraction and Antivirus for processing. If the SHA1 of the file is not cached, then these modules may need to receive a complete file. Files can be extracted from streams and security gateways have the option to act as a mail relay (Mail Transfer Agent), which allows us to inspect encrypted SMTP traffic over TLS. With SMTP inspection users don’t notice delays whilst inspection like that required in Threat Emulation (sandboxing) of large or complex attachments is carried out. When files are transferred over HTTP/HTTPS, then users may notice a delay while Threat Emulation occurs. SandBlast Threat Extraction technology eliminates threats by removing exploitable content and reconstructing documents using known safe elements. Safe content is delivered to the user. Users are allowed access to original files after Threat Emulation completes its analysis. Picture from Check Point article. Final Decision - After the security module analysis is done, their verdict is passed to the CMI to perform the final action. The CMI is responsible for the final action to be performed on the packet, given several considerations. The considerations include: Activation status of the protection. e.g. IPS prevent, detect or inactive Exceptions either on traffic or on IPS protection Bypass mode status (the software fail open capability) Troubleshooting mode status Are we protecting the internal network only or all traffic UserCheck - In the Application Control, URL Filtering and DLP security policy, the action Ask User presents users with a UserCheck dialog window prompting them to confirm they wish to complete the connection and if needed provide a reason. For example they may be visiting a risky site or sending confidential content to an external recipient. Depending upon their response the connection will be allowed or denied. Active Streaming (CPAS) Active Streaming (CPAS) - Check Point Active Streaming active streaming allow the changing of data, we play the role of “man in the middle”. CPAS breaks the connection into two parts using our own stack – this mean, we are responsible for all the stack work (dealing with options, retransmissions, timers etc.). An application is register to CPAS when a connection start and supply callbacks for event handler and read handler. Several protocols uses CPAS, for example: Client Authentication, VoIP (SIP, Skinny/SCCP, H.323, etc.), Data Leak Prevention (DLP) blade, Security Servers processes, etc. In some scenarios, HTTP is also using CPAS, when IPS Web Intelligence protections are enabled. As for now, CPAS doesn't support accelerated traffic, this mean that each CPAS packet will be F2F. CPAS works through the F2F path in R80.10 and R77.30. Now CPASXL is offered in SecureXL path in R80.20. This should lead to a higher performance. General overview: CPAS breaks the connection into two parts using our own stack – this mean, we are responsible for all the stack work (dealing with options, retransmissions, timers etc.) An application is register to CPAS when a connection start and supply callbacks for event handler and read handler. On each packet, CPAS send the application the packet data with cpas_read, allow the application to change the data as it like, and send the data forward with cpas_write. As for now, CPAS doesn't support accelerated traffic, this mean that each CPAS packet will be F2F. Since all the packet will pass SecureXL (even if they were F2F) it need to offload the connection to SecureXL. The offload is done on the outbound, without CPAS, the offload will occur when the SYN packet leave to the server, because CPAS break the connection, the first outbound will occur on the SYN-ACK that is sent to the Client CPAS does not synchronize connection between cluster members. If a failover occur and the connection was handled by CPAS then the connection will terminate. The only exception is VOIP connection. CPAS is visible in the input fw monitor chain as "TCP streaming (cpas)" and in the output chain as "TCP streaming (cpas)" and "TCP streaming post VM (cpas)". Active Streaming – https: With PSL, connection that is encrypted with SSL (TLS) was not supported, the reason for this is that the encryption keys are known only to the Client and Server since they are the one that initiated the connection (preformed the SSL handshake), because of this we couldn’t get the data out of the packet and the application couldn’t scan it for malicious information. CPAS plays the rule of “man in the middle”, because of this, it can intercept the SSL handshake and change the keys so he will be able to understand the encryption. The Client preform an SSL handshake with the gateway (thinking it is the Server) while the Server preform SSL handshake with the gateway (thinking he is the Client). The gateway have both keys and he’s able to open the encryption, check the packet and re-encrypt the packet with the corresponding keys. In order to encrypt / decrypt the SSL connection, CPAS add another layer before the application queue. The new layer will send the packet to the SSL engine for decryption/encryption and then resume the normal flow. Active Streaming – https content step by step: Packets of SSL handshake are passed to the SSL engine to exchange keys. When the connection and the SSL handshake is fully established, an hook will be register for this connection to handle the decrypt / encrypt of the packets. When a packet arrive to CPAS, a trap will be sent and the SSL engine will receive the encrypted packet, decode the packet and return it to CPAS. The packet will enter the receive queue and the application will be able to work on it, once he done he will send it to the write queue. The packet will pass to the SSL engine for encryption and pass to the other side (Client, Server). HTTPS - Inspection Check Point's Active Streaming infrastructure (CPAS) supplies applications above it the means of handling the TCP data stream, without having to care about the implementation below (e.g. 3 way handshake enforcement, retransmissions, checksums, etc.). CPAS, as opposed to PSL, supplies means of modifying the data passed, such as enabling the changing of data, removing parts of it, and even injecting completely new data. These capabilities where once achieved through the Security Servers. This included folding the connection to the firewall and having context switches to and from user-space. Since CPAS is implemented completely in the kernel, there is no more need for context switches, hence a performance increase. In order to inspect HTTPS encrypted traffic, we implement man-in-the-middle on the SSL protocol. The Security Gateway poses as a client with the server, and as a server with the client (for this purpose the Security Gateway creates a fake server certificate with all fields equals to the origin except the public key, issuer, issuer signature and CRL DP). Thus the Security Gateway creates two different sets of keys: one (CF key) to encrypt/decrypt client side traffic, another (SF key) to encrypt/decrypt server side traffic.The HTTPS Inspection RuleBase runs mostly in the kernel. The enforcement is done completely in the kernel, while policy installation is done partially in the kernel and partially in user space. There is no dedicated process. SSL Inspection requires CPAS streaming to be initialized. CPAS (active) streaming is a much more resource-consuming infrastructure than PSL (passive) streaming, therefore we try to avoid CPAS streaming initialization, as soon as possible. In order to avoid CPAS streaming initialization, when we do not really need it, we run HTTPS Inspection rulebase twice on the same connection. The first time it is run on the SYN packet, as part of the Security Rulebase execution. Thus, if HTTPS Inspection Rulebase returns NO MATCH, then SSL Inspection will not influence the steaming type decision (PSL will be used in most cases), but if HTTPS Inspection Rulebase returns MATCH/POSSIBLE MATCH (we cannot make final decision), then active (CPAS) streaming will be initialized. The second time it is run on the Client Hello packet (https_inspection_exe_rulebase function). This time, we make the final match (URL is taken from DN cache or from previous CONNECT request), calculate the action that should be performed (according to action from the matched rule, blades defined on the matched rule and exceptions) and perform the action (INSPECT/BYPASS) on the connection.Connection could be held during HTTPS Inspection rulebase execution due to Identity Awareness (IDA) or RAD/URL Filtering request for a sync execution. HTTPS Inspection supports the following software blades: Application Control, URL Filtering, IPS, Anti-Virus, Anti-Bot, Threat Emulation, Data Loss Prevention (DLP). Unified policies Check Point R80.10, part of Check Point Infinity, takes security to new levels, merging security into a single system which enables administrators to easily identify security risks across the organization. Unified policies in R80.10 gateways enables organizations to translate their security definitions into a simple set of rules, which then streamline policy administration and enforcement throughout the organization. Policy layers provide the ability to separate the policy into independent segments, which can be independently managed and automated. In the kernel the rule base is represented as a database table. This enables the gateway security policy to automatically adapt as dynamic objects change, providing simple automated security across physical, virtual and cloud environments. NGTP Architecture - Session-based processing enforces advanced access control and threat detection and prevention capabilities. To do this we assemble packets into a stream, parse the stream for relevant contexts and then security modules inspect the content. When possible, a common pattern matcher does simultaneous inspection of the content for multiple security modules. In multi-core systems this processing is distributed amongst the cores to provide near linear scalability on each additional core. Security modules use a local cache to detect known threats. This local cache is backed up with real-time lookups of an cloud service. The result of cloud lookups are then cached in the kernel for subsequent lookups. Cloud assist also enhances unknown threat detection and prevention. In particular a file whose signature is not known in a local cache is sent to our cloud service for processing where compute, disk and memory are virtually unlimited. Our sandboxing technology, SandBlast Threat Emulation, identifies threats in their infancy before malware has an opportunity to deploy and evade detection. Newly discovered threats are sent to the cloud database to protect other Check Point connected gateways and devices. When possible, active content is removed from files which are then sent on to the user while the emulation is done. UP Manager - The UP Manager controls all interactions of the components and interfaces with the Context Management Infrastructure (CMI) Loader, the traffic director of the CMI. The UP Manager also has a list of Classifiers that have registered for “first packets” and uses a bitmap to instruct the UP Classifier to execute these Classifier Apps to run on the packet. The “first packets” arrive directly from the CMI. Parsing of the protocol and streaming are not needed in this stage of the connection. For “first packets” the UP Manager executes the rule base. Classifier - When the “first packet” rule base check is complete Classifiers initiate streaming for subsequent packets in the session. The “first packet” rule base check identifies a list of rules that possibly may match and a list of classifier objects (CLOBs) that are required to complete the rule base matching process. The Classifier reads this list and generates the required CLOBs to complete the rule base matching. Each Classifier App executes on the packet and tells the result of the CLOB to the UP Manager. The CMI then tells the Protocol Parser to enable streaming. In some cases Classifier Apps do not require streaming, e.g. the first packet information is sufficient. Then the rule base decision can be done on the first packet. Dynamic Objects Domain Objects Only the firewall is enabled On subsequent packets the Classifier can be contacted directly from the CMI using the CMI Loader infrastructure, e.g. when the Pattern Matcher has found a match it informs the CMI it has found application xyz. The CMI Loader passes this information to the Classifier. The Classifier runs the Classification Apps to generate CLOBs required for Application Control and sends the CLOBs to the Observer. Observer - The Observer decides if enough information is known to publish a CLOB to the security policy. CLOBs are observed in the context of their transaction and the connection that the transaction belongs to. The Observer may request more CLOBs for a dedicated packet from the Classifier or decides that it has sufficient information about the packet to execute the rule base on the CLOB, e.g. if a file type is needed for Content Awareness and the gateway hasn’t yet received the S2C response containing the file. Executing the rule base on a CLOB is called “publishing a CLOB”. The Observer may wait to receive more CLOBs that belong to the same transaction before publishing the CLOBs. Security Policy - The Security Policy receives the CLOB published by the Observer. The CLOB includes a description of the Blade it belongs to so that matching can be performed on a column basis. The security policy saves the current state on the transaction Handle; either to continue the inspection or final match. The first packets are received directly from the UP Manager. Subsequent packets are received by the rule base from the Observer. Handle - Each connection may consist of several transactions. Each transaction has a Handle. Each Handle contains a list of published CLOBs. The Handle holds the state of the security policy matching process. The Handle infrastructure component stores the rule base matching state related information. Subsequent Packets - Subsequent packets are handled by the streaming engine. The streaming engine notifies the Classifier to perform the classification. The Classifier will notify the UP Manager about the performed classification and pass the CLOBs to the Observer. The CLOBs will then be received by the Observer that will need to wait for information from the CMI. The CMI sends the information describing the result of the Protocol Parser and the Pattern Matcher to the Classifier. The Classifier informs the UP Manager and sends the CLOB to the Observer. The UP Manager then instructs the Observer to publish the CLOBs to the Rule Base.The Rule Base is executed on the CLOBs and the result is communicated to the UP Manager. The CLOBs and related Rule Base state are stored in the Handle. The UP Manager provides the result of the rule base check to the CMI that then decides to allow or to drop the connection. The CMI generates a log message and instructs the streaming engine to forward the packets to the outbound interface. R80.10 unified access control policy - defines firewall, application control, URL filtering, mobile access and now content awareness in one policy rule. Content awareness adds visibility and control over data using data types based on content, file types and direction as the content transverses the network. In R80.10 you can also group interfaces of gateways into security zones in source and destination definitions. Core IPS protections are included in the access control policy while other IPS protections are now included in the Threat Prevention policy unifying IPS, Antivirus, Anti-Bot, Threat Extraction, and Threat Emulation into one threat prevention profile. Likewise the threat prevention policy can include multiple profiles for each security gateway to apply more granular Threat Prevention policies. Layers in the R80.10 policy are a new way to organize security. A policy can have one or more layers as its building blocks. You can use combinations of Ordered Layers, Inline Layers, and Domain Layers (in Multi-Domain environments). Build a rule base with layers, each with a set of the security rules. Layers are inspected in the order in which they are defined, giving control over the rule base flow and precedence of security functionality. If an "Accept" action is done in a layer, inspection continues in the next layer. Sub-Policies are sets of rules that you attach to specific rules. If the rule is matched, inspection continues in the sub-policy attached to the rule. If the rule is not matched, the sub-policy is skipped. For example, a sub-policy can manage a network segment or branch office. Unified policy - In R80.10 the unified policy classifies subsequent packets against the column definitions until enough information is known to identify the traffic. Like column-based rule matching as more is known about the connection, then rules are eliminated from the matched rules array. Example classification objects are application, access role source, access role destination, content and file. Applications and Recommended Services- We have seen how R80.10 Service objects and application signatures are combined. Likewise, R80.10 applications now include recommended services such as HTTP and HTTPS in the application definition simplifying the unified policy configuration. For instance the R80.10 Facebook application definition includes the Web services where it is common to see Facebook traffic; HTTP/S over port 80/443 and HTTP/S proxy connections over port 8080. In the UI you can see the services included in the application definition. Content Awareness Content Awareness - (CTNT) is a new blade introduced in R80.10 as part of the new Unified Access Control Policy. Using Content Awareness blade as part of Firewall policy allows the administrator to enforce the Security Policy based on the content of the traffic by identifying files and its content. Content Awareness restricts the Data Types that users can upload or download. Content Awareness can be used together with Application Control to enforce more interesting scenarios (e.g. identify which files are uploaded to DropBox). During policy installation, the CMI Loader collects signatures from multiple sources (e.g. IPS, CTNT, APPI, ...) and compiles them together into unified Pattern Matchers (PM) (one for each context - such as URL, Host header, etc.). Picture from Content Awareness SK. The CMI process calls for Content Awareness blade for install policy process: Read data types and other settings information retrieved from the Security Management server. Registered to FileApp contexts. Load all Data Types signatures and CPcodes into CMI. Get the latest list of supported file types. Download all policy information to the kernel. Afterwards, FileApp parser updates the File Signatures (file type identification method) to its latest version to support new file types. FileApp - is able to extract text from Microsoft Office 2007 and above files (.docx, .pptx, .xlsx, ...), .zip/.gzip and text files. For other file types (such as .pdf), FileApp sends the file to User Mode for extraction. Anti-Bot and Anti-Virus Malware detection (AntiBot and AntiVirus) - Check Point's comprehensive Threat Prevention solution offers a multi-layered, pre- and post-infection defence approach and a consolidated platform that enables enterprise security to deal with modern malware: Anti-Virus - Pre-infection detection and blocking of malware at the gateway. The Anti-Virus Software Blade is continuously updated from ThreatCloud. It detects and blocks malware by correlating multiple detection engines before users are affected. Anti-Bot - Post-infection detection of bots on hosts. Prevents bot damages by blocking bot C&C (Command and Control) communications. The Anti-Bot Software Blade is continuously updated from ThreatCloud, a collaborative network to fight cybercrime. Anti-Bot discovers infections by correlating multiple detection methods. AntiBot - A bot is malicious software that can invade your computer. There are many infection methods. These include opening attachments that exploit a vulnerability and accessing a web site that results in a malicious download. One bot can often create multiple threats. Bots are frequently used as part of Advanced Persistent Threats (APTs) where cyber criminals try to damage individuals or organizations. The Anti-Bot Software Blade detects and prevents these bot and botnet threats. A botnet is a collection of compromised and infected computers. The Anti-Bot Software Blade uses these procedures to identify bot infected computers: Identify the C&C addresses used by criminals to control bots - These web sites are constantly changing and new sites are added on an hourly basis. Bots can attempt to connect to thousands of potentially dangerous sites. It is a challenge to know which sites are legitimate and which are not. Identify the communication patterns used by each botnet family - These communication fingerprints are different for each family and can be used to identify a botnet family. Research is done for each botnet family to identify the unique language that it uses. There are thousands of existing different botnet families and new ones are constantly emerging. Identify bot behavior - Identify specified actions for a bot such as, when the computer sends spam or participates in DoS attacks. AntiVirus - Malware is a major threat to network operations that has become increasingly dangerous and sophisticated. Examples include worms, blended threats (combinations of malicious code and vulnerabilities for infection and dissemination) and Trojans. The Anti-Virus Software Blade scans incoming and outgoing files to detect and prevent these threats. It also gives pre-infection protection from malware contained in these files. The Anti-Virus Software Blade: Identifies malware in the organization using the ThreatSpect engine and ThreatCloud repository: Prevents malware infections from incoming malicious files types (Word, Excel, PowerPoint, PDF, etc.) in real-time. Incoming files are classified on the gateway and the result is then sent to the ThreatCloud repository for comparison against known malicious files, with almost no impact on performance. Prevents malware download from the internet by preventing access to sites that are known to be connected to malware. Accessed URLs are checked by the gateway's caching mechanisms or sent to the ThreatCloud repository to determine if they are permissible or not. If not, the attempt is stopped before any damage can take place. Uses the ThreatCloud repository to receive binary signature updates and query the repository for URL reputation and Anti-Virus classification. ThreadSpect Engine and ThreadCloud - The ThreatSpect engine is a unique multi-tiered engine that analyzes network traffic and correlates information across multiple layers to find bots and other malware. It combines information on remote operator hideouts, unique botnet (a collection of compromised computers) traffic patterns and behavior to identify thousands of different botnet families and outbreak types. Reputtion Layer - Analyzes the reputation of URLs, IP addresses and external domains that computers in the organization access. The engine searches for the known or suspicious activity, such as Command and Control (C&C).The Reputation layer classifies per connection: IP address - from handle first packet before security rule base. Only in kernel, not in the cloud. DNS - host from DNS request (for both TCP and UDP) URL - complete URL from the HTTP request After the discovery of bot infected machines, the Anti-Bot Software Blade blocks outbound communication to C&C sites based on the Rule Base. This classification has 3 stages: First, every URL is searched in the local database, located in $FWDIR/conf/urlrep.eng file that contains some malware data - commonly used signatures, URLs, and their related reputations.Local database is loaded to the kernel, and compiled to one Pattern Matcher (PM) that is executed on the incoming URLs to find a match.If the URL is found there, a response to the client is returned.If there is no match against local database, continue to next Step. RAD (Resource ADvisor) cache (for DNS and URL only), that gives answers to 99% of URL reputation requests. RAD (Resource ADvisor) service in the cloud (for DNS and URL only) If the URL was not found in the cache, a request is sent out to RAD and a response is returned (asynchrony) back to the client, and the relevant data is kept in the cache. Anti-Bot Resource Classification mode for DNS is overridden to "background" on the Security Gateway. These settings override the existing default settings in the SmartDashboard. The Resource Classification settings for Anti-Bot are now configurable from the Security Gateway using the malware_config file. Using the Security Gateway configuration file provides more granularity and divide classification to DNS, HTTP and SMTP. There are three values for each option: hold - for hold classification bg - for background classification policy - for classification according to Security policy The default values for HTTP and SMTP is policy, but for DNS the default is now background. Application Control The use of internet applications comes with risks that administrators must know about: Malware threats: Application use can open networks to threats from malware. Popular applications like Twitter, Facebook, and YouTube can cause users to download viruses unintentionally. File sharing can easily cause malware to be downloaded into your network. Bandwidth hogging: Applications that use a lot of bandwidth, for example, streaming media, can limit the bandwidth that is available for important business applications. Loss of Productivity: Employees can spend time on social networking and other applications that can seriously decrease business productivity. Employers do not know what employees are doing on the internet and how that really affects them. The Check Point Solution for Application Control Check Point's latest firewall innovation brings the industry's strongest application and identity control to organizations of all sizes. You can easily create policies which detect or block thousands of applications. Granular Application Control: Identify, allow, or block thousands of applications. This provides protection against the increasing threat vectors and malware introduced by internet applications. Application control and Custom applications signatures are kept in the Application Database (APP DB), based on two files, C and appi_custom_db.C, located on the Security Gateway. User defined URLs and HTTPS Inspection User defined URLs data is kept in the Security Gateway database. We integrated URL Filtering together with Application Control, and both features are set in the Application Control engine (Application Control) rulebase.URL Filtering is represented by a new CMI application client: URL Filtering. This client behaves like an Application Control client. It uses Application Control DB for metadata, but instead of using signatures, it uses the application detection engine (RAD) service. (RAD (Short for Resource Advisor) is responsible for the detection of Social Network widgets, e.g Mafia Wars).) URL Filtering Check Point's provied strongest URL Filtering, application and identity control to organizations of all sizes. You can easily create policies which detect or block thousands of applications and internet sites. The URL Filtering software blade uses a hosted categorization service (cloud-based categorization) that is being constantly updated to cope with the dynamic nature of the web. The Security Gateway is not required to download updates, the categorization database is not limited by the storage space on the Security Gateway, and customers receive the latest categorization to keep policies current. As requests are being sent to the cloud, responses are being cached locally on the Security Gateway so that subsequent requests for the same sites will not query the cloud, but will be answered by the local cache. In deployments of R75.20 URL Filtering, we have seen that more than 99% of requests are being met by the cache, providing customers with the most up-to-date categorization (with the cloud) on the one hand and minimal latency (as most requests are being met by the cache), on the other.When there are critical updates, for example, a previously benign site has been hacked and is now used to spread malware, the cache is invalidated and forced to retrieve updates from the cloud. Custom Applications, Sites, Categories and Groups -You can create applications, websites, categories and groups that are not in the Application and URL Filtering Database for use in the policy. Use these custom objects to create a Rule Base that meets your organization's requirements. It is also possible to contact Check Point to create customized application signatures that can be imported into the database. This file can contain, for example, a database with an organization's internal applications that are not necessarily web-based. URL Filtering - The URL Filtering blade uses a single user mode process called RAD ($FWDIR/bin/rad) to communicate with the URL Filtering service. The process is initiated once application control, or URL filtering blades, are active on the Security Gateway. Whenever RAD successfully categorizes a URL, it caches it in its kernel cache. The URL Filtering kernel cache holds the entire URL Family (i.e. all the URLs within the same host). Whenever there is a request for a new URL categorization, RAD sends to the categorization service only the host part of the URL and receives the entire URL family. The response is then cached for future use. Web categorization - You can select the Web categorization mode that is used for web site categorization: Background: requests are allowed until categorization is complete. When a request cannot be categorized with a cached response, an uncategorized response is received. Access to the site is allowed. In the background, the Check Point Web Service continues the categorization procedure. The response is then cached locally for future requests (default). This option reduces latency in the categorization procedure. Hold: requests are blocked until categorization is complete. When a request cannot be categorized with the cached responses, it remains blocked until the Check Point Web Service completes categorization. IPS Starting from R80, IPS Software Blade is part of Threat Prevention solution. Check Point IPS is an Intrusion Prevention System (IPS). IPS is a complete IPS cyber security solution, for comprehensive protection against malicious and unwanted network traffic, which focuses on application and server vulnerabilities, as well as in-the-wild attacks by exploit kits and malicious attackers. The IPS Software Blade delivers complete and proactive intrusion prevention. It delivers 1,000s of signatures, behavioral and preemptive protections. It gives another layer of security on top of Check Point firewall technology. IPS protects both clients and servers, and lets you control the network usage of certain applications. The hybrid IPS detection engine provides multiple defense layers which allows it excellent detection and prevention capabilities of known threats, and in many cases future attacks as well. It also allows unparalleled deployment and configuration flexibility and excellent performance. Elements of protection - included in IPS protection: Detection and prevention of specific known exploits. Detection and prevention of vulnerabilities, including both known and unknown exploit tools, for example protection from specific CVEs. Detection and prevention of protocol misuse which in many cases indicates malicious activity or potential threat. Examples of commonly manipulated protocols are HTTP, SMTP, POP, and IMAP. Detection and prevention of outbound malware communications. Detection and prevention of tunneling attempts. These attempts may indicate data leakage or attempts to circumvent other security measures such as web filtering. Detection, prevention or restriction of certain applications which, in many cases, are bandwidth consuming or may cause security threats to the network, such as Peer to Peer and Instant Messaging applications. Detection and prevention of generic attack types without any pre-defined signatures, such as Malicious Code Protector. Check Point constantly updates the library of protections to stay ahead of emerging threats.Capabilities of IPS - The unique capabilities of the Check Point IPS engine include: Clear, simple management interface. Reduced management overhead by using one management console for all Check Point products. Integrated management with SmartConsole. Easy navigation from business-level overview to a packet capture for a single attack. Up to 15 Gbps throughput with optimized security, and up to 2.5 Gbps throughput with all IPS protections activated. First security coverage for Microsoft and Adobe vulnerabilities. Resource throttling so that high IPS activity will not impact other blade functionality. Complete integration with Check Point configuration and monitoring tools in SmartConsole, to let you take immediate action based on IPS information. For example, some malware can be downloaded by a user unknowingly when he browses to a legitimate web site, also known as a drive-by-download. This malware can exploit a browser vulnerability to create a special HTTP response and sending it to the client. IPS can identify and block this type of attack even though the firewall may be configured to allow the HTTP traffic to pass. PS uses the CoreXL technology which leverages multi-core. Each firewall instance can run on a separate core and many instances can run in parallel. The IPS code runs in a monolithic part of the firewall kernel. Each such per-core instance runs firewall and IPS and Dispatchers are bound to a network interface using IRQ affinity. IPS use the 4 layers: Passive Streaming Library (PSL) Protocol Parsers Context Management Interface layer (CMI) Protections Based on the IPS policy, the CMI determines which protections should be activated on every context discovered by a protocol parser. If policy dictates that no protections should run, then the relevant parsers on this traffic are bypassed in order to improve performance and reduce potential false positives. For example, if the IPS policy is activating server protections only, then the HTTP parser will not analyze the server to client (S2C) traffic. When a protection is activated, it can decide whether the given packet or context is OK or not. It does not decide what to do with this packet. The CMI is responsible for the final action to be performed on the packet, given several considerations.The considerations include: Activation status of the protection (Prevent, Detect, Inactive) Exceptions either on traffic or on protection Bypass mode status (the software fail open capability) Troubleshooting mode status Are we protecting the internal network only or all traffic Threat Emulation (SandBlast) Threat Emulation (SandBlast) can protect your network against new malware, zero-day vulnerabilities and targeted attacks with Sandbox technoligies. SandBlast customers with Check Point gateway R77 or higher can offload Threat Emulation to a dedicated SandBlast Appliance (TEX) or ThreatCloud. Threat Emulation - gives networks the necessary protection against unknown threats in files that are downloaded from the Internet or attached to e-mails. When emulation is done on a file: The file is opened on more than one virtual computer with different operating system environments. The virtual computers are closely monitored for unusual and malicious behavior, such as an attempt to change registry keys or run an unauthorized process. Any malicious behavior is immediately logged and you can use Prevent mode to block the file from the internal network. The cryptographic hash of a new malicious file is saved to a database and the internal network is protected from that malware. Information about malicious files and malware is shared with the Check Point ThreatCloud (administrator can disable this) and helps protect all ThreatCloud users. Sandboxing: Inspect all incoming files to an organization and send them to a safe virtual environment Open the files in the virtual environment to see what would happen if someone inside of the organization would open this file Prevent the malicious files from entering the organization CPU-Level Emulation - Monitoring CPU buffer and finding exploits to preventing malware before it executes: New leading edge technology unique to Check Point Next generation of sandboxing to catch more malware No guesswork required, detection is definitive Not based on heuristics or statistics Resistant to evasions Fast and effective Supported on Threat Emulation appliances (TE100X, TE250X, TE1000X, TE2000X) OS-Level Emulation - is a Traditional sandboxing technology whit will give a very good emulation report. It is most times effective, but are susceptible to evasions and longer you emulate the file, the better catch rate you will get. It works by: Monitoring System Registry Monitoring Network Connections Monitoring File System Activity Monitoring System Processes SandBlast emulation on gateway components: DLPU - File arrives at the kernel and we decide whether we need to scan it or not (according to the policy). The file is sent to the correlating DLPU process instance in the user space. The DLPU process handles the file reassembly. Once DLPU finishes file reassembly, it is send it to TED (Threat Emulation Daemon). Threat Emulation Daemon (TED) - Receives the complete file and processes it through file type checks to understand if emulation is needed. Next steps: Checks cache if the file was already emulated. Checks system resources (CPU/Memory) to create an emulation queue if needed. Static analysis Executes emulation according to policy settings Collects forensics details from the VM activity agent Collects statistics of the emulation environment Local logging/reporting and shares data with ThreatCloud Handling Mode - While Threat Emulation processes the file, it can apply the following handling modes: Background - The connection over HTTP / SMTP is allowed and the file is sent to the destination even if the Threat Emulation analysis is not finished. Hold - A connection over HTTP / SMTP that must have emulation is blocked and Threat Emulation holds the file until the Threat Emulation analysis is finished (default minimum is 60 sec). Custom - Allows configuration of "Background" and "Hold" modes independently for HTTP and SMTP protocols. ThreatCloud: When sending files to the Check Point ThreatCloud: Security Gateway detects that a file was received from the Internet or an external network. The Security Gateway compares the cryptographic hash of the file with the database. Security Gateway encrypts the file and sends (over an SSL connection) to the ThreatCloud. Frontend servers at the ThreatCloud Pod perform Support Contract verification against Check Point User Center. ThreatCloud Pod transfers the file (over an SSL connection) to a Check Point Emulator located on a dedicated protected Check Point site. Check Point Emulators decrypts the file and runs emulation on the file. After this Check Point Emulator sends a report (over an SSL connection) to a ThreatCloud Pod, which saves it in the shared database. Afterward ThreatCloud Pod sends a report (over an SSL connection) to the customer's Security Gateway for the applicable action. Local Threat Emulation Appliance: When emulating files on the local Threat Emulation Appliance (TE100X, TE250X, TE1000X, TE2000X/HPP) installed on your network: The Local Emulation appliance receives the files for emulation. The Local Emulation appliance compares the cryptographic hash of the file with the database. The file is inspected in Threat Emulation engine, including static analysis, resource checking on the appliance to see that the appliance can accommodate the scanning, checking the metadata of the file for intelligence and then emulating the file in the sandbox environment. Afterward the file is emulated according to policy on the relevant OS. During the emulation, there are behaviour indicators that allow Threat Emulation to determine whether the file is malicious or benign. When the investigation ends, a verdict is returned to sending Security Gateway. Short excursion to the local emulaton: DLPU + DLP kernel module - The parsers (Web, Mail) whitch deeply inspect the connections for traffic. Afterwards the DLPU (DLP Kernel module), witch transfers the file parts to the DLPU daemon (one per CoerXL instance) and the DLPU process reassamble the file and write it to „$FWDIR/tmp/te“. Cache - The cache is very lightweight, but has an enormous positive effect on the performance. Static Analysis - The static analysis is composed of python processes, whitch is aimed on skipping emulation of files that we can be sure with very high level of confidence, that they don’t contain malware. It is relatively heavy in IO. Emulation (qemu) - The emulation ist he most recource intensive process in the system and runs the qemu based virtuel machines as well as the fake internet server (python). MTA The MTA can run on a SandBlast appliance or on a gatway (cluster). Traffic to the MTA can easily conntroled by the firewall or Sandblast appliance. The MTA works as next hop between Internet mail servers and internal Mail Server. It does not use PSL but works as a real MTA. A PostFix server receives and handles the emails. Emails are forwarded to the in.emaild.mta daemon, which parses the emails and passes the attachments to the scrubd or TED process, if needed. The scrubd or TED process returns the mail to the in.emaild.mta daemon. The in.emaild.mta daemon forwards the email to its destination. scrubd - The scrubd daemon pro-actively cleans potential threats from incoming documents (Threat Extraction). The scrubd process returns the Safe copy of the original threat. scrub_cp_file_convertd - Used to convert-to-PDF, or extract potentialy malicious content from various file formats. Parser in CoreXL FW Instance -> Postfix -> in.emaild.mta -> TED -> in.emaild.mta -> Postfix: More coming soon. R80.20 Falcon Acceleration Path Now we have three new acceleration path in R80.20 Falcon cards: Host Path - No acceleration connections (eg. local connections) and connections on now-AC interface. Buffer Path - HTTP request, HTTP response header and TSL handshakes Inline Path - HTTP response body (until 1 tier match) and TSL bulk encryption and decryption. More coming soon. Other Security Blades Coming soon: DLP References SecureKnowledge: Application Control SecureKnowledge: URL Filtering SecureKnowledge: Content Awareness (CTNT) SecureKnowledge: IPS SecureKnowledge: Anti-Bot and Anti-Virus SecureKnowledge: Threat Emulation SecureKnowledge: Threat Extraction SecureKnowledge: Check Point Active Streaming (CPAS) and Passive Streaming Layer (PSL)SecureKnowledge: HTTPS Inspection FAQ SecureKnowledge: Best Practices - Security Gateway PerformanceDownload Center: R80.10 Next Generation Threat Prevention PlatformsDownload Center: R77 Security Gateway Packet FlowDownload Center: R77 Security Gateway ArchitectureSupport Center: Check Point Security Gateway Architecture and Packet Flow Checkmates: Check Point Threat Prevention Packet Flow and ArchitectureCheckmates: Infinity NGTP architecture R80.x Security Gateway Architecture (Logical Packet Flow)Checkmates: R80.x Security Gateway Packet Flow and Acceleration - with Diagrams Versions EA Version: 0.1f - final version (08-15-2018)0.1g - add "https inspection" (08-15-2018)0.1h - add "Anti-Bot and Anti-Virus" +change flow colors (08-16-2018)0.1k - add more pictures (08-20-2018)0.1h - add "CPAS" (08-20-2018)0.1i - add "IPS, AC, URLF" (08-21-2018)0.1j - add "TE" (08-25-2018)0.1k - bug fixed (08-28-2018)0.1l - add local Threat Emulation (09-03-2018)0.1m - bug fixed (09-06-2018)0.1n - QoS (09-24-2018)1.1o - correct a mistak in picture (09-26-2018)1.1p - add CPASXL and PSLXL path in R80.20 (09-27-2018)1.1q - add Falcon acceleration path in R80.20 (10-09-2018)1.1r - bug fixed Falcon card wording (11-04-2018)1.2a - new forum format (03-17-2019) Copyright by Heiko Ankenbrand 1994-2019
