R82 Interesting Findings

[Timothy_Hall]

Been looking under the hood at R82 GA all morning, here are a few things I've uncovered.  I'll try to avoid restating findings that are already in the release notes, also keep in mind these are initial impressions which may not be 100% accurate.

1) All NIC drivers (igb/ixgbe/vmxnet3/mlx_core etc) are taking a major driver version update due to the new 4.18 kernel, this may result in changes in network interface behavior upon upgrade (like this when we moved to kernel 3.10: sk166424: Number of RX packet drops on interfaces increases on a Security Gateway R80.30 and higher ...).  The e1000 driver (which is the default interface in many virtualized environments) is deprecated in R82 and will throw a console warning if you are trying to use it.  The e1000 driver still seems to work anyway on an SMS in VMWare, but does NOT work for me on a security gateway in VMWare so look out.  sk180572: Hardware Compatibility Notifications

2) If you create a new R82 security gateway or cluster, Anti-Bot and Anti-Virus will be enabled by default.  These features under R82 now also support a kind of hybrid between "Background" and "Hold", whereas if Hold is causing excessive latency connections start slowly getting treated as background in a very interesting trade-off (this hybrid mode doesn't appear to be enabled by default): sk181434: "Maximum Security" feature in Anti-Virus and Anti-Bot R82 and higher

3) The Advanced DNS protections introduced in R81.20 have been exposed somewhat in the TP profile settings, but it looks like all you can do is turn them off for testing purposes, not sure if exceptions could be created for these:

4) A new "Dynamic Layer" of rules can be added directly on a gateway even if the SMS is unreachable, kind of like SAM rules on steroids: sk182252: Dynamic Layer in Access Control Policy.  Also it looks like the R82 gateways now cache multiple versions of previously installed policies so you can revert to an older known-good policy directly on the gateway itself, kind of like the Installation History screen in SmartConsole.  sk181437: Access Control Policy Revert Tool (policy_rev_tool)  Could be very handy when a policy install causes all hell to break loose including cutting off the gateway from the SMS, to recover you could just jump directly onto the gateway and reinstall the known-good policy that was installed just before the "bad" one was, as opposed to fw unloadlocal causing a full outage.

5) Looks like a new daemon called cpview_api_service has been added to R82 according to sk97638, I assume this would allow cpview data to be retrieved directly via API calls which opens up some verrrry interesting monitoring/reporting capabilities.

6) Looks like the ips command has gotten pimped out with some new options beyond just looking up the name of a protection ID you see in a debug, appears you now can switch off individual IPS protections right from the gateway's CLI without a policy install.  Guess Check Point got tired of administrators running ips off and gave them some more granular options:

ips protections print_name <protection_id (hex)> # Display protection name
ips protections print_id '<protection name>' # Display protection ID
ips protections disabled show # Show IPS disabled and excluded protections
ips protections disabled exclude <protection_id> # Exclude IPS disabled protection
ips protections disabled delete_exclude <protection_id>|all # Delete excluded IPS disabled protection

7) New command called turbostat shows very detailed information about the processors such as power consumption, current clock rate and lots of other goodies.  @Bob_Zimmerman would approve.

😎 Looks like we picked up two new paths in the output of fwaccel stats -s, presumably for Hyperflow boosting of QUIC and SMB/CIFS.  Also the connection_pipelining command has been renamed hyperflow.

UDP IS XL pkts/Total pkts : 0/541805 (0%)
UDP IS pipeline pkts/Total pkts : 0/541805 (0%)

9) The great new connview tool requires USFW to function, it doesn't work if your firewall is in kernel mode.

10) The classic Unix command strace is back in Gaia (although it was previously available via the undocumented perf trace command).  This great command can be used to sniff the system calls between an existing  runaway/malfunctioning process and the kernel.  You can see everything it is trying to access (files, sockets, etc) and even the data it is reading/writing.  Awesome troubleshooting tool for processes that can be attached to a process that is already running.

11) All the configuration options for HTTPS Inspection including CA updates and such are now located in the main SmartConsole GUI and no longer in the legacy SmartDashboard which means they have been extricated from the vile clutches of fwm, speaking of which...

12) It seems that while the legacy single-threaded fwm process still exists in R82, it has had even more responsibilities taken away from it on the SMS which is a good thing.  Policy installations seem WAY faster in R82 and watching the output of top during a policy installation shows fwm is doing very little now.  In R81.20 and earlier fwm is always the major bottleneck for policy installation performance, even for accelerated ones.

13) Enhanced VPN Link Selection looks very nice, should make setting up ISP-redundant VPNs with non-Check Point gateways much easier.

[Bob_Zimmerman]

Very nice!

1) Great chance to remind everybody that e1000 has been a mediocre option in VMware for a long time. vmxnet3 is much better.

4) Important note in that Dynamic Layer SK: "It is not supported to edit or delete individual dynamic rules on the Security Gateway after you add them." This seems like a really, really good way to shoot yourself in the foot. Use with extreme caution.

6) Really looking forward to more granular ways to work with IPS on the command line. I'd love the ability to take a flow tuple and get a list of which IPS protections it will hit so I can target just those ones.

7) turbostat(8) has actually been included at least since the move to kernel 3.10. I don't have any 2.6 systems handy to see if it's present that far back. It's a nice tool for monitoring processor performance, though I wish it had some kind of structured output like JSON instead of the top-like table. Unfortunately, Linux still doesn't have great tools for monitoring cross-NUMA-node traffic, so it's still hard to notice the QPI/UPI links are saturated. Intel's PCM (Performance Counter Monitor) is available, but not included in any Linux I'm aware of. Definitely not included in Gaia.

[the_rock]

Yes, agree there, vmxnet3 is much better.

Andy

[PhoneBoy]

You went full board into twiddling the nerd knobs 🙂

The Dynamic Layer stuff is basically a definition for a policy layer via a JSON file and a gateway can support multiple of them.
The policy pushed to the gateway must include the Dynamic Layers in order to be used.
Dynamic Layers are not meant to be manipulated in the same way a regular policy is.
The actual details of what you can do in this JSON are in the Gaia API documentation.
As I type this, it appears the v1.8 documentation is NOT public.
However, if you have R82 installed (even the EA), you can read the documentation here: https://mgmt-ip/gaia_docs/#web/set-dynamic-content~v1.8%20   

From my read of this documentation, you can set a policy using objects defined in the JSON (most object types are supported).
Some types of objects can be "externally referenced" (taken from management), specifically:

• Services
• Custom Application/Sites
• Application Categories
• Access Layers
• Updatable Objects

The externally referenced objects used in the policy must be explicitly specified in the JSON along with the ones you're using in the policy.
In fact, any changes to the dynamic layer must be done with a new JSON file (replacing the existing one).

It's a bit more complicated than sam rules for sure, but you can do a whole lot more with it, it appears.

[Timothy_Hall]

Looks like in R82 we can FINALLY on a per-gateway basis tell it which IP address of the SMS (real or NAT) to initiate the logging connections to and also for fetching policies.  (sk171055: Security Gateway tries to fetch the policy from / sends logs to the real IP address of the...) 

This replaces creating those clunky fake log server objects when there was a mixture of gateways managed across the Internet and some through an internal WAN, and is a huge improvement:

[the_rock]

Thanks for pointing that out, very nice!

Andy

[HeikoAnkenbrand]

Hi @Timothy_Hall,

You can find more changes between the releases (R81.20 and R82) in the following SK:

sk180180 - Software changes between Quantum release versions 

CUT>>>

• Management Server
  • Security Gateways R77.30 are not supported.
  • The search in SmartConsole Object Explorer and "Objects" sidebar was improved in a specific scenario. The partial search in text fields (name of an object, comment, and so on) does not require entering the wildcard character "*"(asterisk) anymore. See sk182006.
    
    
• Gaia Operating System
  • Updated the Gaia OS Linux kernel version to 4.18.
  • CPView Utility saves its log messages in these files:
    • On a Management Server / Log Server / Security Gateway:
      • $CPDIR/log/cpviewd.elg
      • $CPDIR/log/cpview_api_service.elg
    • On a VSX Gateway:
      • $CPDIR/log/cpviewd.elg.vs<VSID>
      • $CPDIR/log/cpview_api_service.elg.vs<VSID>
  • Added the Python v3.11 package.
  • Introducing a dedicated messaging daemon MSGD
  • You can use the Gaia Clish command "set dns timeout <value>" to control how long Gaia OS waits for a response from a DNS server before it sends the DNS request to the next configured DNS server.
  • The log files in the $RTDIR/laas/adjuster_service/log/ directory moved from the root partition "/" to the "/var/log/" partition.
  • More user space log files are now rotated based on the settings in the /etc/cpshell/log_rotation.conf configuration file.
  • The name template of a Gaia regular backup file changed:
    from "backup_--_<HostName>.<Domain>_<DD>_<MM>_<YYYY>_<HH>_<MM>_<SS>.tgz"
    to "backup_--_<HostName>.<Domain>_<YYYY>_<MM>_<DD>_<HH>_<MM>_<SS>.tgz"
  • The name template of a Gaia scheduled backup file changed:
    from "backup_-<Name_of_Scheduled_Backup>-_<HostName>.<Domain>_<DD>_<MMM>_<YYYY>_<HH>_<MM>_<SS>.tgz"
    to "backup_-<Name_of_Scheduled_Backup>-_<HostName>.<Domain>_<YYYY>_<MM>_<DD>_<HH>_<MM>_<SS>.tgz"
  • User Space Firewall (USFW) is now enabled by default on all environments except Threat Emulation (TE) Appliances and Standalone setup.
  • Default disk space limit for storing core dump files was increased:
    • Management Server - from 1000 MB to 5000 MB
    • Security Gateway in the Kernel Space Firewall (KSFW) mode - from 1000 MB to 5000 MB
    • Security Gateway in the User Space Firewall (USFW) mode - from 10000 MB to 15000 MB
• VSX
  • In the Legacy VSX mode, the default value for concurrent connections in the Virtual System object was increased from 15,000 to 50,000 (Optimizations section > Capacity Optimization page).
  • In the VSNext mode, the Expert mode command "clish -c" now supports the context of a Virtual Gateway / Virtual Switch with this syntax:
    clish -v <Virtual Device ID> -c "<Gaia Clish Command>"
    
    
• VPN
  • When a Check Point Management Server creates an IKE certificate, by default this certificate contains the "Server Authentication" attribute within the "Extended Key Usage" field.
  • Changed the default value of "Maximum concurrent IKE negotiations" from 1,000 to 10,000 in the Security Gateway / ClusterXL object > the "Optimization" page.
  • Changed the default value of cphwd_medium_path_qid_by_mspi parameter from 1 to 0 and cphwd_medium_path_qid_by_cpu_id parameter from 0 to 1.
    
    
• Quantum Maestro, Scalable Chassis, and ElasticXL
  • Newly added Scalable Platform Member always clones the image from the SMO Member, regardless of the SMO Image Cloning state.
  • Outputs of CLI commands were unified to use the same terms on an ElasticXL Cluster, a Maestro Security Group, and a Scalable Chassis:
    • "Site" (instead of "Chassis")
    • "Member" (instead of "SGM")
  • The feature name changed from "Unique IP Address per Chassis" (UIPC) to "Unique IP Address per Site" (UIPS).
  • On the Maestro Orchestrator MHO-175 ports, increased the default MTU size from 9216 to 10240 bytes.
  • Automated creation of the management bond interface (MAGG). All management interfaces assigned to a Security Group are automatically assigned to this MAGG interface.
  • If an administrator stops a Maestro Orchestrator with the "orchd stop" command (or reboots it), and the Orchestrator detects that other Orchestrators on the Maestro Site are not operational, then before stopping (or rebooting) the Orchestrator shows a warning and a prompt to the administrator.
  • When an administrator changes the administrative state of a port on a Maestro Orchestrator, this change now survives an Orchestrator reboot and the restart of the Orchestrator daemon with the "orchd restart" command.
  • On the Orchestrator, the Gaia Portal > Network Management section > Network Interfaces page now hides interfaces that are used for internal purposes:
       Sync-ext
       Sync-int
       dl<number>
       eth<number>
       eth<number>-CIN<number>
       swid0_eth
  • On the Maestro Orchestrator, it is no longer supported to convert an existing Security Group from the 'Gateway' mode to the 'VSX' mode (by selecting the corresponding checkbox in the Security Group properties).
  • The output of the Gaia gClish / Gaia Clish command "show interfaces" on Scalable Platforms was aligned with the output of this command on a regular Security Gateway.
  • These CLI commands were deprecated and replaced (where possible):
    

    Deprecated Command	Use this Command in the Expert mode	Use this Command in Gaia gClish
    asg cluster_site_admin	cluster_site_admin -c <Site ID - 1 or 2> {down | up}	set cluster site-id <Site ID> admin-state {up | down} set cluster sites-admin-state id <Site ID> {down | up}
    asg conns	insights cluster-cli show connection --help g_connview --help	insights show cluster info connection <parameter>
    asg cores_stat	insights cluster-cli show cpu	insights show cluster info cpu
    asg diag asg_diag asg6 diag	insights hcp --help (run the applicable tests)	insights
    asg if asg_if asg6 if	insights hcp --help (run the applicable tests) cluster-cli show interfaces	insights show cluster info interfaces
    asg perf	insights cluster-cli show --help	insights show cluster info <parameter>
    asg resource asg6 resource	insights cluster-cli show --help	insights show cluster info <parameter>
    asg search asg6 search	insights cluster-cli show connection --help	insights show cluster info connection <parameter>
    asg_bond	hcp --help (run the "Bond Health" test)	N / A
    asg_chassis_admin	cluster_site_admin -c <Site ID - 1 or 2> {down | up}	N / A
    toggle_same_vmac	toggle_same_vmac_os	N / A
    show smo set smo delete smo		show cluster <parameters> set cluster <parameters>
    show chassis id {1|2} general unique_ip set chassis id {1|2} general unique_ip delete chassis id {1|2} general unique_ip	N / A	show cluster configuration unique-ip <Site ID> interface <parameters> set cluster configuration unique-ip <Site ID> interface <parameters> delete cluster configuration unique-ip site-id <Site ID> interface <parameters>
    show chassis high-availability <parameters> set chassis high-availability <parameters>	N / A	show cluster configuration high-availability <parameters> set cluster configuration high-availability <parameters>
    asg_collect_vsx_logs	cpinfo -h (see sk92739)	cpinfo -h (see sk92739)
    drop_monitor	N / A	N / A
    asg_affinity_enhance	N / A	N / A

  
  
  
• Security Gateway
  • In the feature "Hide NAT behind IP Address Range", it is now possible to configure the Security Gateway to select the Hide NAT IP address based on the combination of the source IP address and the source port. See sk105302.
  • Improved the output of the adlogconfig command. See the R82 CLI Reference Guide.
  • In the Threat Prevention Engine Settings, the default "Connection Unification" period changed from 600 minutes to 180 minutes (in SmartConsole, click "Manage & Settings" > "Blades" > in the "Threat Prevention" section, click "Advanced Settings" > click the "General" page).
    
    
• Mobile Access
  • Changed the default value of the "max_concurrent_vpn_tunnels" parameter from 200 to 10000 in VSX environments.
    
    
• QoS
  • QoS policy now supports different Service objects with the same Destination Port and different Source Ports.
    
    
• SmartConsole
  • Upgraded the SmartConsole .NET Framework from 4.5 to 4.8.
  • Upgraded the SmartConsole Visual C++ Redistributable from 2012 to 2019.
  • Hovering over the SmartConsole icon on the Windows OS taskbar now shows the SmartConsole version in the tooltip in this format:
    <IP_Address>-<Version>-SmartConsole
  • The "HTTPS Inspection" tab was removed from the Legacy SmartDashboard.

<<<CUT

[HeikoAnkenbrand]

Monitoring dashboard for Scalable Platforms (ElasticXL Cluster, Maestro, Scalable Chassis).

# insights

[Timothy_Hall]

Just realized R82 is actually missing something, the annoying "Security Tip" Infinity services popups that could only be disabled for a month seem to be gone, or at least I haven't run into them yet on R82.  Also the "why did you disable this Threat Prevention blade" popups don't seem to be present any more either. 

This is a good thing, has anyone else encountered any unexpected and/or annoying SmartConsole popups in R82?  The disk space warning popup for /opt is still there (which is fine); the only other use of SmartConsole popups I'd feel is appropriate would be issues such as the VPN Information Disclosure vulnerability awhile back.  However it looks like another security vendor has had far more serious issues than just warning popups could cover over the last week...

[the_rock]

Yea, I totally forgot about it Tim, but I had not seen any of those either in my R82 lab so far.

Andy