CloudGuard: Automated firewall Cluster Deployment with auto-scaling option

[Nicolas_Boisse]

If you are playing with the API's, you will realise there is no API call yet available for Cluster Deployment. In the meantime, with little help from R&D, we've created this automation script: "vsecClusterObject.sh"

The script run from the management server and as many functions available. We leverage DBEDIT code and API Calls to help automate the cluster deployment and auto-scaling.

Here the function available:

# createClusterObject   (4 variables needed):
This will create the cluster object: CreateClusterObject Cluster_Name Cluster_IP SYNC_Network SYNC_Netmask

EX:

./vsecClusterObject.sh createClusterObject vSECCluster 192.168.1.14 1.1.1.0 255.255.255.0

# Adding Member 1:

# createMemberObject (8 Variables):
This will add member 1 into the cluster object

createMemberObject Cluster_Name Member_Name Management_IP Management_Netmask Sync_IP Sync_Mask External_IP External_Netmask

EX:

./vsecClusterObject.sh createMemberObject vSECCluster member1 192.168.1.15 255.255.255.0 1.1.1.2 255.255.255.0 192.168.2.40 255.255.255.0

./vsecClusterObject.sh createSICWithObject vSECCluster member1 MXEydzNlNHI=

# Adding Member 2:
This will add member 2 into the cluster object
./vsecClusterObject.sh createMemberObject vSECCluster member2 192.168.1.16 255.255.255.0 1.1.1.3 255.255.255.0 192.168.2.41 255.255.255.0
./vsecClusterObject.sh createSICWithObject vSECCluster member2 MXEydzNlNHI=

# createSICWithObject

This function create the SIC with previously defined cluster member. IMPORTANT NOTE: SIC password needs to be encoded in base64

Once the members are added into the cluster object, we need to define the virtual IP (VIP). This second script do the job:

vip.sh Cluster_Name VIP Interface_Name

EX: for a Cluster with 3 interfaces, we call the script 3 times:

./vip.sh vSECCluster 192.168.1.14 eth0
./vip.sh vSECCluster 1.1.1.1 eth1 # NEED VIP ON SYNC INTERFACE FOR AUTOSCALEUP AND DOWN option
./vip.sh vSECCluster 192.168.2.39 eth2

Now its time to push the policy:

# pushing Policy:
installPolicyOnObject Cluster_Name Policy_Package_Name
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest

Now we have a cluster with two members auto deployed. This open up the door for Auto-Scaling. Since we have a HA cluster deployed, we can add a cluster member and switch the cluster mode to LoadSharing. This part of the  script doing this function:

#!/bin/bash

#
# First, we need to add cluster member 3:
echo "=========================="
echo "Adding member3 to cluster "
echo "=========================="
./vsecClusterObject.sh createMemberObject vSECCluster member3 192.168.1.17 255.255.255.0 1.1.1.4 255.255.255.0 192.168.2.42 255.255.255.0
./vsecClusterObject.sh createSICWithObject vSECCluster member3 MXEydzNlNHI=
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest
echo "=========================="

echo "=========================="
echo "set cluster in LoadSharingMode"
./vsecClusterObject.sh setHAMode vSECCluster LoadSharing
echo "=========================="

# 5
# pushing Policy:
echo "=========================="
echo "Installing policy..."
echo "=========================="
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest

We now have a cluster of 3 members in loadsharing mode. 

To Scale-Down we just need to delete member3 and switch back to HA mode:

#!/bin/bash

echo "=========================="
echo "Scaling down..."
echo "=========================="
./vsecClusterObject.sh setHAMode vSECCluster HighAvailability
./vsecClusterObject.sh deleteMemberObject member3 vSECCluster
echo "=========================="
echo "Installing policy..."
echo "=========================="
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest

One way to orchestrate is if by using Ansible and calling those scripts with SSH command on the management server. See attached Ansible Document for an how to. For a quick test, Here is a bash script example to call all those functions:

create.sh

#!/bin/bash
# 1
# Creating cluster Object:
echo "=========================="
echo "Creating cluster object..."
echo "=========================="
./vsecClusterObject.sh createClusterObject vSECCluster 192.168.1.14 1.1.1.0 255.255.255.0
echo "=========================="

# 2
# Adding Member 1:
echo "=========================="
echo "Adding member1 to cluster "
echo "=========================="
./vsecClusterObject.sh createMemberObject vSECCluster member1 192.168.1.15 255.255.255.0 1.1.1.2 255.255.255.0 192.168.2.40 255.255.255.0
./vsecClusterObject.sh createSICWithObject vSECCluster member1 MXEydzNlNHI=
echo "=========================="

# 3
# Adding Member 2:
echo "=========================="
echo "Adding member2 to cluster "
echo "=========================="
./vsecClusterObject.sh createMemberObject vSECCluster member2 192.168.1.16 255.255.255.0 1.1.1.3 255.255.255.0 192.168.2.41 255.255.255.0
./vsecClusterObject.sh createSICWithObject vSECCluster member2 MXEydzNlNHI=
echo "=========================="

# 4
# Creating Cluster Virtual IP:
echo "==========================="
echo "Creating cluster virtual IP"
echo "==========================="
mgmt_cli login --root true > login.txt
./vip.sh vSECCluster 192.168.1.14 eth0
./vip.sh vSECCluster 1.1.1.1 eth1 # NEED VIP ON SYNC INTERFACE FOR AUTOSCALEUP AND DOWN
./vip.sh vSECCluster 192.168.2.39 eth2
mgmt_cli publish -s login.txt
mgmt_cli logout -s login.txt
rm login.txt
echo "=========================="

# 5
# pushing Policy:
echo "=========================="
echo "Installing policy..."
echo "=========================="
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest

I hope you enjoy and happy Scripting! 

🙂

For the full list of White Papers, go here. 

[PhoneBoy]

Nice

[Robert_Decker]

Excellent work! You master the API and JQ really good.

Please be careful using the "generic-objects" API, it is not supported and it will be dismissed in the future, once the new gateway/cluster schema changes.

Take a look at our Ansible development kit on GitHub:

GitHub - CheckPoint-APIs-Team/cpAnsible: Ansible module provides control over a Check Point Manageme... 

You can try and further leverage it for your future uses.

Robert.

[Nicolas_Boisse]

Thanks Robert

[HeikoAnkenbrand]

nice, THX

[Jim_Oqvist]

Great material, thanks Nicolas!

[JozkoMrkvicka]

As R80.20 is in EA stage, I would expect that the API will support Cluster handling.

In fact this is not true and R80.20 API (version 1.2) cannot do anything with Cluster deployment.

Any plans on that ? For example create new VLANs using API ?

[Robert_Decker]

Hi Jozko,

The development of new gateway/cluster/vsx objects is still in progress.

This is a major shift from R77.x into R80.x and it takes time.

Once these objects development will be completed, it will also include full API support.

Robert.

[JozkoMrkvicka]

Hi Robert,

Thank you for letting us know that this topic is still ongoing

Hope it will be included in R80.30.

[Joachim_Zint1]

Great stuff. New API commands needs a lot of time and this seems to be a great option.

[JozkoMrkvicka]

When we can FINALLY expect such a basic feature like manipulating Cluster objects within R80 ? R80.30 is GA, without any single API command for this purpose. What a shame.

[Nicolas_Boisse]

The cluster API was supposed to be released in R80.30 M1. This is postponed to R80.40 version.

[Mark_Colatosti]

Let me start off by saying there are not many people aware of your fix to add network interface on R80 mgmt servers, pretty key in my opinion if you are trying to do any automation and want to work with existing objects.  For instance I have a WAN tier set of devices that I need to be able to add/remove interfaces to on the management side and there is no native API.  I've been struggling a bit to get your code to work on a single simple-gateway object, not a cluster.  Would really appreciate your help as support has really been unhelpful up to this point.

You code in question is below:

# set cluster and members with newly created interfaces
mgmt_cli set generic-object uid $cluster_uid interfaces.add.create "com.checkpoint.objects.classes.dummy.CpmiClusterInterface" interfaces.add.owned-object.netmask "255.255.255.0" interfaces.add.owned-object.ipaddr $vip_ip interfaces.add.owned-object.memberNetwork.create "com.checkpoint.objects.classes.dummy.CpmiSubnet" interfaces.add.owned-object.memberNetwork.owned-object.netmask "255.255.255.0" interfaces.add.owned-object.memberNetwork.owned-object.ipaddr $cluster_net_ip interfaces.add.owned-object.officialname $interface_name interfaces.add.owned-object.monitoredByCluster true interfaces.add.owned-object.ifindex $if_index --format json --session-file login.txt > cluster_set_response.json

Could you let me know what this API/CLI call would like for a single gateway.  just can't seem to get it right!

Would be very thankful!  Cheers,

[Nicolas_Boisse]

Hi Mark,

First my code was for Cluster Object deployment. In your case, if you are using Simple Gateway deployment, don't use the Generic Object API.

What version of Management server are you using? (.10 .20 or .30) 

Keep in mind that each release has its own version of API:

Management API Version	Check Point Release
v1.5	R80.30
v1.4	R80.20.M2
v1.3	R80.20
v1.2	R80.20.M1
v1.1	R80.10
v1	R80

That been said, in the API call "add simple-gateway", you have the option to add interfaces/IP addresses in a simpler way.

Something like this:

mgmt_cli add simple-gateway name "gw1" color "yellow" ipv4-address "192.0.2.230" version "R80" one-time-password "aaaa" firewall true vpn true application-control true url-filtering true ips true anti-bot true anti-virus true threat-emulation true interfaces.1.name "eth0" interfaces.1.ipv4-address "192.0.2.230" interfaces.1.ipv4-network-mask "255.255.255.128" interfaces.1.anti-spoofing true interfaces.1.topology "EXTERNAL" interfaces.2.name "eth1" interfaces.2.ipv4-address "192.0.2.88" interfaces.2.ipv4-network-mask "255.255.255.0" interfaces.2.anti-spoofing true interfaces.2.topology "INTERNAL" --format json

The online documentation is available here:

https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/add-simple-gateway~v1.5%20

Hope this help.

Nicolas.

[Nicolas_Boisse]

If the object is already there, use the "set simple-gateway" instead to change or add an interface:

https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/set-simple-gateway~v1.5%20

[Mark_Colatosti]

I'll do a quick test, but the API is explicit about all existing interfaces being deleted when using that call?!

[Mark_Colatosti]

I'm basically trying to simulate a "get interfaces without topology" or an interface add on a device that has 30+ interfaces and is expected to grow as we add alot of vpnt interfaces.

[Nicolas_Boisse]

Hi Mark, you are right. The set simple-gateway reset the whole topology and use only the interface you provide in the set command.

What we can do is to read the information from a show simple-gateway call before adding the new interface:

 show simple-gateway name gw1 --format json details-level full

This will give you a json like this:

[Expert@R80.20_Management:0]# cat simplegateway.json
{
"uid" : "6073406b-bc78-43aa-97ff-fcfaa6319fe9",
"name" : "gw1",
"type" : "simple-gateway",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"interfaces" : [ {
"name" : "eth0",
"ipv4-address" : "192.0.2.230",
"ipv4-network-mask" : "255.255.255.128",
"ipv4-mask-length" : 25,
"ipv6-address" : "",
"comments" : "",
"color" : "black",
"icon" : "NetworkObjects/network",
"topology" : "external",
"anti-spoofing" : true,
"anti-spoofing-settings" : {
"action" : "prevent"
},
"security-zone" : false
}, {
"name" : "eth1",
"ipv4-address" : "192.0.2.88",
"ipv4-network-mask" : "255.255.255.0",
"ipv4-mask-length" : 24,
"ipv6-address" : "",
"comments" : "",
"color" : "black",
"icon" : "NetworkObjects/network",
"topology" : "internal",
"topology-settings" : {
"ip-address-behind-this-interface" : "not defined",
"interface-leads-to-dmz" : false
},
"anti-spoofing" : true,
"anti-spoofing-settings" : {
"action" : "prevent"
},
"security-zone" : false
} ],
"ipv4-address" : "192.0.2.230",
"dynamic-ip" : false,
"version" : "R80",
"os-name" : "Gaia",
"hardware" : "Open server",
"sic-name" : "",
"sic-state" : "initialized",
"firewall" : true,
"firewall-settings" : {
"auto-maximum-limit-for-concurrent-connections" : true,
"maximum-limit-for-concurrent-connections" : 25000,
"auto-calculate-connections-hash-table-size-and-memory-pool" : true,
"connections-hash-size" : 131072,
"memory-pool-size" : 6,
"maximum-memory-pool-size" : 30
},
"vpn" : true,
"vpn-settings" : {
"maximum-concurrent-ike-negotiations" : 1000,
"maximum-concurrent-tunnels" : 10000
},
"application-control" : true,
"url-filtering" : true,
"ips" : true,
"content-awareness" : false,
"anti-bot" : true,
"anti-virus" : true,
"threat-emulation" : true,
"threat-extraction" : false,
"save-logs-locally" : false,
"send-alerts-to-server" : [ "R80.20_Management" ],
"send-logs-to-server" : [ "R80.20_Management" ],
"send-logs-to-backup-server" : [ ],
"logs-settings" : {
"rotate-log-by-file-size" : false,
"rotate-log-file-size-threshold" : 1000,
"rotate-log-on-schedule" : false,
"alert-when-free-disk-space-below-metrics" : "mbytes",
"alert-when-free-disk-space-below" : true,
"alert-when-free-disk-space-below-threshold" : 20,
"alert-when-free-disk-space-below-type" : "popup alert",
"delete-when-free-disk-space-below-metrics" : "mbytes",
"delete-when-free-disk-space-below" : true,
"delete-when-free-disk-space-below-threshold" : 5000,
"before-delete-keep-logs-from-the-last-days" : false,
"before-delete-keep-logs-from-the-last-days-threshold" : 0,
"before-delete-run-script" : false,
"before-delete-run-script-command" : "",
"stop-logging-when-free-disk-space-below-metrics" : "mbytes",
"stop-logging-when-free-disk-space-below" : true,
"stop-logging-when-free-disk-space-below-threshold" : 100,
"reject-connections-when-free-disk-space-below-threshold" : false,
"reserve-for-packet-capture-metrics" : "mbytes",
"reserve-for-packet-capture-threshold" : 500,
"delete-index-files-when-index-size-above-metrics" : "mbytes",
"delete-index-files-when-index-size-above" : false,
"delete-index-files-when-index-size-above-threshold" : 100000,
"delete-index-files-older-than-days" : false,
"delete-index-files-older-than-days-threshold" : 14,
"forward-logs-to-log-server" : false,
"perform-log-rotate-before-log-forwarding" : false,
"update-account-log-every" : 3600,
"detect-new-citrix-ica-application-names" : false,
"turn-on-qos-logging" : true
},
"groups" : [ ],
"comments" : "",
"color" : "yellow",
"icon" : "NetworkObjects/gateway",
"tags" : [ ],
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1560216629523,
"iso-8601" : "2019-06-10T21:30-0400"
},
"last-modifier" : "admin",
"creation-time" : {
"posix" : 1560216612422,
"iso-8601" : "2019-06-10T21:30-0400"
},
"creator" : "admin"
},
"read-only" : false
}

From that json, we can show the information that we want. Interface name, ip, mask. JQ will be helpful here:

This JQ  will show all interfaces of the gw and format the output in CSV:

cat simplegateway.json | jq '.interfaces[] | [."name", ."ipv4-address", ."ipv4-network-mask"] | @csv' -r

"eth0","192.0.2.230","255.255.255.128"
"eth1","192.0.2.88","255.255.255.0"

You can then process the CSV with a loop to set simple-gateway interfaces + the new one.

Once done, a policy install will be required.

Hope this helps.

Nicolas.

[Mark_Colatosti]

Hey Nicolas,

Thanks for the prompt replies, ultimately I was hoping to use this approach to "ADD" non-destructively new interfaces to existing checkpoint management gateway objects.  And to be very specific, to be able to add numbered VPN interfaces.  I have Terraform based automation to spin up AWS transitGateway environments, connect them to a custom inter-region Checkpoint WAN routed tier, but just can't get what you'd think would be the trivial operation of updating the checkpoint management object completed.  Either a "network interface refresh with no topology" (like done in the management GUI), or manual adds of vpn tunnel interfaces in the management system.  Its frustrating to get stuck at what seemingly is an obvious need to edit existing objects.  Even if I was willing to export all the existing interface details somehow, and the use the simple-gateway API to "set" them all over (probably traffic disruptive), there seems to be no way to add VPN interfaces!

I'll probably take one last attempt at using this code to create an ethernet interface, and then edit all the properties of the interface ifindex:# to try to "convert" it to a vpn tunnel interface by setting the properties visible with a show-generic-object on a comparable functional VPN tunnel interface.

Again though, can't stress how crazy all of this coding is, when its just the result of a "missing"  single "non-destructive ADD simple-gateway VPN interface" API call.

[Mark_Colatosti]

Looks like I was inadvertantly grabbing some "Endpoint" uid instead of the gateway object uid, though commands were still succeeding to create an interface, it prevented me from successfully "set"/associating it.

The following code seems to work and is simplified to not include additional cluster properties.

iac_gwnetadd=$(mgmt_cli add generic-object create "com.checkpoint.management.cdm.objects.interfaces.EthernetInterface" name $int_name gatewayOwner $cp-gateway_uid gatewayNetwork $gatewaynetobject_uid $ip_addr ipv4MaskLength $mask_len --format json)

Thanks for your original article as everything else tried was pretty useless!

[Nicolas_Boisse]

Hi Mark, you still need to figure out the AntiSpooging configuration. Since this is not a cluster, you should use the Set simple-gateway instead of generic-object and set only what you want to change in the API call. You will able to handle the set interface + Anti-Spoofing in 1 call: 

mgmt_cli set simple-gateway name "gw1" interfaces.1.name "eth0" interfaces.1.ipv4-address "192.0.2.230" interfaces.1.ipv4-network-mask "255.255.255.128" interfaces.1.anti-spoofing true interfaces.1.topology "internal"

You can use the UID instead of name if you prefer.

To be tested in your lab first.

Good luck 🙂

Nicolas.

[Mark_Colatosti]

#Get wan simplegateway uid:
iac_wan1gw_uid=$(mgmt_cli --port 4434 -r true show-generic-objects name "test-test" --format json | (${CPDIR}/jq/jq -r '.objects[] | select (.type == "simple-gateway") | .uid'))

# Add network interface to simplegateway
iac_wan1net1_uid=$(mgmt_cli --port 4434 -r true add generic-object create "com.checkpoint.management.cdm.objects.network.GatewayNetwork" name ${iac_int_name} clusterNetworkType "CLUSTER" gatewayOwner ${iac_wan1gw_uid} --format json | (${CPDIR}/jq/jq -r '.uid'))

# Add Ip address details, how to make this a point-to-point VPN connection?
mgmt_cli --port 4434 -r true add generic-object create "com.checkpoint.management.cdm.objects.interfaces.EthernetInterface" name ${iac_int_name} gatewayOwner ${iac_wan1gw_uid} gatewayNetwork ${iac_wan1net1_uid} ipv4Address ${iac_ipaddr} ipv4MaskLength 30

Note:

This creates an ethernet interface of unspecified topology.  Now if I could find out how to create a virtual VPN point-to-point interface!  Though I suspect this will still work....

[Nicolas_Boisse]

FYI,

I'm working on a pre-relase off add simple-cluster API.  Its coming in R80.40. Stay tuned.

[vivekachar09]

Hi @Mark_Colatosti 

Were you able to add the point-to-point-VPN connection using the mgmt_cli. Did it work for you? We have a similar use case and were looking for a simple-gateway VPN interface API call or else some workarounds.