cancel
Showing results for 
Search instead for 
Did you mean: 
Create a Post
Employee+
Employee+

CloudGuard: Automated firewall Cluster Deployment with auto-scaling option

If you are playing with the API's, you will realise there is no API call yet available for Cluster Deployment. In the meantime, with little help from R&D, we've created this automation script: "vsecClusterObject.sh"

 

The script run from the management server and as many functions available. We leverage DBEDIT code and API Calls to help automate the cluster deployment and auto-scaling.

 

Here the function available:

# createClusterObject   (4 variables needed):
This will create the cluster object: CreateClusterObject Cluster_Name Cluster_IP SYNC_Network SYNC_Netmask

 

EX:

./vsecClusterObject.sh createClusterObject vSECCluster 192.168.1.14 1.1.1.0 255.255.255.0

 

 

# Adding Member 1:

# createMemberObject (8 Variables):
This will add member 1 into the cluster object

createMemberObject Cluster_Name Member_Name Management_IP Management_Netmask Sync_IP Sync_Mask External_IP External_Netmask

 

EX:

./vsecClusterObject.sh createMemberObject vSECCluster member1 192.168.1.15 255.255.255.0 1.1.1.2 255.255.255.0 192.168.2.40 255.255.255.0

./vsecClusterObject.sh createSICWithObject vSECCluster member1 MXEydzNlNHI=

 

# Adding Member 2:
This will add member 2 into the cluster object
./vsecClusterObject.sh createMemberObject vSECCluster member2 192.168.1.16 255.255.255.0 1.1.1.3 255.255.255.0 192.168.2.41 255.255.255.0
./vsecClusterObject.sh createSICWithObject vSECCluster member2 MXEydzNlNHI=

 

# createSICWithObject

This function create the SIC with previously defined cluster member. IMPORTANT NOTE: SIC password needs to be encoded in base64

 

Once the members are added into the cluster object, we need to define the virtual IP (VIP). This second script do the job:

vip.sh Cluster_Name VIP Interface_Name

 

EX: for a Cluster with 3 interfaces, we call the script 3 times:

./vip.sh vSECCluster 192.168.1.14 eth0
./vip.sh vSECCluster 1.1.1.1 eth1 # NEED VIP ON SYNC INTERFACE FOR AUTOSCALEUP AND DOWN option
./vip.sh vSECCluster 192.168.2.39 eth2

 

Now its time to push the policy:

# pushing Policy:
installPolicyOnObject Cluster_Name Policy_Package_Name
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest

 

Now we have a cluster with two members auto deployed. This open up the door for Auto-Scaling. Since we have a HA cluster deployed, we can add a cluster member and switch the cluster mode to LoadSharing. This part of the  script doing this function:

#!/bin/bash

#
# First, we need to add cluster member 3:
echo "=========================="
echo "Adding member3 to cluster "
echo "=========================="
./vsecClusterObject.sh createMemberObject vSECCluster member3 192.168.1.17 255.255.255.0 1.1.1.4 255.255.255.0 192.168.2.42 255.255.255.0
./vsecClusterObject.sh createSICWithObject vSECCluster member3 MXEydzNlNHI=
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest
echo "=========================="


echo "=========================="
echo "set cluster in LoadSharingMode"
./vsecClusterObject.sh setHAMode vSECCluster LoadSharing
echo "=========================="

# 5
# pushing Policy:
echo "=========================="
echo "Installing policy..."
echo "=========================="
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest

 

We now have a cluster of 3 members in loadsharing mode. 

 

To Scale-Down we just need to delete member3 and switch back to HA mode:

#!/bin/bash

echo "=========================="
echo "Scaling down..."
echo "=========================="
./vsecClusterObject.sh setHAMode vSECCluster HighAvailability
./vsecClusterObject.sh deleteMemberObject member3 vSECCluster
echo "=========================="
echo "Installing policy..."
echo "=========================="
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest

 

 

One way to orchestrate is if by using Ansible and calling those scripts with SSH command on the management server. See attached Ansible Document for an how to. For a quick test, Here is a bash script example to call all those functions:

 

create.sh

#!/bin/bash
# 1
# Creating cluster Object:
echo "=========================="
echo "Creating cluster object..."
echo "=========================="
./vsecClusterObject.sh createClusterObject vSECCluster 192.168.1.14 1.1.1.0 255.255.255.0
echo "=========================="

# 2
# Adding Member 1:
echo "=========================="
echo "Adding member1 to cluster "
echo "=========================="
./vsecClusterObject.sh createMemberObject vSECCluster member1 192.168.1.15 255.255.255.0 1.1.1.2 255.255.255.0 192.168.2.40 255.255.255.0
./vsecClusterObject.sh createSICWithObject vSECCluster member1 MXEydzNlNHI=
echo "=========================="


# 3
# Adding Member 2:
echo "=========================="
echo "Adding member2 to cluster "
echo "=========================="
./vsecClusterObject.sh createMemberObject vSECCluster member2 192.168.1.16 255.255.255.0 1.1.1.3 255.255.255.0 192.168.2.41 255.255.255.0
./vsecClusterObject.sh createSICWithObject vSECCluster member2 MXEydzNlNHI=
echo "=========================="

 

# 4
# Creating Cluster Virtual IP:
echo "==========================="
echo "Creating cluster virtual IP"
echo "==========================="
mgmt_cli login --root true > login.txt
./vip.sh vSECCluster 192.168.1.14 eth0
./vip.sh vSECCluster 1.1.1.1 eth1 # NEED VIP ON SYNC INTERFACE FOR AUTOSCALEUP AND DOWN
./vip.sh vSECCluster 192.168.2.39 eth2
mgmt_cli publish -s login.txt
mgmt_cli logout -s login.txt
rm login.txt
echo "=========================="

# 5
# pushing Policy:
echo "=========================="
echo "Installing policy..."
echo "=========================="
./vsecClusterObject.sh installPolicyOnObject vSECCLuster AutomationTest

 

 

I hope you enjoy and happy Scripting! 

 

🙂

 

For the full list of White Papers, go here

21 Replies
Admin
Admin

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Nice Smiley Happy

Employee++
Employee++

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Excellent work! You master the API and JQ really good.

Please be careful using the "generic-objects" API, it is not supported and it will be dismissed in the future, once the new gateway/cluster schema changes.

Take a look at our Ansible development kit on GitHub:

GitHub - CheckPoint-APIs-Team/cpAnsible: Ansible module provides control over a Check Point Manageme... 

You can try and further leverage it for your future uses.

Robert.

Employee+
Employee+

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Thanks Robert

0 Kudos

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

nice, THX

0 Kudos
Highlighted
Employee+
Employee+

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Great material, thanks Nicolas!

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

As R80.20 is in EA stage, I would expect that the API will support Cluster handling.

In fact this is not true and R80.20 API (version 1.2) cannot do anything with Cluster deployment.

Any plans on that ? For example create new VLANs using API ?

Kind regards,
Jozko Mrkvicka
0 Kudos
Employee++
Employee++

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Hi Jozko,

The development of new gateway/cluster/vsx objects is still in progress.

This is a major shift from R77.x into R80.x and it takes time.

Once these objects development will be completed, it will also include full API support.

Robert.

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Hi Robert,

Thank you for letting us know that this topic is still ongoing Smiley Happy

Hope it will be included in R80.30.

Kind regards,
Jozko Mrkvicka
Employee
Employee

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Great stuff. New API commands needs a lot of time and this seems to be a great option.

0 Kudos

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

When we can FINALLY expect such a basic feature like manipulating Cluster objects within R80 ? R80.30 is GA, without any single API command for this purpose. What a shame.

Kind regards,
Jozko Mrkvicka
0 Kudos
Employee+
Employee+

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

The cluster API was supposed to be released in R80.30 M1. This is postponed to R80.40 version.

0 Kudos

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Let me start off by saying there are not many people aware of your fix to add network interface on R80 mgmt servers, pretty key in my opinion if you are trying to do any automation and want to work with existing objects.  For instance I have a WAN tier set of devices that I need to be able to add/remove interfaces to on the management side and there is no native API.  I've been struggling a bit to get your code to work on a single simple-gateway object, not a cluster.  Would really appreciate your help as support has really been unhelpful up to this point.

You code in question is below:

# set cluster and members with newly created interfaces
mgmt_cli set generic-object uid $cluster_uid interfaces.add.create "com.checkpoint.objects.classes.dummy.CpmiClusterInterface" interfaces.add.owned-object.netmask "255.255.255.0" interfaces.add.owned-object.ipaddr $vip_ip interfaces.add.owned-object.memberNetwork.create "com.checkpoint.objects.classes.dummy.CpmiSubnet" interfaces.add.owned-object.memberNetwork.owned-object.netmask "255.255.255.0" interfaces.add.owned-object.memberNetwork.owned-object.ipaddr $cluster_net_ip interfaces.add.owned-object.officialname $interface_name interfaces.add.owned-object.monitoredByCluster true interfaces.add.owned-object.ifindex $if_index --format json --session-file login.txt > cluster_set_response.json

 

Could you let me know what this API/CLI call would like for a single gateway.  just can't seem to get it right!

Would be very thankful!  Cheers,

0 Kudos
Employee+
Employee+

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Hi Mark,

First my code was for Cluster Object deployment. In your case, if you are using Simple Gateway deployment, don't use the Generic Object API.

What version of Management server are you using? (.10 .20 or .30) 

Keep in mind that each release has its own version of API:

Management API Version Check Point Release
v1.5 R80.30
v1.4 R80.20.M2
v1.3 R80.20
v1.2 R80.20.M1
v1.1 R80.10
v1 R80

 

That been said, in the API call "add simple-gateway", you have the option to add interfaces/IP addresses in a simpler way.

Something like this:

mgmt_cli add simple-gateway name "gw1" color "yellow" ipv4-address "192.0.2.230" version "R80" one-time-password "aaaa" firewall true vpn true application-control true url-filtering true ips true anti-bot true anti-virus true threat-emulation true interfaces.1.name "eth0" interfaces.1.ipv4-address "192.0.2.230" interfaces.1.ipv4-network-mask "255.255.255.128" interfaces.1.anti-spoofing true interfaces.1.topology "EXTERNAL" interfaces.2.name "eth1" interfaces.2.ipv4-address "192.0.2.88" interfaces.2.ipv4-network-mask "255.255.255.0" interfaces.2.anti-spoofing true interfaces.2.topology "INTERNAL" --format json

The online documentation is available here:

https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/add-simple-gateway~v1.5%20

Hope this help.

 

Nicolas.

0 Kudos
Employee+
Employee+

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

If the object is already there, use the "set simple-gateway" instead to change or add an interface:

https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/set-simple-gateway~v1.5%20

 

0 Kudos

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

I'll do a quick test, but the API is explicit about all existing interfaces being deleted when using that call?!

0 Kudos

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

I'm basically trying to simulate a "get interfaces without topology" or an interface add on a device that has 30+ interfaces and is expected to grow as we add alot of vpnt interfaces.

0 Kudos
Employee+
Employee+

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Hi Mark, you are right. The set simple-gateway reset the whole topology and use only the interface you provide in the set command.

What we can do is to read the information from a show simple-gateway call before adding the new interface:

 show simple-gateway name gw1 --format json details-level full

This will give you a json like this:

[Expert@R80.20_Management:0]# cat simplegateway.json
{
"uid" : "6073406b-bc78-43aa-97ff-fcfaa6319fe9",
"name" : "gw1",
"type" : "simple-gateway",
"domain" : {
"uid" : "41e821a0-3720-11e3-aa6e-0800200c9fde",
"name" : "SMC User",
"domain-type" : "domain"
},
"interfaces" : [ {
"name" : "eth0",
"ipv4-address" : "192.0.2.230",
"ipv4-network-mask" : "255.255.255.128",
"ipv4-mask-length" : 25,
"ipv6-address" : "",
"comments" : "",
"color" : "black",
"icon" : "NetworkObjects/network",
"topology" : "external",
"anti-spoofing" : true,
"anti-spoofing-settings" : {
"action" : "prevent"
},
"security-zone" : false
}, {
"name" : "eth1",
"ipv4-address" : "192.0.2.88",
"ipv4-network-mask" : "255.255.255.0",
"ipv4-mask-length" : 24,
"ipv6-address" : "",
"comments" : "",
"color" : "black",
"icon" : "NetworkObjects/network",
"topology" : "internal",
"topology-settings" : {
"ip-address-behind-this-interface" : "not defined",
"interface-leads-to-dmz" : false
},
"anti-spoofing" : true,
"anti-spoofing-settings" : {
"action" : "prevent"
},
"security-zone" : false
} ],
"ipv4-address" : "192.0.2.230",
"dynamic-ip" : false,
"version" : "R80",
"os-name" : "Gaia",
"hardware" : "Open server",
"sic-name" : "",
"sic-state" : "initialized",
"firewall" : true,
"firewall-settings" : {
"auto-maximum-limit-for-concurrent-connections" : true,
"maximum-limit-for-concurrent-connections" : 25000,
"auto-calculate-connections-hash-table-size-and-memory-pool" : true,
"connections-hash-size" : 131072,
"memory-pool-size" : 6,
"maximum-memory-pool-size" : 30
},
"vpn" : true,
"vpn-settings" : {
"maximum-concurrent-ike-negotiations" : 1000,
"maximum-concurrent-tunnels" : 10000
},
"application-control" : true,
"url-filtering" : true,
"ips" : true,
"content-awareness" : false,
"anti-bot" : true,
"anti-virus" : true,
"threat-emulation" : true,
"threat-extraction" : false,
"save-logs-locally" : false,
"send-alerts-to-server" : [ "R80.20_Management" ],
"send-logs-to-server" : [ "R80.20_Management" ],
"send-logs-to-backup-server" : [ ],
"logs-settings" : {
"rotate-log-by-file-size" : false,
"rotate-log-file-size-threshold" : 1000,
"rotate-log-on-schedule" : false,
"alert-when-free-disk-space-below-metrics" : "mbytes",
"alert-when-free-disk-space-below" : true,
"alert-when-free-disk-space-below-threshold" : 20,
"alert-when-free-disk-space-below-type" : "popup alert",
"delete-when-free-disk-space-below-metrics" : "mbytes",
"delete-when-free-disk-space-below" : true,
"delete-when-free-disk-space-below-threshold" : 5000,
"before-delete-keep-logs-from-the-last-days" : false,
"before-delete-keep-logs-from-the-last-days-threshold" : 0,
"before-delete-run-script" : false,
"before-delete-run-script-command" : "",
"stop-logging-when-free-disk-space-below-metrics" : "mbytes",
"stop-logging-when-free-disk-space-below" : true,
"stop-logging-when-free-disk-space-below-threshold" : 100,
"reject-connections-when-free-disk-space-below-threshold" : false,
"reserve-for-packet-capture-metrics" : "mbytes",
"reserve-for-packet-capture-threshold" : 500,
"delete-index-files-when-index-size-above-metrics" : "mbytes",
"delete-index-files-when-index-size-above" : false,
"delete-index-files-when-index-size-above-threshold" : 100000,
"delete-index-files-older-than-days" : false,
"delete-index-files-older-than-days-threshold" : 14,
"forward-logs-to-log-server" : false,
"perform-log-rotate-before-log-forwarding" : false,
"update-account-log-every" : 3600,
"detect-new-citrix-ica-application-names" : false,
"turn-on-qos-logging" : true
},
"groups" : [ ],
"comments" : "",
"color" : "yellow",
"icon" : "NetworkObjects/gateway",
"tags" : [ ],
"meta-info" : {
"lock" : "unlocked",
"validation-state" : "ok",
"last-modify-time" : {
"posix" : 1560216629523,
"iso-8601" : "2019-06-10T21:30-0400"
},
"last-modifier" : "admin",
"creation-time" : {
"posix" : 1560216612422,
"iso-8601" : "2019-06-10T21:30-0400"
},
"creator" : "admin"
},
"read-only" : false
}

From that json, we can show the information that we want. Interface name, ip, mask. JQ will be helpful here:

This JQ  will show all interfaces of the gw and format the output in CSV:

cat simplegateway.json | jq '.interfaces[] | [."name", ."ipv4-address", ."ipv4-network-mask"] | @csv' -r

"eth0","192.0.2.230","255.255.255.128"
"eth1","192.0.2.88","255.255.255.0"

You can then process the CSV with a loop to set simple-gateway interfaces + the new one.

Once done, a policy install will be required.

Hope this helps.

Nicolas.

0 Kudos

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Hey Nicolas,

Thanks for the prompt replies, ultimately I was hoping to use this approach to "ADD" non-destructively new interfaces to existing checkpoint management gateway objects.  And to be very specific, to be able to add numbered VPN interfaces.  I have Terraform based automation to spin up AWS transitGateway environments, connect them to a custom inter-region Checkpoint WAN routed tier, but just can't get what you'd think would be the trivial operation of updating the checkpoint management object completed.  Either a "network interface refresh with no topology" (like done in the management GUI), or manual adds of vpn tunnel interfaces in the management system.  Its frustrating to get stuck at what seemingly is an obvious need to edit existing objects.  Even if I was willing to export all the existing interface details somehow, and the use the simple-gateway API to "set" them all over (probably traffic disruptive), there seems to be no way to add VPN interfaces!

I'll probably take one last attempt at using this code to create an ethernet interface, and then edit all the properties of the interface ifindex:# to try to "convert" it to a vpn tunnel interface by setting the properties visible with a show-generic-object on a comparable functional VPN tunnel interface.

Again though, can't stress how crazy all of this coding is, when its just the result of a "missing"  single "non-destructive ADD simple-gateway VPN interface" API call.

0 Kudos

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Looks like I was inadvertantly grabbing some "Endpoint" uid instead of the gateway object uid, though commands were still succeeding to create an interface, it prevented me from successfully "set"/associating it.

 

The following code seems to work and is simplified to not include additional cluster properties.

iac_gwnetadd=$(mgmt_cli add generic-object create "com.checkpoint.management.cdm.objects.interfaces.EthernetInterface" name $int_name gatewayOwner $cp-gateway_uid gatewayNetwork $gatewaynetobject_uid $ip_addr ipv4MaskLength $mask_len --format json)

 

Thanks for your original article as everything else tried was pretty useless!

0 Kudos
Employee+
Employee+

Re: Cloud Guard: Automated firewall Cluster Deployment with auto-scaling option

Hi Mark, you still need to figure out the AntiSpooging configuration. Since this is not a cluster, you should use the Set simple-gateway instead of generic-object and set only what you want to change in the API call. You will able to handle the set interface + Anti-Spoofing in 1 call: 

mgmt_cli set simple-gateway name "gw1" interfaces.1.name "eth0" interfaces.1.ipv4-address "192.0.2.230" interfaces.1.ipv4-network-mask "255.255.255.128" interfaces.1.anti-spoofing true interfaces.1.topology "internal"

You can use the UID instead of name if you prefer.

To be tested in your lab first.

Good luck 🙂

 

Nicolas.

0 Kudos

Adding a network interface to a gateway, sample code that works

#Get wan simplegateway uid:
iac_wan1gw_uid=$(mgmt_cli --port 4434 -r true show-generic-objects name "test-test" --format json | (${CPDIR}/jq/jq -r '.objects[] | select (.type == "simple-gateway") | .uid'))

# Add network interface to simplegateway
iac_wan1net1_uid=$(mgmt_cli --port 4434 -r true add generic-object create "com.checkpoint.management.cdm.objects.network.GatewayNetwork" name ${iac_int_name} clusterNetworkType "CLUSTER" gatewayOwner ${iac_wan1gw_uid} --format json | (${CPDIR}/jq/jq -r '.uid'))

# Add Ip address details, how to make this a point-to-point VPN connection?
mgmt_cli --port 4434 -r true add generic-object create "com.checkpoint.management.cdm.objects.interfaces.EthernetInterface" name ${iac_int_name} gatewayOwner ${iac_wan1gw_uid} gatewayNetwork ${iac_wan1net1_uid} ipv4Address ${iac_ipaddr} ipv4MaskLength 30

 

 

Note:

This creates an ethernet interface of unspecified topology.  Now if I could find out how to create a virtual VPN point-to-point interface!  Though I suspect this will still work....

0 Kudos