I recently ran into a situation where one member of one of my clusters was aware of 15 VIPs and the other member was only aware of 13. It was annoying to find and to fix, so I created a small script to help me confirm whether my environment had any other instances of the problem. I thought it might be useful for others.
The script uses the management API to get a list of all CMAs on an MDS. If it's not on an MDS, it creates a fake CMA to represent the SmartCenter. It then iterates through the CMAs and looks for normal (non-VSX) clusters. Once it has the list of clusters, it dumps each of them to get a list of the members. It then uses cprid_util to connect to each member and run a command or set of commands, and it saves the output to files in /tmp on the MDS or SmartCenter.
When it is done running the commands on a cluster, it prints the name of that cluster and whether it found any differences.
NOTE: This script has no support for VSX right now, and no support for clusters with more than two members.
#!/usr/bin/env bash
######################################################################
### For non-VSX clusters
######################################################################
portNumber=443
unset cmaList
. /etc/profile.d/CP.sh
cmaList=$(mgmt_cli --port "${portNumber}" -f json -r true show domains limit 500 details-level full \
| jq -c '.objects[]|{name:.name,server:.servers[]|{host:."multi-domain-server",ipAddress:."ipv4-address"}}' \
| grep $(hostname) \
| jq -c '[.name,.server.ipAddress]')
if [ ${#cmaList} -eq 0 ];then cmaList=("[\"$(hostname)\",\"\"]");fi
for cmaRow in $cmaList; do
cmaName=$(echo "${cmaRow}" | jq '.[0]' | sed 's#"##g')
cmaAddress=$(echo "${cmaRow}" | jq '.[1]' | sed 's#"##g')
mdsenv "${cmaAddress}" 2>/dev/null
firewallList=$(mgmt_cli --port "${portNumber}" -f json -d "${cmaAddress}" -r true show gateways-and-servers limit 500 details-level full \
| jq -c '.objects[]|.' \
| grep CpmiGatewayCluster \
| jq -c '.uid' \
| xargs -L 1 mgmt_cli --port "${portNumber}" -f json -d "${cmaAddress}" -r true show object details-level full uid \
| jq -c '.object|{clusterName:.name,member:."cluster-members"[]} | {clusterName:.clusterName,memberName:.member.name,address:.member."ip-address"}')
clusterList=$(echo "$firewallList" | jq -c ".clusterName" | sort | uniq | sed 's#"##g')
for clusterName in $clusterList; do
for firewallLine in $(echo "$firewallList" | grep "$clusterName"); do
memberName="$(echo "${firewallLine}" | jq '.memberName' | sed 's#"##g')"
firewall="$(echo "${firewallLine}" | jq '.address' | sed 's#"##g')"
cprid_util -verbose -server "${firewall}" rexec -rcmd sh -c '
############################################################
cphaprob -a if | sort
############################################################
' > /tmp/"$clusterName"-"$memberName".output
done
echo "========================================"
echo -n "$clusterName:"
diffOut=$(diff /tmp/"$clusterName"-*)
if [ $? -eq 0 ]; then
echo " NO DIFFERENCES"
else
echo ""
echo "$diffOut"
fi
/bin/rm /tmp/"$clusterName"-*
done
done;echo "========================================"
The command you want to run on each cluster member goes between the hash rows, where I have "cphaprob -a if | sort". It can be multiple lines. Note that a lot of things on Check Point firewalls (such as clish configs) don't have stable ordering between devices. It's generally a good idea to sort the output like in my example.
There are also a lot of expected differences between cluster members. For example, each member has its own hostname, and its own unique IP addresses on each interface.
