Easy Backup Tool

[HeikoAnkenbrand]

asy Backup Tool

Features

This tool creates a backup of all GAIA gateway configurations with one CLI command "ebackup":

- Only one CLI command "ebackup"
- Backup of all Gaia gateway configurations (Check Point appliances, Open Server, SMB appliances 11xx, 14xx)
- Migrate export on SMS
- Migrate-server on MDS
- Backup all files to one TGZ file
- FTP upload support backup file
- CP upload support for backup file via cprid_util

- MDS   > All CMA's are read out and their gateways backuped.
- SMS    > All gateways are read out and backuped.

Note:
- Tested with R80.10, R80.20 and R80.30.
- If the tool is started on a MDS, a mdsstop and mdsstart is performed during the migrate_server export.

CLI Parameter

Syntax	Description
-s	The option -s  performs a cpstop and cpstart when the migrate export tool is executed.
-v	The option -v shows the gateway OS, JHF, Kernel, Type of all gateways.
-l	The option -l shows all ebackup tgz files in /var/log/.
-d	The option -d delete all ebackup tgz files in /var/log/.
-no_migrate / -n	The option -no_migrate has the consequence that no migrate export is executed.
-port <sms port> / -p <sms port>	The option -port <sms port> add the management server port, if it's not running on port 443.
-ftpserver <ftp server ip> -ftpuser <username> -ftppw <password>	The ftp options allow to upload the tar file to a ftp server.
- cpupload <cp_system_ip>	The option -cpupload performs a backup upload to a other Check Point gateway or SMS via cprid_util.

Example

# ebackup                                                                                                -> Backup all GAIA configs from all gateways + migrate export with locale backup file (/var/log/[date]_ebackup.tgz)
# ebackup -s                                                                                           -> Backup all GAIA configs  from all gateway + migrate export with cpstop and cpstart for migrate export
# ebackup -no_migrate                                                                         -> Backup all GAIA configs from all gateway without migrate export
# ebackup -ftpserver 1.1.1.1 -ftpuser username -ftppw test123   -> Backup all GAIA configs from all gateway + migrate export with ftp upload

# ebackup -cpupload 1.1.1.1                                                                 -> Backup all GAIA configs from all gateway + migrate export with cp upload via cprid_util

Install Tool

Use this auto installer script from "Spoiler" on the SMS or MDS as CLI command in expert mode:

Click to Expand

curl_cli -k http://www.ankenbrand24.de/inst_ebackup > /tmp/inst_ebackup && chmod 770 /tmp/inst_ebackup && /tmp/inst_ebackup

Or "copy and past" the following script block from "Spoiler" to your SMS or MDS (in expert mode):

Click to Expand

cat <<EOT > /usr/bin/ebackup
#!/bin/bash
#
# Script Author  : Heiko Ankenbrand
# Script Website : https://www.ankenbrand24.de
# Script Source  : https://www.ankenbrand24.de/inst_ebackup
# License        : GNU General Public License (GPL)
#
if [[ -e /etc/profile.d/CP.sh ]]; then source /etc/profile.d/CP.sh; else echo "Unsupported Environment"; exit 1; fi
if [[ -e /etc/profile.d/vsenv.sh ]]; then source /etc/profile.d/vsenv.sh; fi
FILE_PATH="/var/log/ebackup";
mkdir \$FILE_PATH > /dev/null 2>&1;
rm /tmp/ebackup_topo > /dev/null 2>&1;
rm /tmp/ebackup_MGTIP > /dev/null 2>&1;
# rm /usr/local/bin/ebackup > /dev/null 2>&1;
rm /tmp/ebackup_mdsstatus > /dev/null 2>&1;
rm /tmp/ebackup_login > /dev/null 2>&1;
rm /tmp/ebachup_cpupload /dev/null 2>&1;
REMOTE_RM="\$FILE_PATH/202*";
rm \$REMOTE_RM > /dev/null 2>&1;
NOW=\$(date +"%Y_%m_%d_%H%M");
clear;
echo -e "\e[7m";
printf '%.s-' {1..78};echo;
echo '- Easy Backup Tool v3.0                    - Copyright Heiko Ankenbrand 2023 -';
printf '%.s-' {1..78};
echo -e "\e[0m"
SHOW_SUM="0";
MIGRATE="1";
MGPORT="443";
FTPSERVER="";
FTPPW="non";
FTPUSER="anonymous";
serv_value="99";
REMOTE_INFO_V="0";
RVER=`fwm ver | awk '{print \$8}'`;

mSITIONAL=()
while [[ \$# -gt 0 ]]
do
key="\$1"
case \$key in
    -h|--help|-\?)
       shift;
       echo;echo 'Help: ';echo;
       echo "-l, --list                 The option -l  shows all ebackup tgz files in /var/log/.";
       echo "-d, --delete               The option -d  delete all ebackup tgz files in /var/log/.";
       echo "-s, --stop                 The option -s  performs a cpstop and cpstart when the migrate export tool is executed.";
       echo "-n, --no_migrate           The option -n  has the consequence that no migrate export/ migrate_server is executed.";
       echo "-v, --version              The option -v  shows the gateway OS, JHF, Kernel, Type";
       echo "-u, --user                 The option -u  shows the conneted users on the SMS";
       echo "-p <port>, -port <port>    The option -p  <port> add the management server port, if it's not running on port 443.";
       echo "-ftpserver <ftp server ip> The ftp options allow to upload the tar file to a ftp server.";
       echo "-ftpuser <username> ";
       echo "-ftppw <password>";
       echo "-cpupload <cp system ip>   The option -cpupload performs a backup upload to a other Check Point gateway or SMS via cprid_util.";
       echo "";
       echo "Examples:";
       echo "ebackup -s -ftpserver 192.168.1.1 -ftpuser test -ftppw secret -> backup all gateways + migrate export + cpstop + ftp upload";
       echo "ebackup -s -cpupload  192.168.1.1                             -> backup all gateways + migrate export + cpstop+ ";
       echo "                                                                 backup of the tgz file to another gateway or SMS";
       echo "ebackup -n                                                    -> backup all gateways + without migrate export";
       echo;printf '%.s-' {1..78};echo;
       SHOW_SUM="0";
       exit 1;
    ;;
    -s|--stop)
       shift;
       SHOW_SUM="1";
    ;;
    -v|--version)
       shift;
       SHOW_SUM="1";
       REMOTE_INFO_V="1";
    ;;
    -u|--user)
       shift;
       SHOW_SUM="1";
       echo; cpstat mg |grep -e '|\|-' | cat; echo;
       exit 1;
    ;;
    -d|--delete)
       shift;
       SHOW_SUM="1";
       echo; echo "Delete all ebackup tgz files.";echo;
       rm /var/log/202* > /dev/null 2>&1;
       exit 1;
    ;;
    -l|--list)
       shift;
       SHOW_SUM="1";
       echo;echo -e "File \t\t\t\t\tBytes";
       printf '%.s-' {1..78};
       echo; ls -la /var/log/202* 2>&1| grep -v access|  awk '{print \$9 "\t\t" \$5}' |sed 's/\/var\/log\///'; echo;
       exit 1;
    ;;
    -port|-p)
       MGPORT="\$2";
       shift;
       shift;
       SHOW_SUM="0";
    ;;
    -ftpserver)
       FTPSERVER="\$2";
       shift;
       shift;
       SHOW_SUM="0";
    ;;
    -ftpuser)
       FTPUSER="\$2";
       shift;
       shift;
       SHOW_SUM="0";
    ;;
    -ftppw)
       FTPPW="\$2";
       shift;
       shift;
       SHOW_SUM="0";
    ;;
     -cpupload)
       CPUPLOAD="\$2";
       shift;
       shift;
       SHOW_SUM="0";
    ;;
    --no_migrate|-no_migrate|-n)
       shift;
       SHOW_SUM="0";
       MIGRATE="0";
    ;;

    *)    # unknown option
    #echo "unknown"
    POSITIONAL+=("\$1") # save it in an array for later
    shift # past argument
    SHOW_SUM="1";
    ;;
esac
done

############ Check MDS and SMS     #########
echo;

if [ "X\$MDSDIR" != "X" ]; then
     MDSSTATUS_HA=\$(mdsstat | grep -m 1 MDS | awk '{print \$8}');
     if [ \$MDSSTATUS_HA = "up"  ]; then
           mgmt_cli -r --port \$MGPORT true show version 2>&1> /tmp/ebackup_login;
           if grep -q Gaia /tmp/ebackup_login; then
               echo -n "The MDS is running in version "\$RVER;echo ".";echo;
               serv_value="1";
           else
               echo "The MDS is running but a connect to management API isn't possible.";echo;
               exit 1;
           fi
     else
         echo -n "MDS is not running. (Status > "\$MDSSTATUS_HA;echo ")";echo;
         exit 1;
     fi
else
     #Check SMS
     CPM_PID=\$(ps -efww | grep java | grep cpm.Cpm | awk '{print \$2}');
     if [ -z \$CPM_PID ]; then
           echo "The SMS is not running.";echo;
     exit 1;
     else
           mgmt_cli -r --port \$MGPORT true show version 2>&1> /tmp/ebackup_login;
           if grep -q Gaia /tmp/ebackup_login; then
               echo -n "The SMS is running in version "\$RVER;echo ".";echo;
               serv_value="0";
           else
               echo "The SMS is running but a connect to management API isn't possible.";echo;
               exit 1;
           fi
     fi
fi
if [ \$serv_value -eq 1  ]; then
   mdsenv > /dev/null 2>&1;
   mdsstat | grep CMA | awk '{print \$6}' >> /tmp/ebackup_MGTIP
   mdsstat | grep MDS | awk '{print \$6}' >> /tmp/ebackup_MDSIP
fi
############ Read gateways     #########
if  [ \$serv_value -gt 0  ]; then
       echo "Read gateways from CMA: ";echo;
       while read -r hostip;
       do
           CMASTATUS_HA=\$(mdsstat | grep CMA| grep \$hostip | awk '{print \$8}');
           CMANAME_HA=\$(mdsstat | grep CMA| grep \$hostip | awk '{print \$4}');
           if [ \$CMASTATUS_HA = "up"  ]; then
               echo "   OK     - "\$CMANAME_HA;
               CMACOUNT=\$(mgmt_cli -r --port \$MGPORT true -d \$hostip show gateways-and-servers offset 0 limit 500 details-level full --format json | \$CPDIR/jq/jq -r '.objects[] | select(.type | contains("Member","simple-gateway")) | ."ipv4-address",."name"  ' |xargs -n 2| grep -c ".");
               if [ \$CMACOUNT != 0 ]; then
                     mgmt_cli -r --port \$MGPORT true -d \$hostip show gateways-and-servers offset 0 limit 500 details-level full --format json | \$CPDIR/jq/jq -r '.objects[] | select(.type | contains("Member","simple-gateway","CpmiClusterMember", "cluster-member")) | ."ipv4-address",."name"  ' |xargs -n 2 | grep -v 0.0.0.  | sed -e "s/\$/ \$hostip/"  >> /tmp/ebackup_topo;
               fi
           else
               echo " Failed   - "\$CMANAME_HA;
           fi
       done < /tmp/ebackup_MGTIP
       echo;
else
     # read SMS gateways
     mgmt_cli -r --port \$MGPORT true show gateways-and-servers details-level full limit 500 offset 0 --format json | \$CPDIR/jq/jq -r '.objects[] | select(.type | contains("Member","simple-gateway","CpmiClusterMember", "cluster-member")) | ."ipv4-address",."name"  ' |xargs -n 2 | grep -v 0.0.0. |grep -v null | sed -e "s/\$/ x.x.x.x/" 2>&1> /tmp/ebackup_topo;
fi
######## GAIA Config save all gateways   #########
if [ ! -f /tmp/ebackup_topo ]; then
   echo "No gateways found!";
else
   echo "clish -c 'show configuration' > /var/log/g_output.txt" > /tmp/g_command.txt;
   echo "Backup GAIA config gateways:";echo;
   while read -r REMOTE_IP REMOTE_NAME REMOTE_CMA;
     do
     REMOTE_FILE="\$NOW-\$REMOTE_NAME";
     REMOTE_DATEI="\$FILE_PATH/\$REMOTE_FILE";
     mdsenv \$REMOTE_CMA > /dev/null 2>&1;
     $CPDIR/bin/cprid_util -server \$REMOTE_IP putfile -local_file /tmp/g_command.txt -remote_file /var/log/g_command.txt > /dev/null 2>&1;
     $CPDIR/bin/cprid_util -server \$REMOTE_IP -verbose rexec -rcmd /bin/bash -f /var/log/g_command.txt > /dev/null 2>&1;
     $CPDIR/bin/cprid_util -server \$REMOTE_IP getfile -remote_file /var/log/g_output.txt -local_file \$REMOTE_DATEI > /dev/null 2>&1;
     #echo -n "";
     if [ ! -f \$REMOTE_DATEI ]; then
       echo -n " Failed   - ";
       else
       if grep -q hostname \$REMOTE_DATEI; then
          echo -n "   OK     - ";
          if [ \$REMOTE_INFO_V == "1" ] ; then
            $CPDIR/bin/cprid_util -server \$REMOTE_IP putfile -local_file /usr/bin/ebackup_command -remote_file /var/log/ebackup_command.txt > /dev/null 2>&1;
            $CPDIR/bin/cprid_util -server \$REMOTE_IP -verbose rexec -rcmd /bin/bash -f /var/log/ebackup_command.txt > /dev/null 2>&1;
            $CPDIR/bin/cprid_util -server \$REMOTE_IP getfile -remote_file /var/log/g_output1.txt -local_file /tmp/ebackup_gateway_info.txt > /dev/null 2>&1;
          fi
       else
          echo -n " Failed   - ";
       fi
     fi
     echo -n \$REMOTE_NAME;
     echo -n "   ";
     if [ \$REMOTE_INFO_V == "1" ] ; then
        if [ -f /tmp/ebackup_gateway_info.txt ]; then
           REMOTE_INFO=\$(cat /tmp/ebackup_gateway_info.txt);
           echo; echo -n "           " \$REMOTE_INFO;
           rm /tmp/ebackup_gateway_info.txt;
           else
           echo; echo -n "            (no gateway information available)";
        fi
     fi
     if [ \$serv_value -eq 1  ]; then
        CMANAME_HA=\$(mdsstat | grep \$REMOTE_CMA | awk '{print \$4}');
        echo;    echo -n "            (CMA: "\$CMANAME_HA;echo -n ")";
     fi
     echo;
    done < /tmp/ebackup_topo;
fi
echo;
######## GAIA config management save     #########
echo "Backup GAIA config Management:";echo;
REMOTE_FILE="\$NOW-Management";
REMOTE_DATEI="\$FILE_PATH/\$REMOTE_FILE";
clish -c 'show configuration' > \$REMOTE_DATEI;
     if [ ! -f \$REMOTE_DATEI ]; then
       echo -n "  Failed - ";
       else
       if grep -q hostname \$REMOTE_DATEI; then
          echo -n "   OK     - ";
       else
          echo -n " Failed   - "
       fi
     fi
     echo -n "Management   ";
     /usr/bin/ebackup_command;
     if [ \$REMOTE_INFO_V == "1" ] ; then
        if [ -f /var/log/g_output1.txt ]; then
           REMOTE_INFO=\$(cat /var/log/g_output1.txt);
           echo; echo -n "           " \$REMOTE_INFO;
           rm /var/log/g_output1.txt;
        fi
     fi
     echo;echo;

############## SMS ############################
# SMS migrate export
if [ \$serv_value -eq 0  ]; then
  if [ \$MIGRATE == "1" ] ; then
    REMOTE_FILE="\$NOW-SMS-Migrate-Export";
    REMOTE_DATEI="\$FILE_PATH/\$REMOTE_FILE";
    echo;echo "Migrate Export SMS:";echo;
    if [ \$SHOW_SUM == "1" ] ;
      then
        cpstop > /dev/null 2>&1;
        echo "   OK     - cpstop";
    fi
    REMOTE_DATEI_CHK="\$REMOTE_DATEI.tgz";

    $FWDIR/bin/upgrade_tools/migrate export -n \$REMOTE_DATEI 2>&1> \tmp\ebackup_migrate ;
       if [ ! -f \$REMOTE_DATEI_CHK ]; then
           echo " Failed   - SMS migrate export";
       else
           echo "   OK     - SMS migrate export";
       fi

    if [ \$SHOW_SUM == "1" ] ;
      then
        cpstart > /dev/null 2>&1;
        echo "   OK     - cpstart";
    fi
  fi
fi
############## MDS ############################
if [ \$MIGRATE == "1" ] ; then
 if  [ \$serv_value -gt 0  ]; then
   VER=`fwm ver | awk '{print \$8}' | sed 's/R80.//g'`
   if [ \$VER -lt 30  ]; then
        ### MDS version R80, R80.10, R80.20 migrate export
        REMOTE_FILE="\$NOW-MDS-Migrate-Export";
        REMOTE_DATEI="\$FILE_PATH/\$REMOTE_FILE";
        mds/tmp/ebackup_migrateenv > /dev/null 2>&1
        echo -n "MDS migrate export (migrate export) "\$RVER;echo ":";echo;
        if [ \$SHOW_SUM == "1" ] ;then
           mdsstop > /dev/null 2>&1;
           echo "   OK     - MDS mdsstop";
        fi
        \$MDS_FWDIR/bin/upgrade_tools/migrate export -n \$REMOTE_DATEI 2>&1> /tmp/ebackup_migrate ;
        if [ ! -f \$REMOTE_DATEI_CHK ]; then
           echo " Failed   - MDS R80, R80.10, R80.20 (migrate export)";
        else
           echo "   OK     - MDS R80, R80.10, R80.20 (migrate export)";
        fi
        if [ \$SHOW_SUM == "1" ] ;then
           mdsstart > /dev/null 2>&1;
           echo "   OK     - MDS mdsstart";
        fi
   else
        ### MDS version R80.30+ migrate_server
        REMOTE_FILE="\$NOW-MDS-Migrate-Server";
        REMOTE_DATEI="\$FILE_PATH/\$REMOTE_FILE";
        mdsenv > /dev/null 2>&1;
        echo;echo -n "MDS migrate server (migrate_server) "\$RVER;echo ":";echo;
        \$MDS_FWDIR/scripts/migrate_server export -skip_upgrade_tools_check -n -v \$RVER \$REMOTE_DATEI 2>&1> /tmp/ebackup_migrate ;

            if grep -q "sk135172" /tmp/ebackup_migrate; then
                echo " Failed   - MDS R80.30+ migrate_server export (No upgrade tools package is installed refer to sk135172)";
            else
                if [ ! -f \$REMOTE_DATEI ]; then
                    echo " Failed   - MDS R80.30+ migrate_server export";
                else
                    echo "   OK     - MDS R80.30+ migrate_server export";
                fi
            fi
   fi
 fi
fi
############## TAR ############################
echo;echo "Tar file location:";echo;
REMOTE_FILE="\$NOW-ebackup";
REMOTE_DATEI="/var/log/\$REMOTE_FILE.tgz";
REMOTE_TAR="\$FILE_PATH";
# echo \$REMOTE_TAR
# echo \$REMOTE_DATEI

tar -czf \$REMOTE_DATEI \$REMOTE_TAR> /dev/null 2>&1;

     if [ ! -f \$REMOTE_DATEI ]; then
            echo " Failed   - tar file";
       else
          echo -n "   OK     - tar file (" ;
          echo -n \$REMOTE_DATEI;
          echo ")";
     fi
echo;
############## FTP ############################
if [ "X\$FTPSERVER" != "X" ]; then

   FTPDATEI="\$REMOTE_FILE.tgz";
   echo;echo "FTP upload:";echo;
   echo "
   verbose
   user \$FTPUSER \$FTPPW
   bin
   lcd /var/log
   put \$FTPDATEI
   bye
   " | /usr/bin/ftp -in \$FTPSERVER > /tmp/ebackup_ftp 2>&1;
   if grep -q 226 /tmp/ebackup_ftp; then
       echo "   OK     - FTP upload";
   else
       echo " Failed   - FTP upload (Log: /tmp/ebackup_ftp)";
   fi
   echo;
fi

############## CPUPLOAD ############################
if [ "X\$CPUPLOAD" != "X" ]; then


     #REMOTE_FILE="\$NOW-\$REMOTE_NAME";
     #REMOTE_DATEI1="\$FILE_PATH/\$REMOTE_FILE";
     CPUPLOADDATEI="\$REMOTE_DATEI";
     echo "CP upload to \$CPUPLOAD:";echo;
     echo "ls \$CPUPLOADDATEI > /var/log/g_output.txt" > /tmp/g_command.txt;
     $CPDIR/bin/cprid_util -server \$CPUPLOAD putfile -local_file /tmp/g_command.txt -remote_file /var/log/g_command.txt > /dev/null 2>&1;
     $CPDIR/bin/cprid_util -server \$CPUPLOAD putfile -local_file \$CPUPLOADDATEI -remote_file \$CPUPLOADDATEI  > /dev/null 2>&1;
     $CPDIR/bin/cprid_util -server \$CPUPLOAD -verbose rexec -rcmd /bin/bash -f /var/log/g_command.txt > /dev/null 2>&1;
     $CPDIR/bin/cprid_util -server \$CPUPLOAD getfile -remote_file /var/log/g_output.txt -local_file /tmp/ebachup_cpupload > /dev/null 2>&1;
     if [ ! -f /tmp/ebachup_cpupload ]; then
       echo " Failed   - CP upload to \$CPUPLOAD ";
       else
       if grep -q tgz /tmp/ebachup_cpupload; then
          echo "   OK     - CP upload file (\$CPUPLOADDATEI)";

       else
          echo " Failed   - CP upload to \$CPUPLOAD";
       fi
     fi
     echo;
fi



REMOTE_RM="\$FILE_PATH/202*";
rm \$REMOTE_RM > /dev/null 2>&1;
rm /tmp/g_command.txt > /dev/null 2>&1;
rm /tmp/ebackup_topo > /dev/null 2>&1;
rm /tmp/ebachup_cpupload > /dev/null 2>&1;
rm /tmp/ebackup_MGTIP > /dev/null 2>&1;
rm /tmp/ebackup_migrate > /dev/null 2>&1;
rm /tmp/ebackup_mdsstatus > /dev/null 2>&1;
rm /tmp/ebackup_login > /dev/null 2>&1;
EOT
cat <<EHA > /usr/bin/ebackup_command
#!/bin/bash
J=\`cpinfo -y all 2>/dev/null | grep JUMBO | tail -n1 | awk '{print \$3}'\`;
O=\`cat /etc/cp-release | cut -c 13- | sed 's/^ *//g' | sed 's/\s*$//g'\`;
OSM=\`uname -a | grep -c x86_64\`; [ \$OSM == 0 ] && { OSM=32; } || { OSM=64; };
K=\`uname -r | sed 's/\([0-9]\+\.[0-9]\+\)\..*/\1/'\`;
T=\`cpstat os | grep "Appliance Name" | tr -s ' ' | cut -c 17- \`;
echo "(\${O}, Kernel \${K}, \${OSM} Bit, JHF \${J}, \${T})" > /var/log/g_output1.txt ;
EHA
chmod 770 /usr/bin/ebackup;
chmod 770 /usr/bin/ebackup_command;

Create Cronjob

If the ebackup tool should be executed automatically, only a cronjob has to be created.

Click to Expand

Add a user "jobuser" via GAIA web portal:


Set shell:                  /bin/bash
Set assigned role:  adminRole



Set a new password:


FTP upload:

                Add a cronjob via CLI to execute ebackup at Saturday night at 10:00 pm.

                - Add parameter -s for cpstop/cpstart
                - Add ftpserver/ ftpuser and ftppw parameters for ftp upload.

                # crontab -u jobuser -e
                ------------------------------------------------------------------------------------------------------
                22  00  *  *  6   /usr/bin/ebackup -s -ftpserver <server_ip> -ftpuser <username> -ftppw <password>

CP upload:

If an FTP upload is too insecure for you, you can also transfer the backup file to another Check Point system with the option -cpupload via cprid_util.

- Add parameter -s for cpstop/cpstart
- Add ftpserver/ ftpuser and ftppw parameters for ftp upload.

# crontab -u jobuser -e
------------------------------------------------------------------------------------------------------
22  00  *  *  6   /usr/bin/ebackup -s -cpupload <cp_server_ip>

More "Easy Tools"

- Easy Backup Tool - (migrate export + all GAIA configs)                   -> Easy backup of all gateway GAIA configs + migrate export with one CLI command.
- Easy execute CLI commands on all gateways simultaneously        ->  Now you can use the new command to execute bash or clish commands on all gateway simultaneously.
- Easy execute CLI commands from management on gateways        -> Easy execute CLI commands from management on gateways
- Mobile User License Tool - replaced "dtps lic"                                   -> It displays all Secure Client, SSL VPN and Mobile Access Portal licenses in total (sum) on the SMS.
- Easy View Tool - (system infos from all gateways simultaneously) -> This toll shows you quickly an overview of status information of all your gateways with only one CLI command.

Version

0.1   03-15-2020   - oneliner to show backup clish configs
0.6   03-23-2020   - GA version ebackup
0.7   03-25-2020    - add parameter -s for cpstop/cpstart
0.8   03-26-2020   - add parameter -no_migrate  (no migrate export)
0.9   03-26-2020   - bug fixed  (special thanks to Paul_Gademsky)
1.0   03-27-2020   - bug fixed  (SMS and MDS)
1.1   03-30-2020   - add option -port
1.2   03-30-2020   - bug fixed
1.3   03-31-2020   - ftp upload function  (-ftpserver, -ftpuser, -ftppw)

2.0   04-04-2020   - MDS support
2.1   04-06-2020   - bug fixed
2.2   04-07-2020   - add option -v
2.3   04-09-2020   - add option -cpupload (upload tgz file to a other Check Point gateway or SMS)

3.0   06-20-2023   - Support for R81.20

[R80]

This command will save all gaia configurations of all gateways. Do iI understand this correctly?

[HeikoAnkenbrand]

Hi @R80 

That's right!

It reads all gateways (not SMB appliances) from the topology and saves their GAIA configurations in a tar achive.

PS:
Nice username @R80.

[HulugL]

Wow, great tool @HeikoAnkenbrand 

[Paul_Gademsky]

Heiko, another great script that you've created.

I've run it on several different systems to test, and have varying results.

On a SMS (R80.30) with 3 gateways (1 single and 2 clustered), it found the single, but did not find the cluster. Looked at the topo file and it only shows the single gateway. Have you tested it against a cluster?

On a SMS (R80.40) it found the local gateway and was successful, but Failed - checkpoint_cloud_security_1408377501  (not surprised at all at that failure). I'd consider this a success.

When I ran it on my R80.40 MDS (I know this is for SMS, but figured I'd run it anyways), it found no gateways (just "Failed -", this was because I did not run a 'mdsenv' first - was in a cma. after mdsenv, I did get a partial list of gateways from one CMA.  It did succeed in backing up the MDS GAIA config, and thinks it did a migrate export successfully(don't think it really did, as the cpstop said it was successful, but mdsstat never showed the fwm (or other processes) going down

I'll dig into this some more, and see what I can figure out for the MDS (I have a backup script that does all the gw's in the MDS and transfers the files to the management already (though via scp and requires an expert login one time).

Other question is should it be using 'migrate_server export -v R80.xx' vs 'migrate export' as that seems to be the new direction (/opt/CPsuite-R80.40/fw1/scripts/migrate_server)?

Thank you,

Paul G.

[HeikoAnkenbrand]

Hi @Paul_Gademsky,

It is not for a MDS only for SMS.

I tested it with R80.10, R80.20 and R80.30 and it works without problems.

Can you run the following command and see if the IP and the gateway are output:

mgmt_cli -r true show gateways-and-servers details-level full --format json | $CPDIR/jq/jq -r '.objects[] | select(.type | contains("Member","simple-gateway")) | ."ipv4-address",."name"  ' |xargs -n 2 | grep -v null | grep -v 0.0.

[ruggy]

It works fine:

[Paul_Gademsky]

Just re-ran the command provided on a R80.30 SMS with 1 single gateway and 1 cluster(HA made up of 2 5600's running R80.30).

I only get one result, and that's for the single gateway.

In digging into the object via GUIDBEDIT, the class is 'cluster_member' and type is 'cluster_member'.

I'm thinking that is what the issue is.

The single gateway comes up with a class of 'gateway_ckp' and a type of 'gateway'

Paul G.

[HeikoAnkenbrand]

Hi @Paul_Gademsky 

Thanks for the information.

I search in my script for the following in the type field:

- Member
- simple-gateway

Could you please execute the following CLI command.
It shows the following:   IP address,     gateway name,       type

I need the "type" field that is found at the gateway.

mgmt_cli -r true show gateways-and-servers details-level full --format json | $CPDIR/jq/jq -r '.objects[] | select(.type) | ."ipv4-address",."name",."type"  ' | xargs -n 3

Then I can add it to this script.

Here an example:

[HeikoAnkenbrand]

Hi @Paul_Gademsky,

Several of our customers have tested this script over the last few days. None of them have this issue.

I have also looked at this with the oneliner described above.

There are the following types:

simple-gateway                     > Gateway Object
CpmiClusterMember            > Cluster Gateway Object
CpmiGatewayCluster            > Cluster Object
CpmiHostCkp                         > Management Server SMS or Log Server
CpmiSofawareGateway         > old SMB Gateways

I am searching for the following object types in the script:

- simple-gateway
- Member

Thus all gateways should be found.

[Paul_Gademsky]

@HeikoAnkenbrand 

Ran the command per your request and got the following (R80.30 SMS)

[Expert@cp_mgmt:0]# mgmt_cli -r true show gateways-and-servers details-level full --format json | $CPDIR/jq/jq -r '.objects[] | select(.type) | ."ipv4-address",."name",."type" ' | xargs -n 3
172.22.0.1 Gresham simple-gateway
172.30.0.2 cp_mgmt CpmiHostCkp
172.30.0.1 Park_FW_Cluster CpmiGatewayCluster
172.30.0.3 fw CpmiClusterMember
172.30.0.4 fw1b CpmiClusterMember

What I see for the cat /tmp/ebackup_topo
172.22.0.1 Gresham

And for the actual run feedback

[Expert@cp_mgmt:0]# ebackup
------------------------------------------------------------------------------
- Easy Backup Tool v0.7 - Copyright Heiko Ankenbrand 2020 -
------------------------------------------------------------------------------

Backup GAIA config gateways:

OK - Gresham

Backup GAIA config SMS:

OK - SMS

Migrate Export SMS:

[Paul_Gademsky]

@HeikoAnkenbrand 

After doing more testing, it looks like the "grep -v 0.0." in the string is what was causing the gateways to not show up.

So that problem is solved for me.

I was having an issue with only some of the gateways on a different SMS showing up in in ebackup_topo (about 40 out of 73).  I was able to remedy this with adding to the mgmt_cli the following "offset 0 limit 500". I was then able to get all the gateways to show up.

I still have a few other questions on the script itself though.

These are basic bash questions, about the script construction that are unclear:

NOW=\$(date +"%Y_%m_%d");  is what you originally called. I get an error when \ is in there.

[Expert@cp_mgmt:0]# ebackup
/usr/local/bin/ebackup: line 4: syntax error near unexpected token `('
/usr/local/bin/ebackup: line 4: `NOW=\$(date +"%Y_%m_%d");'

If I remove the \ it seems to run fine.

On the mSITIONAL=(), I'm assuming that is the increment for the devices in the ebackup_topo.

The

while [[ \$# -gt 0 ]]

causes the following

/usr/local/bin/ebackup: line 12: [[: $#: syntax error: operand expected (error token is "$#")

If I remove the \ from the while command, it executes properly.

Can you enlighten me on your usage of the \ in these cases and why they may be causing issues in R80.30/40?

Thank you again for the script, and chasing the issues.

Paul G.

[HeikoAnkenbrand]

Hi @Paul_Gademsky 

First of all many thanks for the support.

I use the following syntax to parse the script into a file. If I do not use the / in front of a variable the value will be parsed into the script. 

cat <<EOT  > /usr/local/bin/ebackup
script
EOT

I have modified the line in the script a little bit. Can you try it again.

Regards
Heiko

[Paul_Gademsky]

@HeikoAnkenbrand 

Tried the modified command and got the following:

[Expert@cp_mgmt:0]# mgmt_cli -r true show gateways-and-servers details-level full limit 500 offset 0 --format json | $CPDIR/jq/jq -r '.objects[] | select(.type) | ."ipv4-address",."name",."type" ' | xargs -n 3 | grep -E 'simple-gateway|Member' | grep -v '0\.0\.' | awk '{print $1 " " $2}'
172.22.0.1 Gresham

[Expert@ohs_cp_mgmt:0]# mgmt_cli -r true show gateways-and-servers details-level full limit 500 offset 0 --format json | $CPDIR/jq/jq -r '.objects[] | select(.type) | ."ipv4-address",."name",."type" ' | xargs -n 3 | grep -E 'simple-gateway|Member' | grep -v '0.0.' | awk '{print $1 " " $2}'
172.22.0.1 Gresham

[Expert@ohs_cp_mgmt:0]# mgmt_cli -r true show gateways-and-servers details-level full limit 500 offset 0 --format json | $CPDIR/jq/jq -r '.objects[] | select(.type) | ."ipv4-address",."name",."type" ' | xargs -n 3 | grep -E 'simple-gateway|Member' | awk '{print $1 " " $2}'
172.22.0.1 Gresham
172.30.0.3 fw
172.30.0.4 fw1b

So the -v is still killing it. What are you trying to omit with the -v?

And now the spelling police statement 😉

Also, on the REMOTE_FILE="$NOW-SMS-Migarte-Export";  s/b Migrate

I also had a conversation with CP support on the 'migrate export' vs 'migrate_server export'.

The new 'migrate_server export' has been available since R80.20 and can be called via $MDS_FWDIR (even on a SMS where you'd normally call $FWDIR)  SMS path is /opt/CPsuite-R80.40/fw1/scripts/migrate_server and MDS path is /opt/CPsuite-R80.40/fw1/scripts/migrate_server.

Paul G.

[HeikoAnkenbrand]

If I don't execute a "grep -v 0.0." all SMB appliances will be included and they can't use a cprid_util cli command.

I have tested this on many SMS today and none had the problem with grep -v "0.0.". I don't really understand why this is so.

---

In the next few days I will be able to modify it accordingly for a MDS and SMS. I could also add a parameter -migrate_server to distinguish between "migrate export" and "migrate_server".

[Paul_Gademsky]

@HeikoAnkenbrand 

The grep -v 0.0 is catching the IP address of the gateway and that's why it's having a problem -  172.30.0.1 is an example. If you specify which part of the results it greps, rather than all the data, it might work better.

On the SMS/MDS version, I've been working on that, and have everything but the export sorted out at this point.

I'm doing a R80 to R80.10 and an R80.20 and up fork to handle that, so don't spend to much time. I'll have it uploaded by Monday COB.

Paul G.

[HeikoAnkenbrand]

Hi @Paul_Gademsky ,

👍

I will add grep -v 0.0.0.

Many thanks for the support. Your effort was great.

Regards Heiko

[Paul_Gademsky]

@HeikoAnkenbrand 

So worked this some more over the weekend and have the resulting script attached (version 1.1?)

This works with all versions of R80 released so far.

In particular, I've tested on the following

R80.40 MDS  and SMS

R80.30 MDS and SMS

R80.20 MDS

R80.10  SMS

Several features of the new version attached

  Tests to see if SMS or MDS

  Get's management IP (and if more than one interface on a SMS removes the additionals so that only one IP is in table, otherwise script will replicate gateways to be backed up). This is necessary for the usage in the mgmt_cli -d X.X.X.X portion below

PS:

I noticed when I ran the curl for version 1.0 some issues when it came across:

  CPM_PID put the actual process value in the script instead of the path

  The $FWDIR/bin/cprid_util grabbed the actual path (in the case I looked at /opt/Cshrd-R80.40/bin/cprid_util

  REMOTE_FILE="$NOW-SMS-Migarte-Export" still needs to be Migrate

  The $FWDIR/bin/ugrade_tools/migrate grabbed the actual path (in the case I looked at /opt/CPSuite-R80.40/fw1/bin/upgrade_tools/migrate

If the procces ID changes after a restart then the script may break.  In a case where users don't have access to the script and want to copy it from one system to another, if the versions are different, it may break as well.

  The path for the export file is showing \tmp\ebackup_migrate  s/b /tmp/ebackup_migrate

I noticed that the offset 0 limit 500 was removed from the active command (it failed to get all gateways again on a system with a large number of gateways)

I'd recommend that the -s be deprecated, as backing up a SMS without running a cpstop/cpstart if a admin is logged in is not recommended.  I think the intent is to run this as a cron job in most cases after hours, so I don't see a big impact. In R80.30/R80.40 the 'migrate_server' runs those commands by default.

I'm trying to figure out how to modify or add another 'mgmt_cli' command that will grab the other Management servers to backup the GAIA configs as well, and add it to the ebackup_topo file.  This would be for Mgmt_HA and LogServers. Any help on that is appreciated.

Let me know what you think of these changes.

Paul G.

[Paul_Gademsky]

Testing in another MDS environment today, and it looks like the cprid_util has to be set to the mdsenv before it can run the command properly and get a response. Will work to figure out how to get this implemented. Thinking that the ebackup_topo can have a third field/column with the relevant CMA info that can be triggered before the call.  The interesting thing is that the Mgmt/HA, log servers and SE server work with no domain (as they should).

Paul G.

[Paul_Gademsky]

@HeikoAnkenbrand 

I've worked through the mds issues I was having and have finished a 1.4 version that additionally supports the MDS/CMA structure.  This includes finding the mds/ha, mlm servers and SmartEvent servers that are tied to a mds configuration and backs up their configuration.

It includes the changes for -port and -ftpserver as well (though I'll probably change that to sftp for my own usage)

I've run it on the following (the problem with the previous version I created was cprid needs to be called per domain/cma rather than globally, which is now handled).

Please take a look and let me know if you want to release it as 1.4.

Thanks again for starting this.

Paul G.

[HeikoAnkenbrand]

Hi @Paul_Gademsky,

First of all thanks for the support. I think it is good when users participate in projects and have good ideas.

In version 2.0 I have removed some bugs that may occur:

1) If an MDS or SMS does not contain gateways the table /tmp/ebackup_topo was filled with wrong values. This caused a grep error.

2) I have developed a few more parser issues.

3) I now check if the CMA's are up, otherwise wrong information are written to /tmp/ebackup_topo.

4) I also included the Check Point variables (path informations). So you can start the script as cronjob.

Also I added some new options -l, -d, -u.

Furthermore I have built in a FTP upload with option -ftpserver, -ftpuser and -ftppw

In version 2.2 I will add the option -m (-ms) to choose between "migrate export" and "migrate_server". Thus the user has the possibility to choose. 

[Yury_Anoshyn]

hello, @HeikoAnkenbrand 

what about capture configs from VSX clusters & objects ?

[Paul_Gademsky]

Interesting question, I haven't looked into the structure on the VSX.

What issues did you see when you ran the script against the systems?

Try the 1.4 that I just posted if you are running an mds as well.

Paul G.

[Yury_Anoshyn]

First of all, I apologize for the delay in answering 🙂

i'm not run script on the VSLS VSX yet. But expect issue with few points.

Point1.

For catching config from gaia clish "ebackup" is use next cmd: clish -c "show configuration". But it will be correct only for non VSLS VSX GW. Because in VSLS mode in configuration are present VS ( virtual system ) specific configuration. In VSLS GW can be more then one VS object. As i know, VS specific part of configuration is atleast OSPF and another routing configuration, BOOTP. May be some else also, i'm not know exactly full list.

We need catch configuration from all VS objects, that mean we need use another clish argument "-f". It is using file with cmd list.

what i mean ? as example: we have VSLS VSX GW with 3 VS .

create file with name filecmdname and that contain next output:

set virtual-system 0
show configuration
set virtual-system 1
show configuration
set virtual-system 2
show configuration
set virtual-system 3
show configuration

then run cmd
clish -f filecmdname > allVScfg.txt

Point2.

before we start capture configuration from GW - we need check on what kind of GW we will run clish command. Then in case non VSLS we use "-c" argument. in case VSLS - we need run cphaprob stat, then parse output and catch list of VS IDs. create cfg file like listed above and run clish with "-f" argument.

Point3.

Result file will contain 90% of the same configuration lines and 10% VS specific lines. Шn my opinion it is necessary to divide the contents of teh result configuration file into at least 2 parts. The first contains common, identical configuration lines. The second contains parts distinguishable for each of the VS.

Point4. 

 This point is not VSLS specific 😁

some configuration lines are contain passwords. At least cmd "backup-scheduled"  listed in output of clish cmd with **** instead of password.

[Yury_Anoshyn]

it's me again 🙂

i find script writen by @Niels_van_Sluis  there Save virtual server configurations on a VSX member 

and this script capture not only VSX with al VS gaia configuration , but also additional important cfg files.   by my opinion  "ebackup" script and script from @Niels_van_Sluis  need be combined to one. 

[Mstay]

Hi Heiko

Is the Script working from R81.10 JHF take 55? Since that take is not taking the  show configuration ´s backup of the Gws attached to the SMS.

Regards

[Nikolai_Borhart]

Hello Heiko

first of all a big thank you for another great tool here for the Check Mate community.

So if I understood it correctly.

1. Download the script to the SMS.
2. Execute script.

And then you have the clish configuration backup of all gateways and SMS on the Managment Server and also the migrate export/import backup.

Does the Check Point Migrate Tool need to have a specific version or does it have to be updated?
Because when migrating from R77.x to R80.x the migration tool had to be uploaded to the management server first.

best regards

Nikolai

[HeikoAnkenbrand]

Hi @Nikolai_Borhart 

I use the default migrate tool from the path:
$FWDIR/bin/upgrade_tools/migrate

[_Val_]

Impressive

[ute]

Hi @HeikoAnkenbrand 

We have an environment with about 40 Gateway.  It saves us a lot of work to backup all GAIA settings.

The tool works fine here.

Maybe you can add a parameter to not execute the "migrate export". That would be great.

Thanks for this bash script.

Great work