Cronjob script to schedule MDS Backup

[Anil_Kumar2]

Hi, can someone help me build a cronjob for the MDS_backup without the logs?

I do not know the script to build this cronjob. I would appreciate if someone could help me with the script, to schedule the job every Friday morning 8:15 AM.

Appliance: SMART-1 50 Appliance

Thanks in advance.

[Maarten_Sjouw]

For the backup script itself you should be able to find many examples here or over on CPUG.org.

For adding the command to cron use the following clish command:

  add cron job MDS_backup command "/backup/scripts/BackupMDS >/dev/null 2>&1" recurrence weekly days 5 time 8:15 

Where /backup/scripts/BackupMDS is you backup script file and location. For the day it is number 0-6 where Sunday = 0 then the Friday = 5. 

[Kaspars_Zibarts]

The easiest is to use standard GAIA backup - it will backup MDS config automatically without logs and you can schedule it in the webui

I'm guessing you were looking for this line in cronjob if you wanted to make it really simple but not that "nice". Assuming that you are running R80.10

15 08 * * 5 /opt/CPmds-R80/scripts/mds_backup -b -l >/dev/null 2>&1

it will run mds backup in batch mode and no logs. You might want to add -s flag to stop MDS whilst running backup.

to add the crontab as non admin user refer to sk77300. (in nutshell crontab -e -u <backup_user_name>)

Else create a bash script from command above and schedule it via webui or cli

[Marcel_Wildenbe]

I always thought the standard GAIA System Backup is agnostic of the fact that it is running MDS or not.

True or false?

[Kaspars_Zibarts]

No (if I understood your question correctly) - if GAIA sees MDS running, it will add mds_backup on top of regular GAIA backup content.

[Marcel_Wildenbe]

Ok thanks. Too bad the documentation is not mentioning this.

[Kaspars_Zibarts]

I wasn't aware of that so had to check   You're absolutely right! It does not explicitly state that GAIA backup will perform full MDS backup except this row table in SK108902. But then you have to read "between lines" that product configuration = mds_backup

Best Practices - Backup on Gaia OS 

[Martin_Valenta]

Well with r80.10 there is a small but..with r80.10 majority of people using jump server to get faster SmartConsole connection and usually they keep open SmartConsole opened even during weekends..

[Kaspars_Zibarts]

We kick users out after 2hrs of being idle

[Martin_Valenta]

Yes this is one of options, also on Windows server you can set it to log off idle sessions, but simply some people are just lazy to login to SmartConsole everyday..:)

[Kaspars_Zibarts]

dangerous...  in case Putin gets his hands on your TS  actually I really like the timeout option as it was not there in R77

[Jason_Carrillo]

Kaspars, I see that you are recommend using the web UI backup. Do you know of any reason why my CMAs wouldn't be included in this backup?

[Marcel_Wildenbe]

I am testing this today. I still have my doubts a system backup (CLI or WebUI) will include MDS.

[Kaspars_Zibarts]

 - we have used it for years, restoring it in the lab weekly. Works like a charm. A bit slow for us in R80.10 - we have open TAC case that's apparently very close to be resolved. Our backup size went up from 3GB to 17GB so it takes time pack/unpack such big archives.

Here's backup log btw

[Thu Sep 28 07:35:51 2017]: <<<<<< Start Backup >>>>>>
[Thu Sep 28 07:35:51 2017]: Backup parameters:
[Thu Sep 28 07:35:51 2017]: 1. 'file_list' -> 'SCALAR(0x8c995dc)'
[Thu Sep 28 07:35:51 2017]: 2. 'package_path' -> ''
[Thu Sep 28 07:35:51 2017]: 3. 'group_name' -> 'all'
[Thu Sep 28 07:35:51 2017]: 4. 'backup_type' -> 'backup'
[Thu Sep 28 07:35:51 2017]: 5. 'package_name' -> 'backup__28_Sep_2017_07_35'
[Thu Sep 28 07:35:54 2017]: Info: 'generic_backup': Changing to directory /var/log/CPbackup/backups
[Thu Sep 28 07:35:54 2017]: Status: Collecting information...
[Thu Sep 28 07:35:56 2017]: Scheme groups ID table:
[Thu Sep 28 07:35:56 2017]: 1. 'system' -> 'CPsystem0001'
[Thu Sep 28 07:35:56 2017]: 2. 'cp_products' -> 'CPproducts_dlp_gw, CPproducts_dtps, CPproducts_te, CPproducts_fg1, Provider-1, CPproducts_mgmt, CPproducts_ppak, CPproducts_cvpn, CPproducts_svn, CPproducts_rt, CPproducts_rtm, CPproducts_fw1logs, CPproducts_fw1, CPproducts_uag, CPproducts_vsx'
[Thu Sep 28 07:35:56 2017]: 3. 'snapshot' -> 'CPsnapshot0001'
[Thu Sep 28 07:35:56 2017]: 4. 'all' -> 'CPsystem0001, CPproducts_dlp_gw, CPproducts_dtps, CPproducts_te, CPproducts_fg1, Provider-1, CPproducts_mgmt, CPproducts_ppak, CPproducts_cvpn, CPproducts_svn, CPproducts_rt, CPproducts_rtm, CPproducts_fw1logs, CPproducts_fw1, CPproducts_uag, CPproducts_vsx'
[Thu Sep 28 07:35:56 2017]: Scheme files ID table:
[Thu Sep 28 07:35:56 2017]: 1. 'CPproducts_fw1logs' -> 'fw1logs.cpbak'
[Thu Sep 28 07:35:56 2017]: 2. 'CPproducts_vsx' -> 'vsx.cpbak'
[Thu Sep 28 07:35:56 2017]: 3. 'CPproducts_cvpn' -> 'cvpn.cpbak'
[Thu Sep 28 07:35:56 2017]: 4. 'CPproducts_dtps' -> 'dtps.cpbak'
[Thu Sep 28 07:35:56 2017]: 5. 'CPproducts_rt' -> 'rt.cpbak'
[Thu Sep 28 07:35:56 2017]: 6. 'Provider-1' -> 'mds.cpbak'
[Thu Sep 28 07:35:56 2017]: 7. 'CPproducts_svn' -> 'svn.cpbak'
[Thu Sep 28 07:35:56 2017]: 8. 'CPproducts_ppak' -> 'ppak.cpbak'
[Thu Sep 28 07:35:56 2017]: 9. 'CPsystem0001' -> 'system_configuration.cpbak'
[Thu Sep 28 07:35:56 2017]: 10. 'CPproducts_fw1' -> 'fw1.cpbak'
[Thu Sep 28 07:35:56 2017]: 11. 'CPproducts_rtm' -> 'rtm.cpbak'
[Thu Sep 28 07:35:56 2017]: 12. 'CPproducts_te' -> 'te.cpbak'
[Thu Sep 28 07:35:56 2017]: 13. 'CPsnapshot0001' -> 'snapshot.cpbak'
[Thu Sep 28 07:35:56 2017]: 14. 'CPproducts_uag' -> 'uag.cpbak'
[Thu Sep 28 07:35:56 2017]: 15. 'CPproducts_mgmt' -> 'mgmts.cpbak'
[Thu Sep 28 07:35:56 2017]: 16. 'CPproducts_fg1' -> 'fg1.cpbak'
[Thu Sep 28 07:35:56 2017]: 17. 'CPproducts_dlp_gw' -> 'dlp_gw.cpbak'
[Thu Sep 28 07:35:56 2017]: INFO: include_files_list: /config/db/* /etc/udev/rules.d/00-*.rules /var/lib/net-snmp/snmpd.conf /etc/sysconfig/os_edition /opt/CPsuite-R80/fg1/conf/* /opt/CPsuite-R80/fg1/scripts/* /opt/CPsuite-R80/fg1/boot/modules/* /opt/CPsuite-R80/fg1/log/* /var/log/mdsbackup.tgz /var/opt/CPshrd-R80/registry/* /var/opt/CPshrd-R80/conf/* /opt/CPshrd-R80/database/* /var/opt/CPshrd-R80/log/* /opt/CPshrd-R80/database/postgresql/data/*.conf /opt/CPrt-R80/scripts/* /opt/CPrt-R80/conf/* /var/opt/CPrt-R80/Database/* /opt/CPrt-R80/log/* /var/opt/CPmds-R80/conf/* /var/opt/CPmds-R80/database/* /opt/CPmds-R80/lib/*.pf /var/opt/fw.boot/* /opt/CPsuite-R80/fw1/dlp/config/dlp.conf

[Kaspars_Zibarts]

Check your $MDSDIR/conf/mds_exclude.dat file

How to exclude CMAs from mds_backup script 

[Jason_Carrillo]

Well poop, there it is. Thanks Kaspars!

[Kaspars_Zibarts]

Too easy, happy to help!

[Jason_Carrillo]

Anything I have to do to force the CMAs to be restored? Just running mds_restore isn't seeming to work on my test VM. I looked within the tgz file and the CMAs are there.

[DanielS]

Did your physical box have a different interface name in use such as Mgmt, which is not by default in the virtual machine.

[Jason_Carrillo]

Interfaces are all the same. 

[Kaspars_Zibarts]

Depends really what you mean by "isn't working"  we need some logs and what are you doing as earlier you wanted to do full GAIA backup/restore and now you are talking mds_restore. Give us some clues

If you are running mds_restore remember to copy all necessary files to /var/log/ (actual mds_restore, gtar and gzip from memory) and you must configure the base of GAIA - at least leading interface must be the same as on machine backup was taken.

If you use GAIA restore, it should take care of most of it, but as Daniel said - make sure interface name macthes.

And in both cases - make sure hotfix level is the same as in production!

[Jason_Carrillo]

Meh, I think it is a version issue:

Production:

This is a Check Point Security Management Server R80.10 - Build 011

Test VM:

This is a Check Point Security Management Server R80.10 - Build 023

Below is what I get right before it tells me that the installation was successful:

tmp/installed_hotfixes_log

Restoring the Multi-Domain Server
----------------------------------------------
tar (child): mds_backup_opt.tgz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
./gtar: Child returned status 2
./gtar: Error is not recoverable: exiting now

mds_restore> Failed to restore the static information of the Multi-Domain Server

tar (child): mds_backup_varopt.tgz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
./gtar: Child returned status 2
./gtar: Error is not recoverable: exiting now

mds_restore> Failed to restore the variable information of the Multi-Domain Server

Reading configuration file from /opt/CPsuite-R80/fw1/conf/serverSettings.props
variable NGM_CPM_POSTGRES_CHECKPOINT_SEGMENTS exists
variable name: checkpoint_segments
variable value: 8
variable NGM_CPM_POSTGRES_SHARED_BUFFERS exists
variable name: shared_buffers
variable value: 32MB
variable NGM_CPM_POSTGRES_TEMP_BUFFERS exists
variable name: temp_buffers
variable value: 32MB
variable NGM_CPM_POSTGRES_WORK_MEM exists
variable name: work_mem
variable value: 32MB
variable NGM_CPM_POSTGRES_EFFECTIVE_CACHE_SIZE exists
variable name: effective_cache_size
variable value: 784MB
variable NGM_CPM_POSTGRES_MAINTENANCE_WORK_MEM exists
variable name: maintenance_work_mem
variable value: 128MB
variable NGM_CPM_POSTGRES_MAX_CONNECTIONS exists
variable name: max_connections
variable value: 200
variable name: default_transaction_isolation
variable value: 'repeatable read'
Finished to run successfully postgres_configure.sh
Running pg_ctl start
waiting for server to start.... done
server started
Running pg_ctl reload
server signaled
Running pg_ctl stop
waiting for server to shut down.... done
server stopped
The restoration of the SmartLog Server directories in Domain Management Servers environment completed with status 0
tar (child): mds_backup_supkgs.tgz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
./gtar: Child returned status 2
./gtar: Error is not recoverable: exiting now

mds_restore> Failed to restore the packaging information of the Multi-Domain Server

/bin/rm: cannot remove `mds_backup_opt.tgz': No such file or directory
/bin/rm: cannot remove `mds_backup_varopt.tgz': No such file or directory
/bin/rm: cannot remove `mds_backup_supkgs.tgz': No such file or directory
Checking the log directory of the Multi-Domain Server

The restoration of the Multi-Domain Server completed successfully..

If the Multi-Domain Server was running before the restoration,
you will need to rerun it.

mds_restore> Multi-Domain Server restoration ended successfully !

[Kaspars_Zibarts]

Just to rule out basics: make sure you run exactly the same hotfix level in VM as production and I assume you have copied mds_restore script along with gtar and gzip to the same directory as backup file? 

Error when restoring the MDS from backup 

Oh, don't forget that you will need 4x as much space compare to backup itself in the partition where you saved your backup file. Normally I would put backup file somewhere in /var/log partition as it's usually the biggest. But make sure that your /var/log is 4x the size of the backup file itself as a bare minimum

[DanielS]

If you are wanting to run cron jobs, you should add the cron job via clish

add cron job <name> command <> ....