Sometimes, CDT will report that it could not verify that policy was installed and it will stop. When checking the gateway, policy is there and using CDT with -resume option will continue with deployment plan (for us, it usually means installing JHF). Sometimes, gateway ends up with Initial policy. In that case, pushing policy usually solves it (but not always). Log will sometimes show "/opt/CPsuite-R80.40/fw1/state/local/FW1/local.ifs: Too many levels of symbolic links", pushing policy from management solves that too.
When upgrading several gateways, if one/some fail, it means you have to wait for all of the others to finish before you can attempt resume. If your upgrade time window is short, that can be an issue and it may be required to continue with manual JHF install and/or upgrade of other cluster member.
There is -retry option which never worked for us (neither CDT 1.7 nor 1.8, did not try with 1.9).
[Expert@*********:0]# ./CentralDeploymentTool -retry -server=***.***.***.***
The Domain Management Server is incorrect.
Cannot find a running Central Deployment Tool process in Domain Management Server -server=***.***.***.***.
Make sure that Central Deployment Tool is still running and try again. Contact Check Point support if this problem persists.
[Expert@*********:0]# ps aux | grep Centra
admin 6064 0.0 0.0 2648 584 pts/6 S+ 07:00 0:00 grep --color=auto Centra
admin 17268 0.0 0.0 53564 13096 pts/4 Sl+ 06:09 0:01 ./CentralDeploymentTool -execute -candidates /home/admin/****.csv -deploymentplan /home/admin/
I don't know if it matters, but CDT was started with nohup, could that be a reason that -retry fails?
mdsenv ***.***.***.***
nohup ./CentralDeploymentTool -execute -candidates=/home/admin/*******.csv -deploymentplan=/home/admin/***********.xml -filter=/home/admin/************.txt -server=***.***.***.***
One important note/tip: if CDT fails and you decide to finish the upgrade of cluster manually, you have to check/fix HA version:
fw ctl get int fwha_version
If you get 9999, you have to set it back to correct value for your CP version. Don't forget to change fwkern.conf as well (remove the line which setting fwha_version).
fw ctl setsync off ## stop the sync
cphaprob syncstat ## to confirm sync is off
fw ctl set int fwha_version 3421 ## use correct value for you version
fw ctl get int fwha_version ## to check if change was executed
fw ctl setsync start ## start the sync
cphaprob syncstat ## to confirm sync is on again