Upgrade log files were compared between a successful upgrade and a failure - there are very little differences with the exception of the last dozen or so lines - a failed upgrade simply stops - and reloads - no upgrade. A successful upgrade moves through to the validation - reloads - and upgrades.
FAILURE
2021-Apr-27-21:50:01: Executing command: '/bin/sync'
2021-Apr-27-21:50:01: Executing command: '/bin/umount /mnt/inactive'
umount: can't unmount /mnt/inactive: Device or resource busy
2021-Apr-27-21:50:01: Executing command: '/bin/rm -rf /storage/revert_image.img'
2021-Apr-27-21:50:02: Image upgrade process completed successfully.
2021-Apr-27-21:50:02: Executing command: '/pfrm2.0/bin/sqlcmd update reboot set rebootTime = '60''
2021-Apr-27-21:50:02: Executing command: '/pfrm2.0/bin/sqlcmd update reboot set reboot = '1''
2021-Apr-27-21:50:02: Executing command: '/bin/rm -rf /fwtmp/upgradeRevertInProgress'
2021-Apr-27-21:50:02: Exiting with error code 0.
END
SUCCESSFUL
2021-Apr-27-01:35:45: Executing command: '/bin/sync'
2021-Apr-27-01:35:45: Executing command: '/bin/umount /mnt/inactive'
umount: can't unmount /mnt/inactive: Device or resource busy
2021-Apr-27-01:35:45: Executing command: '/bin/rm -rf /storage/revert_image.img'
2021-Apr-27-01:35:45: Image upgrade process completed successfully.
2021-Apr-27-01:35:45: Executing command: '/pfrm2.0/bin/sqlcmd update reboot set rebootTime = '60''
2021-Apr-27-01:35:46: Executing command: '/pfrm2.0/bin/sqlcmd update reboot set reboot = '1''
2021-Apr-27-01:35:46: Executing command: '/bin/rm -rf /fwtmp/upgradeRevertInProgress'
2021-Apr-27-01:35:46: Exiting with error code 0.
2021-Apr-27-01:40:44 - upgrade_sanity_checks_interval is missing or wrong in /opt/fw1/conf/upgrade_sanity_checks_props.sh
2021-Apr-27-01:40:44 - upgrade_sanity_checks_period is missing missing or wrong in /opt/fw1/conf/upgrade_sanity_checks_props.sh
2021-Apr-27-01:40:44 - upgradeSanityChecks is missing in /opt/fw1/conf/upgrade_sanity_checks_props.sh
2021-Apr-27-01:40:44 - upgrade_sanity_checks_interval=2
2021-Apr-27-01:40:44 - upgrade_sanity_checks_period=30
2021-Apr-27-01:40:44 - upgradeSanityChecks=mgmt_connectivity_and_policy_revert_verification
2021-Apr-27-01:40:44 - isPreUpgradePolicyNonDefault=true (relevant after upgrade)
2021-Apr-27-01:40:44 - preUpgradeConnectivityOK=true
2021-Apr-27-01:40:44 - isPreUpgradePolicyNameLocal=false
2021-Apr-27-01:40:44 - Pre-upgrade connectivity to management was OK. Now testing after upgrade
2021-Apr-27-01:40:44 - Running Connectivity Validations.
2021-Apr-27-01:40:45 - Upgrade validation. Policy installed successfully.
2021-Apr-27-01:40:45 - Upgrade validation is OK
2021-Apr-27-01:40:45: Disabling watchdog.
2021-Apr-27-01:40:45: Post-boot completed successfully.