Backups & Snapshots & LVM_Manager on Maestro Security Groups

[Jon-Telfer]

Hey all
We have noticed issues on a Maestro Configuration, specifically the underlying Security Groups and their Backups - The long and the short of it is that we are Unable to take Backups of a Maestro Security Group.

• In this set up there is 1 Site, 2 Maestro Orchestrators (MHO-140s), 5 Appliances (6200's), set up in 2 Security Groups (3 in SG1, 2 in SG2) All R81.10, JHF 94

Specifically the issue reared its head in the following way on SG1 with the 3 Appliances.

We noted that Backups and Snapshots were stopping with the error “backup: another backup is currently in execution” on Clish.

We followed sk100403, and deleted the locked files on all members. 

We noted at this time that the files were present (and thus could be deleted) on Member 1-02, but were not present on 1-01, and 1-03

So, thinking well that isn't right, but we have made them all the same now, so we should be golden, we attempted to take a backup using Gaia (which is how we first noted the problem of backups)

We get a new message: Backup Failed - failed to remove "lv_current_snap"

Thus I go to lvm_manager and run it on each of the members - and it shows substantially different things per member:

On 1-02 there is an extra partition called lv_Internal (in addition to lv_Internal1) and 0Gb of unallocated space.

On 1-01, and 1-03 there is no such partition (there is a lv_Internal1) and 13Gb of unallocated space

Now I am happy to play around with backups and delete space of the taken backups, but messing with lvm always makes me a little wary and especially as this site is NOT taking backups correctly.

Any advice would be appreciated.

(Output of LVM Manager)

[Expert@CP_6200_FW-ch01-01:0]# 

LVM overview
============
                         Size(GB)   Used(GB)   Configurable    Description
    hwdiag               1          1          no              Snapshot volume
    lv_Internal1         14         14         no              Snapshot volume
    lv_current           32         8          yes             Check Point OS and products
    lv_current_snap      15         15         no              Snapshot volume
    lv_fcd_GAIA          8          8          no              Factory defaults volume
    lv_log               72         20         yes             Logs volume
    upgrade reserved     36         N/A        no              Reserved space for version upgrade
    swap                 32         N/A        no              Swap memory volume
    unallocated space    13         N/A        no              Unused space
    -------              ----
    total                223        N/A        no              Total size



================================

[Expert@CP_6200_FW-ch01-02:0]# 

LVM overview
============
                         Size(GB)   Used(GB)   Configurable    Description
    hwdiag               1          1          no              Snapshot volume
    lv_Internal          12         12         no              Snapshot volume
    lv_Internal1         14         14         no              Snapshot volume
    lv_current           32         9          yes             Check Point OS and products
    lv_current_snap      16         16         no              Snapshot volume
    lv_fcd_GAIA          8          8          no              Factory defaults volume
    lv_log               72         24         yes             Logs volume
    upgrade reserved     36         N/A        no              Reserved space for version upgrade
    swap                 32         N/A        no              Swap memory volume
    unallocated space    0          N/A        no              Unused space
    -------              ----
    total                223        N/A        no              Total size

press ENTER to continue.


==================================

[Expert@CP_6200_FW-ch01-03:0]#

LVM overview
============
                         Size(GB)   Used(GB)   Configurable    Description
    hwdiag               1          1          no              Snapshot volume
    lv_Internal1         14         14         no              Snapshot volume
    lv_current           32         9          yes             Check Point OS and products
    lv_current_snap      15         15         no              Snapshot volume
    lv_fcd_GAIA          8          8          no              Factory defaults volume
    lv_log               72         20         yes             Logs volume
    upgrade reserved     36         N/A        no              Reserved space for version upgrade
    swap                 32         N/A        no              Swap memory volume
    unallocated space    13         N/A        no              Unused space
    -------              ----
    total                223        N/A        no              Total size

press ENTER to continue.

Any Help Appreciated! 

[emmap]

The non-standard 'snapshot volumes' (lv_Internal etc) should be reflected in a 'show snapshots' in gclish. If they're not, probably best to raise a TAC case rather than trying to mess with that directly. Alternatively you can always FCD them back to R81.10 and auto-clone them back into the security group, just be aware that we can't investigate anything after an FCD. 

This may also be related to this fix that is included in JHF t110: "Gaia backup may fail, leaving a temporary partition behind. Any new attempt to create a new backup returns an error." So a JHF update might be good too.