fetch changed policy from management to gateway

[Wolfgang]

Hello folks,

after a misconfigured NAT we lost the abillity to push policy from Management to gateway.

We could revert the misconfiguration but now we can't install policy to the gateway, because there packets are NATed the wrong way.  Access to the gateway is possible via another line and I did "fw fetch .....", but without success.

Following sk119332 I learned you can't fetch changed policies. This is new to me, I thought fw fetch loads the newest policy from management. But it fetched only the last installed policy of the gateway.

Question, are there any possibilities to fetch the changed policy from the management ?

We can't do an fw unloadlocal, we lost connectivity if we do this. Gateway is located in a remote datacenter.

Too we have connectivity to the smartcenter, but packets originating from management to the gateway are send the wrong way because of the problematic NAT. A policy push from management to gateway is not possible.

Any ideas are very welcome,

Thanks, Wolfgang

[JozkoMrkvicka]

Does backup include also policy ? If yes, just revert the last backup and all should be fine 🙂

I guess that you don't have a snapshot to be reverted.

You mentioned that you cannot do "fw unloadlocal" ? Why not ?  Are you using cluster configuration with 2 members in Active - Standby mode ?
If this is the case, then simply put Standby member into a permanent down state (clusteXL_admin down -p), do "fw unloadlocal" on down member, push policy (do not forget to uncheck setting that policy can be pushed if 1 member is not reachable) and you have policy installed on Down member. Bring this Down node back to production by clusterXL_admin up, perform failover and repeat steps on the former active node. It might be that node which was down will not go into Standby state, but it is because the policy is not the same on both members. Failover should be possible anyway.

[HeikoAnkenbrand]

This is also an interesting solution to solve it by backup.

Nice!

[JozkoMrkvicka]

It was just an idea, as I never tested such a solution yet.

Can someone confirm that the backups include also last policy files and once a backup is restored, the policy from backup is installed?

It is not clearly stated within Best Practices - Backup on Gaia OS

EDIT:

According to the following articles, policy should be restored:

Policy is not restored after a restore from backup

Objects and policies are not restored from Gaia Backup, although the restore operation succeeded

[HeikoAnkenbrand]

Description "fw ctl uninstall"

1. Tells the operating system to stop passing packets to Firewall.
2. Unloads the current Security Policy.
3. Unloads the current Firewall Chain Modules.
4. Unloads the current Firewall Connection Modules
   
   

1) Install seriel cabel to configure the appliance remote or ILO for open server.

2) Run the "fw ctl uninstall"  command, the networks behind the Security Gateway become unprotected.

3) Configure routes via clish over the seriel console or KVM

4) Install new policy

5) Run "fw fetch" and fetch the new policy from management server.

[HeikoAnkenbrand]

For remote access via console or ILO I always use a laptop with serial console or network. You can access the laptop via WLAN or LTE and Teamviewer.

[Wolfgang]

Heiko,

serial console is a good idea, but not possible at the moment.

Datacenter is more then 500km far from us and a long way to drive to configure one cable 😞

Something learned again this week... check your changes twice or more before install.

And did backups/snapshots everytime of major changes.

Wolfgang

[Wolfgang]

Thanks for your recommendations.

We don’t have usable backups or snapshots available. I can’t do fw unloadlocal because I‘m connected over a second interface via a site 2 site vpn tunnel with another gateway. If I unload policy I loose the VPN.

We found a way to restore via snapshot.

1. Export of the smartcenter to our lab environment

2. Installed similar gateway hardware from our lab environment (i‘m happy having the same type available)

3. installed policy from recovered smartcenter to the gateway in our lab

4. Create a snapshot of the lab gateway

5. Transfered snapshot to the failing gateway and revert back to this.

6. Gateways reboots and everything was fine.

I‘m happy to not travelling around the country to get onsite to the appliance.

Thanks for help guys,

Wolfgang

[Vladimir]

@Wolfgang , please check the status of your licenses on the getaway recovered from the snapshot created on alternate hardware. It is likely that the MAC addresses of the target have changed to the same as the gateway you have used to create the snapshot.

This being said, I really like this as a last resort recovery method and, with minor modifications, will keep it in my toolbox.

Cheers,

Vladimir

[JozkoMrkvicka]

If the management server is not behind that firewall, then you can use "cprid_util" tool  and connect to the gateway over SIC.

This is why disaster recovery solutions are MUST HAVE in place:

1. Working console connection (if console server is behind FW, NEVER put console server behind the firewall which you want to connect). The same applies to LOM.

2. Scheduled daily backup OUTSIDE of the box (Storage, NAS). In case of management - migrate export, mds_backup

3. Deploy cluster environments (2 cluster members minimum).

4. If possible, rack members in different rooms/racks/buildings/countries.

5. Management traffic should go via External interface, or dedicated interface (not via VPN).

6. Proper monitoring including syslog server in place (monitoring of RAID, PSUs, FANs, ...).

7. Create local users.

8. Do snapshot before a major change.

9. Test revert and restore periodically.

10. Document every change you did to the system and let the change be approved by someone else (more brains, fewer problems). Document also all passwords used (SIC, S2S secrets, local user passwords).

10. Reboot box every half a year.