Migrating from one ISP to another on remote GW

[Ty_King]

I'm trying to best avoid the "cutting off the limb your standing on" scenario with this one, and I apologize if this topic has been covered in detail in any other resource.  I tried Checkmates, official docs, and a call to support, but was left wanting.

I've got a remote gateway (3200, 77.30, centrally managed) connected to a single ISP whose performance has been seriously terrible.  We brought in another provider and got them to terminate on an available interface (eth4).  I've verified I can ping from the gateway's new interface to the new ISP's gateway.  The old ISP's public address (/30 on eth5) is the IP the gateway has SIC registered as well as the VPN termination point for the site-to-site community that connects all our remote branches.

So now I have two ISP paths to this gateway.  The issue I'm facing is how to safely migrate to the new ISP without physically going onsite to do the work.  I've pushed a policy to the gateway that allows any and all traffic from the Mgmt server to the new public IP assigned on eth4.  But of course I can't reach the new IP because the default routes on the GW push everything to the current ISP.

Something tells me I'll have to change the static default route in Gaia to point to the new ISP gateway and hope I can reach it after the fact.  I realize I'll need to then update the SIC address for the GW to the new public IP and pray to the network gods that it works.  Also, failing that, will I be able to access the Gaia page on the GW at the new address to undo the routing change if it doesn't work?

I'll be setting these two ISPs up in redundancy mode on the GW if I can get this remote reconfig to work, but that's out of the scope of this question.

I doubt I'm the first to do this, so any input on previous success or failure with your steps taken would be greatly appreciated.  I'd lab this if I had the spare gear.

***EDIT:

Hristo's comment made me questions something:  If a different interface from the mgmt server can manage the GW through secondary IP, can the IP defined in a GW object be changed without consequence?  I know changing names is complicated.  But is changing just the IP in the object a problem for SIC and certificates used for VPN and pushing policy?

I tested a theory last night with a user on site.  I had a priority 2 route defined in the static default route on the remote GW.  I then had the user unplug the current ISP connection from the GW.  After a few seconds I was able to SSH into the GW once the primary route aged out of the table.

If changing the IP of the GW object is allowed, then all I have to do at that point is update the IP in the mgmt server, verify SIC, and push policy.  Or at least it seems that easy in my head.

Once I've got full mgmt control of the GW back I can update the default route selection to put the new ISP as priority 1 and the old as 2.  Then I have the onsite person plug the old ISP connection back in and move on with my ISP redundancy configuration.

[HristoGrigorov]

What about if you add second interface on the management server and a static route on the GW that will forward traffic to this address through the second ISP ? That way you will have management server reach GW on both ISPs.

[Ty_King]

That's an interesting approach.  However NAT is going to be a problem with the traffic originating from the mgmt server.  I already have one external assigned specifically for the mgmt server traffic and we don't have any to spare for another NAT path.

As long as I'm understanding what you're suggesting, I think that keeps me from using this solution.

Thinking about your suggestion though made me question something else that I may have not considered.  I'll post my additional question in the main thread as a amendment to my original post.

[HristoGrigorov]

Not sure, but I think you will have to re-set SIC and may be generate new license key for the gateway. It depends how VPNs care configured but they will most likely break because you will change external IP and peers will need to be reconfigured with the new one? 

[Ty_King]

I've been slacking off and failing to update this question.  Follow up below with what I ended up doing.

The migration ended up being a little too dicey to go about it remotely after talking with CheckPoint support.  I went onsite to manage the cutover so I could have local access to the gateway.  Things to keep in mind when performing this:

   1)  No need to get new licenses or even regenerate/reset SIC.  SIC only needs to be verified/tested after the migration.

   2)  If at all possible, make it easy on yourself and set up a secondary port for the new ISP connection on the gateway so the original connection can stay live until after migration.  This provides easy fall-back if something goes wrong.

   3) Create a temporary rule to allow explicit, full access (any service) from the central management server to the new public IP being migrated to and be sure to push this policy to gateways before beginning the cutover.

   4) After the new ISP is connected to the gateway, change the IP of the gateway object in the central manager to the new public IP and test SIC.  It should get a valid response from the new IP.

   5) Pull a topology update for the gateway in the central manager object and be sure everything looks correct for the new public interface.

   6) Make sure that VPN IPSec (if used) settings on the gateway are set to use either use primary interface, or manually set to use the new public interface.

   7) Be sure to update routes in Gaia on the local gateway for the new ISP traffic.

   😎 Make any other needed changed to policy rules and push policy to all gateways.  Ensure everything is working as expected, then disconnect the old ISP and clean up unnecessary rules, interfaces, routes, etc.

Something I learned in this process is that there are some limitations to how ISP failover can work.  I wanted to keep both ISPs in service and use the failover feature of the gateway.  The issue is that the next hop gateway must be pingable and my original service provided did no allow their gateway address to respond to ICMP.  Everything I could find would not allow the CheckPoint to work any other way to determine availability of that route.

Other than that, no other real surprises other than missing a couple things like the IPSec VPN interface choice that caused some headaches.  Hopefully somebody else can get a little out of my experience here.  Unfortunately, I didn't end up doing this remotely as I had intended.  Working this up in a lab would be a much more comfortable place to test this procedure out and see if could all be done successfully with no local access to the gateway.

[Maarten_Sjouw]

Sorry I missed your April posting, as a MSP we service a lot of customers that have a worldwide presence. And yeah they also have changes in ISP's and we do a lot of these type of changes. Going onsite is not really possible in most cases. 

As soon as you connect the new ISP line and assign the new ISP's IP to the new interface do the same in the Dashboard/Smartconsole add the interface with its IP as an external interface..

Now you can at least make sure you can access the device on the new IP from the management site. Now you know the new connection is working.

You can now change the main object IP of the Gateway and add a stic route for the management IP to the new ISP.

Next step can be to change the VPN stuff change the link selection page to use the new interface and you could add static routes to the new ISP for the VPN Peers.

Last step would be to change the default route to the new ISP.

On the matter of ISP redundancy, did you try to add a route for 8.8.8.8 to the old IPS' gateway and set a ping to 8.8.8.8 as the test for this ISP?