PBR Query - Check Point vs Cisco

[biskit]

My customer currently has a Cisco ASA.  They've purchased Check Point to replace it, but I'm having a PBR issue.  The issue takes a bit of explaining so please bear with me.

My knowledge of Cisco is virtually zero, so please excuse me if I sound like a novice when describing the ASA functionality 😂

I've very quickly butchered their network diagram to help illustrate what I'm getting at.  Bear in mind, all of their servers on the "inside" are on public address space, so all fully routable without any NAT happening anywhere.

There are two ISP lines.  The normal Default Route is ISP 1.  All of the white networks in the diagram route this way.  PBR is configured to that some subnets route out of ISP 2.  All of the black networks in the diagram follow PBR and route out of ISP 2. 

The problem I have is that the individual servers shown on the diagram all need to route out of ISP 1 for 90% of the time, and out of ISP 2 for 10% of the time.  Traffic to www.website-1.com hits the F5, the F5 sends it on to a member server on either VLAN 107 or VLAN 113 in the diagram, so I've got /32 PBR rules in place for those specific server IP's to route back via ISP 1, while the rest of the subnet they live on hits the /24 PBR and goes out of ISP 2.

All normal and good.

Except....  

These websites use external payment gateways which need to communicate directly with one server.  Let's just say it is Node 1 on VLAN 107.  During a transaction the payment gateway initiates a new connection in to Node 1 directly, so it doesn't hit the VIP on the F5, it hits the Node 1 IP and comes in via ISP 2.  Node 1 then needs to talk back, but the /32 PBR sends the reply back out of ISP 1, and obviously fails.  All traffic is HTTPS so I can't even base another PBR rule on a different port.

I've queried how this currently works on the ASA and I've been pointed at ASA Policy Based Routing, and in particular the bit at the bottom of Pg 3 that says

PBR Policies Not Applied for Output Route Look-up
Policy Based Routing is an ingress-only feature; that is, it is applied only to the first packet of a new incoming
connection, at which time the egress interface for the forward leg of the connection is selected. 

(remember the subnet has PBR on the Cisco to route out of ISP 2)  The guy explained it to me that when a website request comes in via the F5 on ISP 1, the ASA remembers this and always routes traffic for that connection back out of ISP 1, so effectively ignoring the PBR that would sent it to ISP 2.  When a payment gateway connection comes in via ISP 2, traffic will return that way.

This seems to be where the Cisco and Check Point behaviour differs - and herein lies my question and plea for help 🙂

PBR on the Check Point seems to be absolutely strict.  I.e it doesn't care which interface traffic arrives from, it simply follows its PBR config, so return traffic seen from Node-1 will match the /32 PBR rule and always always always send it to ISP 1.  This is enough to stop the payment gateway stuff working as that traffic comes in via ISP 2 and needs to return via ISP 2 to avoid asynchronous routing.  We never know the destination IP's of the payment gateway (Paypal, Stripe, etc etc.) so as I see it I'm a bit stuck.

I presume that even if I NAT the payment gateway traffic on the way in behind the IP of eth2, it still won't work because CP will very strictly match my source IP of Node-1 to PBR and still shove it out of eth1?

What can I do to overcome this?  I need the Check Point to do what the ASA does and respect the interface a connection arrives from.

All suggestions welcome!  Thanks in advance.

Matt

[Wolfgang]

Matt,

Maybee you should have a look at ISP redundancy. This is the feature if you have more then one ISP line.

The default behaviour of ISP redundancy in LoadSharing is to route back incoming traffic the same way and interface it‘s coming to your servers. Meaning if traffic from external to your Webserver A is coming in over ISP 1, the returning traffic is routed the same way.

have a look at the configuration guide: HOW TO CONFIGURE ISP REDUNDANCY IN NGX R65 - R77.30 VERSIONS

“Incoming connections also benefit from the Load Sharing method. In the case of returned packets for a connection initiated outbound, the traffic uses the same ISP link that is used to initiate the connection.“

I‘m not sure if this is a solution for you, because I think ISP redundancy is not supported with PolicyBasedRouting together. But maybee in the newer release R80.20 R80.30.

I checked and there are bad news, from sk100500 it‘s not supported together.

Wolfgang

[biskit]

Hi @Wolfgang, 

Thanks for your thought.  I'll have a chat to the customer about this option but I suspect it isn't going to work in this instance.  They very much want the new firewall to slot straight in to place and replace the existing ASA, and I think without a lot of other networking and routing work we're not going to be able to ignore the PBR yet 😞

Matt

[Wolfgang]

Matt,

I understand your customer. But it‘s hard to replace another vendor with the same exact features.

How about the asynchronous routing. I know, it‘s not the best way, but what‘s the problem with returning packets,coming in on ISP2 and going out through ISP1. Where and why are these packets dropped. As I understand your configuration the payment gateway should be reachable via both ISPs ?

Wolfgang

[biskit]

I'm not sure where it was dropping.  Unfortunately at the time we'd discovered this last issue we had already gone over our maintenance window so we had to roll back (to the Cisco) without me having any time to troubleshoot.  I'm going to try and build a lab to replicate the issue...

[vshoinbayev]

Hi Matt,

did you found out is the checkpoint differs from the cisco in this point?