My customer currently has a Cisco ASA. They've purchased Check Point to replace it, but I'm having a PBR issue. The issue takes a bit of explaining so please bear with me.
My knowledge of Cisco is virtually zero, so please excuse me if I sound like a novice when describing the ASA functionality 😂
I've very quickly butchered their network diagram to help illustrate what I'm getting at. Bear in mind, all of their servers on the "inside" are on public address space, so all fully routable without any NAT happening anywhere.
There are two ISP lines. The normal Default Route is ISP 1. All of the white networks in the diagram route this way. PBR is configured to that some subnets route out of ISP 2. All of the black networks in the diagram follow PBR and route out of ISP 2.
The problem I have is that the individual servers shown on the diagram all need to route out of ISP 1 for 90% of the time, and out of ISP 2 for 10% of the time. Traffic to www.website-1.com hits the F5, the F5 sends it on to a member server on either VLAN 107 or VLAN 113 in the diagram, so I've got /32 PBR rules in place for those specific server IP's to route back via ISP 1, while the rest of the subnet they live on hits the /24 PBR and goes out of ISP 2.
All normal and good.
Except....
These websites use external payment gateways which need to communicate directly with one server. Let's just say it is Node 1 on VLAN 107. During a transaction the payment gateway initiates a new connection in to Node 1 directly, so it doesn't hit the VIP on the F5, it hits the Node 1 IP and comes in via ISP 2. Node 1 then needs to talk back, but the /32 PBR sends the reply back out of ISP 1, and obviously fails. All traffic is HTTPS so I can't even base another PBR rule on a different port.
I've queried how this currently works on the ASA and I've been pointed at ASA Policy Based Routing, and in particular the bit at the bottom of Pg 3 that says
PBR Policies Not Applied for Output Route Look-up
Policy Based Routing is an ingress-only feature; that is, it is applied only to the first packet of a new incoming
connection, at which time the egress interface for the forward leg of the connection is selected.
(remember the subnet has PBR on the Cisco to route out of ISP 2) The guy explained it to me that when a website request comes in via the F5 on ISP 1, the ASA remembers this and always routes traffic for that connection back out of ISP 1, so effectively ignoring the PBR that would sent it to ISP 2. When a payment gateway connection comes in via ISP 2, traffic will return that way.
This seems to be where the Cisco and Check Point behaviour differs - and herein lies my question and plea for help 🙂
PBR on the Check Point seems to be absolutely strict. I.e it doesn't care which interface traffic arrives from, it simply follows its PBR config, so return traffic seen from Node-1 will match the /32 PBR rule and always always always send it to ISP 1. This is enough to stop the payment gateway stuff working as that traffic comes in via ISP 2 and needs to return via ISP 2 to avoid asynchronous routing. We never know the destination IP's of the payment gateway (Paypal, Stripe, etc etc.) so as I see it I'm a bit stuck.
I presume that even if I NAT the payment gateway traffic on the way in behind the IP of eth2, it still won't work because CP will very strictly match my source IP of Node-1 to PBR and still shove it out of eth1?
What can I do to overcome this? I need the Check Point to do what the ASA does and respect the interface a connection arrives from.
All suggestions welcome! Thanks in advance.
Matt