AWS to Onsite VPN Failing to work

Secret-goblin-5

We have a HA pair of firewalls onsite, our network uses static routes as it's pretty simple.

Some of our services are in AWS using a VPN tunnel, when this tunnel goes down we tend to get about 5 minutes of downtime, which is unacceptable.

Our support calls have come to the conclusion we should be using VTI tunnels with BGP as best practice, and it has my task to implement this.

The issue I have is that following instructions https://support.checkpoint.com/results/sk/sk108958 and modifying them to work with HA using post https://community.checkpoint.com/t5/Security-Gateways/VTI-interface-with-Cluster-XL/td-p/129100 (or a similar post, I forgot to bookmark the one I read) I can get the tunnel up, but cannot get BGP to work.

Please help.

How I built the VPN & BGP

#GATEWAY 1
add vpn tunnel 1 type numbered local x.x.x.1 remote x.x.x.17 peer AWS_Tunnel1
set interface vpnt1 state on
set interface vpnt1 mtu 1399
save config
#GATEWAY 2
add vpn tunnel 1 type numbered local x.x.x.2 remote x.x.x.17 peer AWS_Tunnel1
set interface vpnt1 state on
set interface vpnt1 mtu 1399
save config
#SMART CONSOLE
"Get Interfaces" -> "With Topology"
Then edit "vpnt1" to have a VIP of x.x.x.18

#3: Define Network Objects
New Interoperable Device called "AWS_Tunnel1"
Topology is "Empty_Group"

NOTE, "Network Management", in the "VPN Domain" section; I have left this as is for our other, already working VPN tunnels, rather than change to "Empty_Group"

Next I built the Star Gateway community following the instructions (nothing special here, just following the instructions)

OK, so far so good. VPN tunnel will be up at this point.

For the firewall rules I have tried two rules

Neither rule seems to be grabbing the traffic.
An FW Monitor appears to show pings going out to the internet rather than down the tunnel.

I have concluded the issue is with BGP rather than the VPN configuration itself.

BGP has a Router ID of my public IP address
BGP has a Local Autonomous System Number of 65000
BGP has a Peer group of 65256 with a local address of x.x.x.18 and a peer of x.x.x.17

BGP has 0 messages sent and 0 messages received.

"Inbound Route Filters" has a BGP policy for 65256
"Route Redistribution" is advertising all static routes to BGP 65256

Where did I go wrong, why isn't this working?

Also worth nothing, some instructions said to use an Unnumbered VTI, when I did this BGP would never leave the "Idle" state and everything still failed to work.

Nir_Shamir

Hi,

first you need to edit your FW object under the VPN Community and change the encryption domain to empty (see attached). this will only affect this specific Community and not others you have.

after that , can you see in your logs BGP traffic encrypt / decrypt ?

Secret-goblin-5

Thanks, missed this, it was not in any of the instructions I have found.

Also missed that BGP traffic was being blocked. I have enabled a rule to allow BGP traffic through but still got problems.

This hasn't fixed our BGP issues, but has made them better.

the_rock

For what its worth, though I can only speak from my own experience with Azure and AWS, I always found that using empty enc. domains on both sides and UNNUMBERED vtis for BGP is what makes all this work well. Then, to route traffic, you can use those VTIs as DG when you create a route in web UI. By the way, also important, most people may "freak out" if they realize that unnumbered VTI will look EXACTLY like the actual external interface in topology, but thats 100% normal and you can even give it exact same VIP as well, no issues there. Just make sure to do "get interfaces WITHOUT topology. Also, no need to have anti spoofing configured on those.

See if my post below helps.

https://community.checkpoint.com/t5/Security-Gateways/Route-based-VPN-tunnel-to-Azure/m-p/206179/emc...

Andy

Secret-goblin-5

Rebuilt the tunnels using your recommended configuration.

It is working, traffic is flowing (or appears to be) but BGP is still not working.

BGP Config commands listed below.

Gateway1> show bgp

Routing Process BGP
    State is on
    Local Autonomous System is 65000
    Default Weight is 0
    BGP Route Rank is 170
    ECMP is off
    IGP Synchronization is off
Gateway1> show bgp errors

PeerID            Last State        Last Event        Last Error
x.x.x.17    Idle              Start             None
Gateway1> show bgp summary

Routing Process BGP
    State is on
    Local Autonomous System is 65000
    Default Weight is 0
    BGP Route Rank is 170
    ECMP is off
    IGP Synchronization is off
Gateway1> show bgp stats

Peer: x.x.x.17
                      Received            Sent
    Opens                    0               1
    Notifications            0               0
    Updates                  0               0
    Keepalives              98              98
    Route Refresh            0               0

Gateway1> show bgp peers

Flags: R - Peer restarted, W - Waiting for End-Of-RIB from Peer

PeerID           AS           Routes  ActRts  State             InUpds  OutUpds  Uptime
x.x.x.17   65256        0       0       Active            0       0        00:00:00
Gateway1> ping x.x.x.17
PING x.x.x.17 (x.x.x.17) 56(84) bytes of data.
64 bytes from x.x.x.17: icmp_seq=1 ttl=254 time=14.9 ms
64 bytes from x.x.x.17: icmp_seq=2 ttl=254 time=14.0 ms
64 bytes from x.x.x.17: icmp_seq=3 ttl=254 time=14.1 ms
64 bytes from x.x.x.17: icmp_seq=4 ttl=254 time=14.1 ms
64 bytes from x.x.x.17: icmp_seq=5 ttl=254 time=17.1 ms
64 bytes from x.x.x.17: icmp_seq=6 ttl=254 time=14.0 ms
^C
--- x.x.x.17 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5002ms
rtt min/avg/max/mdev = 14.055/14.750/17.121/1.100 ms

Gateway1> ping x.x.x.18
PING x.x.x.18 (x.x.x.18) 56(84) bytes of data.
^C
--- x.x.x.18 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms


Gateway1> show bgp peer x.x.x.17 detailed

----- Peer x.x.x.17 -----
State                         Active
Peer Type                     eBGP Peer
Remote AS                     65256
Local AS                      65000
Peer Capabilities             n/a
Our Capabilities              IPv4 Unicast,4-Byte AS Extension
Authentication                None
Multihop                      Off
Reachability Detection        Off
Graceful Restart              Off
Received
    IPv4 Routes               0 (0 active)
    IPv6 Routes               0 (0 active)
Sent
    IPv4 Routes               0
    IPv6 Routes               0
Gateway1>

the_rock

You are using unnumbered VTIs now? If so, did you make sure you have routes pointing to remote subnet(s) using those unnumbered VTI interfaces?

If yes to all, then I would do basic zdebug and see why its fialing.

Andy

Secret-goblin-5

You are using unnumbered VTIs now?

Yes

If so, did you make sure you have routes pointing to remote subnet(s) using those unnumbered VTI interfaces?

I have a destination route to x.x.x.17 only (The AWS side of the internal tunnel IPs)
Do I need more than 1 destination route?

If yes to all, then I would do basic zdebug and see why its fialing.

OK, will try this and see what I get.

the_rock

Do the route to the whole subnet on the other side using related VTI.

Andy

Secret-goblin-5

- 169.x.x.17/32 as a static route to vpnt1
- 192.168.177.0/24 as static route to vpnt1 (AWS test network)
BGP shows "Idle"

- 169.x.x.16/30 as static route to vpnt1
- 192.168.177.0/24 as static route to vpnt1 (AWS test network)
BGP still show "Idle"

What am I missing?

the_rock

So tunnel itself is fine and all works, except bgp? If so, did you try simple zdebug, just filter for 179?

fw ctl zdebug + drop | grep "179"

Andy

Nir_Shamir

can you run "show configuration bgp" on your GW and paste it here ?

"idle" means there is no BGP activity at all.

also check "/var/log/routed_messages" for any BGP error messages.

Are you a member of CheckMates?

AWS to Onsite VPN Failing to work