We use OSPF for this exact design. A high-level implementation would be:
1. Have your firewalls advertise a default route and have the MPLS routers advertise your internal networks for each office, be sure to change the metric of the default route advertisement so that one firewall doesn't take the internet for both offices
2. configure a VPN between the firewalls, route-based or domain based, and either define routing for them or define their respective encryption domains
If the MPLS fails the default route from the local firewall will be the only route in the office so traffic will go to the firewall and hit your vpn config
If the internet fails over the default route from the MPLS will come through and all office traffic will take the MPLS until the internet is restored