The most straightforward method is probably to treat it like replacing a failed cluster member twice. This method depends on keeping the same version. You will also keep the same cluster object.
- Make a management backup.
- Shut down old member 1 or otherwise disconnect it from the world.
- Bring up new member 1. Give it the same IPs and so on as the old member 1. IMPORTANT: Check Point branded boxes have udev rules causing them to have weird interface names. Your old member 1 will probably have names like "eth1". While your new box will have a few of those, anything in the card slot will have names like "eth1-03" which don't match the names of interfaces on open servers. Create an old-interface-name-to-new-interface-name mapping ahead of time.
- In the management, reestablish SIC with the new member 1. IMPORTANT: Edit the cluster interfaces to use the new interface names as needed.
- Push policy. Check the new member 1's cluster status for any issues.
- If none are found, force a failover to the new member 1 and test traffic.
- If traffic is good and you want to keep going, shut down old member 2 or otherwise disconnect it from the world.
- Bring up new member 2. Give it the same IPs and so on as the old member 2. IMPORTANT: Remap interfaces as needed as above.
- In the management, reestablish SIC with the new member 2. IMPORTANT: Edit the cluster interfaces in the object as above.
- Push policy. Check the new member 2's cluster status for any issues.
- Test failover and traffic to taste.
Rollback is as simple as powering off any new members, powering on the old members (the old members will still have their local copy of the policy, so they should be able to pass traffic as soon as they finish booting), restoring the management backup, and pushing policy.
Again, if at all possible, a version upgrade should be separate from a hardware swap. Changing the cluster object during all this (for example, if you put the hardware model in the name) is possible, but will require an outage (you can't sync between two cluster members which don't belong to the same cluster).
If you don't actually NEED to use separate IP addresses, it would be much easier and safer to just set up the new members as members 3 and 4 in the cluster, then remove the two old members.
ethtool -p <interface> can help you confirm which interface name is which physical port on your old servers.