could someone advice me how to determine the value for "ipsec.replay_counter_window_size"

[Florian_Schneid]

Hi,
could someone direct me how I can adjust the setting to avoid VPN Tunnel termination due to "possible replay attack".

I do have the issue described in sk94984. The issue exists only for one Tunnel. The issue is gone when I disable the replay check. Now I wanted to turn it back on and adjust the window size. In the SK they only say to adjust it to the relevant value.

In the logs I do have the message:

Warning: possible replay attack. Sequence Number 1490945 (Expected 1491179)

Currently I used 1200 as window size but the tunnel is still being terminated.

How can I determine / calculate the value? Seem that it isn’t just 1491179-1490945

Thanks

R80.40 T94

[G_W_Albrecht]

Better use the information found in sk94984: VPN traffic is dropped with "Encryption failure: Warning: possible replay attack" log and involve TAC if this does not help. 

[Florian_Schneid]

Hi Günter,

as mentioned above I followed the SK94984. But i didn't want to have the reply check disabled in general. So i decided to do the route descibed in the additional part of the SK and adjust the window size. I did adjust it to 1200 the log shows it triggered even it was only 234 as from the logs.

regards

Florian

[G_W_Albrecht]

So i would suggest to involve TAC !

[isazonov · Accepted solution]

Response from TAC:

1. Connect with SmartConsole to the Security Management Server / Domain Management Server.

2. Close all SmartConsole windows (SmartDashboard, SmartView Tracker, SmartView Monitor, etc.).

   Verify by running the "cpstat mg" command on Security Management Server / in the context of each Domain Management Server.

3. Connect with GuiDBedit Tool to the Security Management Server / Domain Management Server.

4. In the upper left pane, go to Table - Network Objects - network_objects.

5. In the upper right pane, select the relevant Security Gateway / Cluster object.

6. Press CTRL+F (or go to Search menu - Find) - paste ipsec.replay_counter_window_size - click on Find Next.

7. In the lower pane, right-click on the ipsec.replay_counter_window_size - select Edit... - delete the default value of 64 - enter the relevant value - click on OK.

8. Save the changes: go to File menu - click on Save All.

9. Close the GuiDBedit Tool.

10. Connect with SmartConsole to the Security Management Server / Domain Management Server.

11. Install the policy onto the relevant Security Gateway / Cluster object.

Keep in mind that the default value is 64, and there is no desired value - you will need to lower or higher it until it reaches the correct value where this issue does not re-appear.