SolarWinds Orion

[RemoteUser]

Hi Mates!

The customer is experiencing intermittent ICMP DOWN  alert events reported by SolarWinds, i checked on the firewall site but i dont'see nothing realted, have any idea?
Orion polls ICMP every 3 minutes.

[Chris_Atkinson]

What address is being probed is there a VPN involved or is the gateway under load?

[RemoteUser]

Orion go out through a cluster, there is no VPN involved 

[the_rock]

Hey brother,

Are there any relevant logs in Smart Console you can see about this?

[RemoteUser]

the only thing that i see it is that:
dropped by fw_first_packet_state_checks Reason: ICMP reply does not match a previous request;

[the_rock]

https://support.checkpoint.com/results/sk/sk66443

https://community.checkpoint.com/t5/General-Topics/ICMP-reply-does-not-match-a-previous-request/td-p...

[RemoteUser]

yeah brother, i know about that, but the main purpose it's to understand why it happens

[the_rock]

My most educated guess is assymetric routing. Thats what I always experienced, since R55 days.

[the_rock]

Hey bro,

Just curious. when did this issue happen?

[RemoteUser]

one month ago brother

[the_rock]

Can you find any relevant logs around that time that could potentially help us figure out why this may have happened?

[Chris_Atkinson]

Any aggressive aging active log messages or cluster failover events that correspond?

[RemoteUser]

I'm sorry, I don't understand your question.

[the_rock]

Its aggressive aging protection, brother, check out below.

https://sc1.checkpoint.com/documents/R80.20/SmartConsole_OLH/EN/html_frameset.htm?topic=documents/R8...

[Shyy]

Happened to me once due to asymetric routing,
I'd suggest to check the revisions and look at the changes that were probably made around the time the issue started.

[the_rock]

Hey brother,

I did some more research on this and found some notes about this tool when I worked with one of our customers few years ago on some alerts and it turned out to be false positive. Can you check with the client if thats a possibility?

[genisis__]

Is this just polling a standard gateway or VSX? 

Only issue I've seen is when you attempt to ping the cluster IP and a real IP on the node.

[RemoteUser]

standard gw, what you mean by the real IP

[genisis__]

If you are ping the cluster IP and the real IP it likely won't work on the active gateway (See SK26874),  I specifically experienced this on VSX, so may not be relevant.

[the_rock]

But is it cluster or single gw?

[the_rock]

Hey brother,

Any news about this?

[RemoteUser]

hey brother,
seems to be Asymmetric routing

[the_rock]

Thats what we initially thought as well.

[genisis__]

Did you resolve it then?

[RemoteUser]

nope not yet, what kind of actions do you usually take to determine whether the issue is caused by asymmetric routing?

[the_rock]

Hey bro,

I would double check interface topology, if not sure, just set it per routing option, thats default anyway and recommended too. Now, obviously, goes without saying, dont make any changes if not sure, as it would break things.

[the_rock]

Maybe do ip r g command to relevant IP address. Example ip r g 8.8.8.8