Seeing Odd Behavior in r80.20

[Terri_Hawkins]

We have recently upgraded our Management and Log servers and primary gateways to r80.20 and have been seeing odd behavior since then, just wondering if anyone else is.

1. We have had two instances where we have a firewall rule allowing a server to go out and get updates. The rule has been working for years. In the last week traffic has attempted to go out as normal but for a period of many hours the firewall could not match the normal rule and dropped it on the clean up rule. Both times this occurred in the evening when no one would have been attempting any changes or publishes of the rule. It just stops and drops, same source, same destination, same port.

2. We had another instance where traffic was being dropped and the log shows it on one rule number, but the rule number does not match the rule. If you click the rule number it goes to the correct rule. At the time the traffic was being dropped we were doing a test, no one was publishing anything that would have changed the rule number, it was just wrong. This was actually us trying to ping some devices inside our network from the gateway and we can suddenly no longer do that. It is dropping as unknown internal traffic.

3. We have two instances where people are trying to get to websites using https and their traffic is completely bypassing our access policy and going out a different port (so their websites never open). I can find no other object in the firewall for their workstations or the websites. All their other traffic works fine.

There have been other odd things that I just sort of wrote off when they occurred which I wish I documented now, but in general, r80.20 just seems quirky, which is not good for a firewall. Is anyone else experiencing this type of behavior? I am getting ready to document everything I can and send it to support, but I'm not sure they can help with this type of inconsistent stuff.

Any input is greatly appreciated.

[PhoneBoy]

This is precisely the sort of thing a TAC ticket needs to be opened for.

Can you provide some context about the traffic is and what rules should be matching them?

[Terri_Hawkins]

I am in a training class today and tomorrow which Bill Dunbar is teaching and he also recommends opening a TAC ticket. I need to gather all the documentation and will do so.

But, to answer you question and get your input....

We have a server object defined in our policy. It has a hide nat on it. A rule allows it to go to specific domains via https. The server initiates the traffic and it is a pretty constant connection. The rule has been in place for a few years without incident. Several times now (we are up to 3) the firewall stops accepting the traffic on the rule and drops it on the clean up rule. The source, destination, and service are identical. There is no "first packet wasn't syn" message, it just drops it as though it missed the rule allowing it. The first time it happened it started dropping at 8PM, when no one who makes firewall changes was in the office, then picked back up accepting traffic again at 9AM, and we made no changes to fix it.

We have now identified a second rule this is happening with. Similarities are both servers have a hide nat, both are allowed to go to specific domain objects, and both are allowed on https. This second one also started dropping for several hours then picked back up without us making any modification.

According to our updates page we have the latest patches and hotfixes. We have just recently implemented https inspection, but this particular traffic is bypassed for that, and I do see the bypass in the logs.

Any thoughts are appreciated.

[PhoneBoy]

You say you are using a "Server" object.
Are you referring to a Host object or something else?

You mention a Domain object as well.
Is this an FQDN object?

Given that Domain objects are updated as a result of DNS queries by the gateway, it doesn't take a change in policy to affect a change, it only takes a change in DNS, be it the result of the actual DNS lookup, or the gateways ability to make the specific DNS query.

[Terri_Hawkins]

Yes, I am sorry, it is a host object and FQDN. The gateway should have been able to resolve those requests, they have access to DNS servers from our ISP for resolution.  I will have to look and see if Century Link posts downtimes.  On Tuesday the traffic was being dropped from both rules at the same time, so that is a possibility, I will check it. 

Thanks! Gives me something to look for.