Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
SG22
Explorer

DNS problems possibly due to firewall issue

Jump to solution

Hello everyone,

We have 3 DNS servers, one in my office and 2 in another location. Firewall is doing DHCP, in Firewall configuration DNS in mu location is primary and DNSes in another location are secondary and tertiary. 

Recently we started to experience problems with DNS. It takes forever to resolve queries. It takes a minute to open webpage, it takes 2-5 minutes to log in to workstation. Nslookup doesn't work. 

I temporarily changed DNS on my PC to google DNS, and it started to work. So I thought it is DNS problem. First I changed primary DNS on my workstation to be one from another location, but nothing changed. Then I changed Forwarders in DNS settings, cleared cache, but still same problem.

So I rebooted firewall and after firewall reboot, everything started to work without any issues for a couple of hours. After couple of hours same problem. So I rebooted firewall again, still just a temporary solution.

In other office we have same firewall, and there everything works without issues. DNS resolves queries. But even when I put their DNS as a primary on my workstation, or any workstation in my office, we have a problem.

Do you have an idea what can be issue? Please have in mind that I'm just junior without any firewall experience, so try to explain it to me like I'm an idiot.

0 Kudos
1 Solution

Accepted Solutions
the_rock
Authority
Authority

Yea...personally, it seems like firewall is behaving like it should. You know what I did @SG22 ...since we spoke about 25 mins ago, I logged into 2 customers' environments and did exact same commands with their internal DNS servers and output was literally the same, no drops related to port 53 and also curl_cli gave correct output as well.

Since you mentioned there was power outage about 2 weeks ago, I have a feeling that caused this problem, if it started happening right after. Now, tracing it to specific "culprit" might be a bit of a challenge. Personally, though I know this would be a drag to do in big environments, I would start by process of elimination...so whatever device is eliminated as a problem, there is less things left to be cause of the issue. 

View solution in original post

0 Kudos
(2)
24 Replies
_Val_
Admin
Admin

How do you know the FW is in fault? Any logs to prove it?

SG22
Explorer

Well, I don't know. It is just a guess because it started to work after reboot.

No, I don't. That is why I started the topic, I don't know where to look and what to do to exclude firewall as possible cause.

0 Kudos
the_rock
Authority
Authority

@SG22 ...thats why we are here to help you. So, lets start with basics...if you search the logs in dashboard and filter for say IP of your dns server and action drop, do you see anything when issue is happening? You can also run from fw expert mode this command -> fw ctl zdebug + drop | grep 53 (thats for dns port) or fw ctl zdebug + drop | grep x.x.x.x (just replace with dns server IP) and observe the results. 

0 Kudos
SG22
Explorer

No, no drops. And for fw ctl zdebug i get obsolete command. Do you know alternative command?

0 Kudos
the_rock
Authority
Authority

Can you send exact debug command you ran?

0 Kudos
_Val_
Admin
Admin

Which version of FW are you running?

0 Kudos
SG22
Explorer

Checkpoint 5200

0 Kudos
the_rock
Authority
Authority

Val meant software version, not model. Is it R80.30, R80,40, R81? Something else? What jumbo?

Just run cpinfo -y all and send the output.

0 Kudos
SG22
Explorer

Oh sorry. I also get "Deprecated command" as a resault. But I'm pretty sure we use R80.40

0 Kudos
the_rock
Authority
Authority

Ok, message me directly. Lets do remote session, I have time today. I really want to see this behavior for myself.

0 Kudos
_Val_
Admin
Admin

"fw ctl zdebug" is not a deprecated command on R80.40. A screenshot maybe?

0 Kudos
the_rock
Authority
Authority

Hey @_Val_ ...I just did zoom with @SG22 and expert mode was not set, since firewall was reconfigured, so we did that and fw zdebug ran fine.

We actually tested it with one of DNS server's IP addresses and only saw drops on port 135 on clean up rule, which is expected. Other than that, nothing dropped for port 53. We then ran curl_cli command and it showed correct output. To me, that would logically indicate its not a firewall problem, as it appears its resolving correctly from firewall side. 

0 Kudos
SG22
Explorer

I wasn't in expert mode. That is why it didn't work. @the_rock helped me a lot, but to him, it doesn't seem like a firewall issue.

0 Kudos
the_rock
Authority
Authority

Yea...personally, it seems like firewall is behaving like it should. You know what I did @SG22 ...since we spoke about 25 mins ago, I logged into 2 customers' environments and did exact same commands with their internal DNS servers and output was literally the same, no drops related to port 53 and also curl_cli gave correct output as well.

Since you mentioned there was power outage about 2 weeks ago, I have a feeling that caused this problem, if it started happening right after. Now, tracing it to specific "culprit" might be a bit of a challenge. Personally, though I know this would be a drag to do in big environments, I would start by process of elimination...so whatever device is eliminated as a problem, there is less things left to be cause of the issue. 

View solution in original post

0 Kudos
(2)
Timothy_Hall
Champion
Champion

fw ctl zdebug drop needs to be run from expert mode.

New 2021 IPS/AV/ABOT Self-Guided Video Series
now available at http://www.maxpowerfirewalls.com
0 Kudos
_Val_
Admin
Admin

That's appliance model. What's the SW version? Run "fw ver" command on the console

0 Kudos
_Val_
Admin
Admin

Also, do you have a VPN between your location and the other one?

0 Kudos
SG22
Explorer

Yes we have VPN

0 Kudos
_Val_
Admin
Admin

If you do, please check that if DNS traffic is using VPN, to communicate between DNS servers on both sites. It might be, your VPN S2S tunnel fails after two hours, and this breaks DNS. Is any other site to site communication is affected, when DNS fails?

0 Kudos
SG22
Explorer

The problem is only from our side. They can reach us without issue, but we can't reach them.

0 Kudos
the_rock
Authority
Authority

@_Val_ is correct...I mean, I get what you described here, but its hard to draw logical conclusion that this is a firewall problem. If you reboot the firewall, great, we know it works for a bit, but still does not prove fw problem. Can you post any logs, captures when issue is happening? Try do pings on the firewall at the same time, because the response in mili seconds will give us a good idea. 

Andy

0 Kudos
SG22
Explorer

So, here is what I can tell you.

No DNS errors in Event viewer on server.

I used DNS Query Sniffer on my workstation and on my DNS server. 

On workstation: First 5 queries fail always and 6th one passes. It is always a tertiary server that resolves queries. No matter which one I use as tertiary.

On the server: All queries are passing (around 150 ms for uncached ones). But also I have a problem opening web pages on DNS server.

Ping in internal network I around 1 ms (doesn't matter if I use hostname or IP address)
Ping to other location is around 30 ms (I tried to ping external IP of the Firewall and a couple of servers)

Nslookup doesn't work on workstations
Nslookup works on DNS server, but only if you ping site first (and it is really slow)
 

0 Kudos
the_rock
Authority
Authority

Let me ask you this, by the way, thanks for that explanation, thats very helpful! What happens if someone connects say via CP vpn endpoint client and tries to run these tests, is result exactly the same? Because if yes, that might be something to do with optional parameters on the gateway properties for vpn. Are you using right dns servers/dns suffix in gateway settings? If you navigate to dns and hosts from web UI page, you can check it there.

Andy

0 Kudos
SG22
Explorer

Users on endpoint client don't have any issues. DNS servers are ok, those are the one we use and suffix is ok.

0 Kudos