Excessive DNS Queries from Gateways

[timothyjwitt]

Hello,
We started to see an excessive amount of DNS queries coming from our gateways seemingly looking up FQDN objects.  This started 12/17/2023 at 1am and we see it on multiple gateways at multiple sites with different DNS servers in Gaia config.
When I say excessive, one site 'normal' operation prior to the issue had 92k (10MB) dns queries from the gateway per hour and after this past Sunday it's at 17Million (1.8GB) per hour.
No changes to the environments since 12/14/2023, we are currently in a code freeze.
We are on R81.10 JHF 95.
We have ~580 FQDN objects in policies.
Policy install didn't fix, nor did failover but a reboot seems to have resolved it.
Wondering if anyone else has seen anything strange like this.
TAC has been engaged.
Thanks,
Tim

[PhoneBoy]

The DNS cache is limited to 25000 entries.
With that many FQDN objects, it's possible you're exceeding this limit and you may need to adjust it: https://support.checkpoint.com/results/sk/sk157493
Otherwise, I suggest involving the TAC.

[timothyjwitt]

The fw tab -t dns_reverse_cache_tbl command output is completely different on my GW's but if I'm reading that correctly the size (limit) is 28672 and the number of entries is 4114.  However, the limit in table.def is set at 25000.
-------- dns_reverse_cache_tbl --------
htab_bl, id 35, size 28672, attributes: expire, no links, #vals 4114 #slinks 0

I'm stuck on this occurring across our GW fleet at a specific date and time, seems like some sort of automatic update?
I've got a case open with TAC.

[PhoneBoy]

If you've hit the high water mark of 25,000, I believe it will show in the output of fw tab -t dns_reverse_cache_tbl -s.
That would at least tell us if my theory is correct.

[timothyjwitt]

Looks like peak was 12725
fw tab -t dns_reverse_cache_tbl -s
HOST NAME ID #VALS #PEAK #SLINKS
localhost dns_reverse_cache_tbl 38 12725 0 0

[timothyjwitt]

What appears to be happening is that every time an FQDN rule gets hit, the gateway is looking up the IP rather than using DNS cache.

[the_rock]

You could try below

https://support.checkpoint.com/results/sk/sk32224

Andy