topic Re: custom url query in General Topics

custom url query

kb1 — Fri, 07 May 2021 19:34:52 GMT

So we have a url whose ip changes frequently and im not able to make a working rule to for the url. The url uses a specific port as well and in the rule there are 2 specific source ips.

Ive tried the following rule:

the ems2.swims.faa.gov object looks as shown below:

So as you can see in the pic above i have used "*.ems2.swim.faa.gov", i just changed it to this expression and do not know if this one will work as we haven't tested it yet, the previous expression i used was "ems2.swim.faa.gov" which did not work.

Now if the expression used above also doesn't work what should i use to make it work?

Note that https inspection is not enabled but categorize https inspection is enabled.

Firewall cluster is running on R80.20 with cpinfo -y all shown below:

cpinfo -y all

This is Check Point CPinfo Build 914000202 for GAIA
[IDA]
No hotfixes..

[CPFC]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 118

[MGMT]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 118

[FW1]
HOTFIX_MAAS_TUNNEL_AUTOUPDATE
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 118

FW1 build number:
This is Check Point's software version R80.20 - Build 163
kernel: R80.20 - Build 151

[SecurePlatform]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 118

[CPinfo]
No hotfixes..

[PPACK]
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 118

[DIAG]
No hotfixes..

[CVPN]
HOTFIX_ESOD_SCANNER_AUTOUPDATE
HOTFIX_ESOD_CSHELL_AUTOUPDATE
HOTFIX_ESOD_SWS_AUTOUPDATE
HOTFIX_R80_20_JUMBO_HF_MAIN Take: 118

[CPUpdates]
BUNDLE_HCP_AUTOUPDATE Take: 29
BUNDLE_ESOD_SCANNER_AUTOUPDATE Take: 9
BUNDLE_ESOD_CSHELL_AUTOUPDATE Take: 13
BUNDLE_ESOD_SWS_AUTOUPDATE Take: 14
BUNDLE_MAAS_TUNNEL_AUTOUPDATE Take: 53
BUNDLE_INFRA_AUTOUPDATE Take: 41
BUNDLE_DEP_INSTALLER_AUTOUPDATE Take: 23
BUNDLE_R80_20_JUMBO_HF_MAIN Take: 118

[CPDepInst]
No hotfixes..

[AutoUpdater]
No hotfixes..

[hcp_wrapper]
HOTFIX_HCP_AUTOUPDATE

Also there is another rule which is being used to block traffic to Microsoft URLs as shown below and it works:

The "Block custom URLs" object looks as shown below:

So if this rule works then im assuming https inspection (we will be using a different solution for https inspection) need not be enabled ?

So yeah bottom-line is i need to make the ems2.swims.faa.gov rule to work whenever the ip changes dynamically.

Update : Testing has been done and it looks like that url (*.ems2.swims.faa.gov) also doesnt work.

Thank You.

Re: custom url query

the_rock — Sat, 08 May 2021 21:08:20 GMT

What does the log show when you filter for this destination? Does filtering for blade "url filtering" show you anything for this at all? Based on the screenshot, looks correct to me.

Re: custom url query

kb1 — Sat, 08 May 2021 21:30:21 GMT

The logs do not show unlike the logs that are showing up for the microsoft rule that I posted.

Re: custom url query

the_rock — Sat, 08 May 2021 21:51:29 GMT

I believe you are correct in this case https inspection might not be needed, but maybe someone else can confirm 100%. Well, if that same rule is hit, then we know rule is indeed working...can you please tell us what are exact FQDNs you are having issues with? I can try in my lab.

Re: custom url query

kb1 — Mon, 10 May 2021 16:21:16 GMT

These are the fqdns (and they do not work) that i tried:

ems2.swims.faa.gov
*.ems2.swims.faa.gov

The user is trying to hit the url "ems2.swims.faa.gov" on a specific port. This url has a dynamic ip that changes every few weeks or so that is why i need to make it work.

Re: custom url query

Tobias_Moritz — Tue, 11 May 2021 12:49:33 GMT

You always talk about "URL". Are you sure you are using HTTP(S) here?

I'm asking this, because you are using a Custom Application/Site Object with URL List definition and this only works for:

HTTP traffic over tcp ports included in "Web Browsing" object (see URL Filtering blade settings) -> HTTP Header is parsed
HTTPS traffic over tcp ports included in "Web Browsing" object (see URL Filtering blade settings) AND ((HTTPS Inspection blade is used and this traffic is indeed inspected -> HTTP Header is parsed) OR (HTTPS Inspection Lite feature of URL Filtering Blade is used AND server certificate is trusted by CP gateway (see Trusted CA list in CP database) -> TLS handshake is parsed))

When using Internet in the destination column, you have to make sure that the ip address(es) behind this FQDN is indeed Internet from perspective of this gateways topology.

Also, please take care of the service field of your rule. In your first screenshot, you have three objects in the service field:

your Custom Application/Site Object
a TCP object named TCP-55443
a ICMP object

These objects are combined as OR-conjunction. So your rule currently allows TCP-55443 and ICMP to everything which matches Internet. I'm not sure, this is what you want.

Last but not least:

Regarding your specific requirement:

Why you do you not skip URL-Filtering blade (and its Custom Application/Site Objects) here and just use a FQDN network object instead of Internet in the destination field? This is enforced by firewall blade only. Just make sure you keep the FQDN checkbox ticked. Since R80.20, this feature is really usable.

Re: custom url query

kb1 — Tue, 11 May 2021 20:12:16 GMT

ok i will try it out and will update here.

Re: custom url query

Sorin_Gogean — Wed, 12 May 2021 06:15:08 GMT

Why not try-it like we do with cases like this (as it was recommended also by others).
We define the .ems2.swims.faa.gov as an FQDN object (will replace the Internet destination in you initial rule)
Then we remove the URL object and leave only the required special TCP port - ICMP would not be needed as ICMP is allowed by default through CKP Firewall .

That will solve the problem with IP change on servers where the ems2.swims.faa.gov is hosted, and allow access to that particular port you require.

Re: custom url query

kb1 — Wed, 12 May 2021 17:56:06 GMT

ok i did as you suggested, this is how the rule looks:

the fqdn box is checked on that object.

Now i need to find a way to test it out and if its working will update here thanks.

Re: custom url query

kb1 — Wed, 12 May 2021 18:16:27 GMT

By the way does the "FQDN" box need to be checked?

Re: custom url query

Tobias_Moritz — Fri, 14 May 2021 06:37:31 GMT

Yes, as I said in my previous post, the "FQDN" checkbox absolutly needs to be checked. Please read the thread I linked in my previous post to understand why.

Re: custom url query

kb1 — Wed, 19 May 2021 16:21:05 GMT

So looks like the rule didnt work, the ip changed dynamically again recently and today the user complained that its not working again, is the rule correct? it does not require the url filtering blade to function right?

Re: custom url query

Tobias_Moritz — Tue, 01 Jun 2021 05:34:51 GMT

It should work. URL Filtering blade is not needed for domain objects, these are handled by firewall blade.

Are you sure your gateway can do correct name resolution for that domain?

You can debug it on expert shell on gateway.

Here an example for registry-1.docker.io, created as FQDN object .registry-1.docker.io

[Expert@gateway:0]# dig registry-1.docker.io ; <<>> DiG 9.3.6-P1-RedHat-9.3.6-25.P1.11.cp994000013 <<>> registry-1.docker.io ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39231 ;; flags: qr rd ra; QUERY: 1, ANSWER: 8, AUTHORITY: 4, ADDITIONAL: 1 ;; QUESTION SECTION: ;registry-1.docker.io. IN A ;; ANSWER SECTION: registry-1.docker.io. 9 IN A 34.197.211.151 registry-1.docker.io. 9 IN A 107.23.149.57 registry-1.docker.io. 9 IN A 54.85.56.253 registry-1.docker.io. 9 IN A 35.153.88.109 registry-1.docker.io. 9 IN A 3.224.96.239 registry-1.docker.io. 9 IN A 3.229.227.53 registry-1.docker.io. 9 IN A 18.214.230.110 registry-1.docker.io. 9 IN A 34.238.187.50 ;; AUTHORITY SECTION: docker.io. 30290 IN NS ns-1827.awsdns-36.co.uk. docker.io. 30290 IN NS ns-1168.awsdns-18.org. docker.io. 30290 IN NS ns-421.awsdns-52.com. docker.io. 30290 IN NS ns-513.awsdns-00.net. ;; ADDITIONAL SECTION: ns-421.awsdns-52.com. 112798 IN A 205.251.193.165 ;; Query time: 3 msec ;; SERVER: 192.168.a.b#53(192.168.a.b ;; WHEN: Tue Jun 1 07:30:21 2021 ;; MSG SIZE rcvd: 322 [Expert@gateway:0]# domains_tool -d registry-1.docker.io Domain is not attached to any IP address Wait for the next chunk... Domain is not attached to any IP address Wait for the next chunk... Domain is not attached to any IP address Wait for the next chunk... --------------------------------------------------------------------------------------------------- | Given Domain name: registry-1.docker.io FQDN: yes | --------------------------------------------------------------------------------------------------- | IP address | sub-domain | --------------------------------------------------------------------------------------------------- | 34.231.251.252 | no | | 54.161.109.204 | no | | 54.152.28.6 | no | --------------------------------------------------------------------------------------------------- Total of 3 IP addresses found Wait for the next chunk... --------------------------------------------------------------------------------------------------- | Given Domain name: registry-1.docker.io FQDN: yes | --------------------------------------------------------------------------------------------------- | IP address | sub-domain | --------------------------------------------------------------------------------------------------- | 52.55.168.20 | no | | 54.85.56.253 | no | --------------------------------------------------------------------------------------------------- Total of 2 IP addresses found Wait for the next chunk... --------------------------------------------------------------------------------------------------- | Given Domain name: registry-1.docker.io FQDN: yes | --------------------------------------------------------------------------------------------------- | IP address | sub-domain | --------------------------------------------------------------------------------------------------- | 3.209.182.229 | no | | 3.229.227.53 | no | | 35.175.91.243 | no | | 35.153.88.109 | no | --------------------------------------------------------------------------------------------------- Total of 4 IP addresses found Wait for the next chunk... --------------------------------------------------------------------------------------------------- | Given Domain name: registry-1.docker.io FQDN: yes | --------------------------------------------------------------------------------------------------- | IP address | sub-domain | --------------------------------------------------------------------------------------------------- | 52.72.232.213 | no | | 3.220.36.210 | no | --------------------------------------------------------------------------------------------------- Total of 2 IP addresses found Wait for the next chunk... --------------------------------------------------------------------------------------------------- | Given Domain name: registry-1.docker.io FQDN: yes | --------------------------------------------------------------------------------------------------- | IP address | sub-domain | --------------------------------------------------------------------------------------------------- | 54.236.165.68 | no | | 3.224.96.239 | no | | 35.169.249.115 | no | | 107.23.149.57 | no | | 18.214.230.110 | no | | 34.197.211.151 | no | --------------------------------------------------------------------------------------------------- Total of 6 IP addresses found Wait for the next chunk... --------------------------------------------------------------------------------------------------- | Given Domain name: registry-1.docker.io FQDN: yes | --------------------------------------------------------------------------------------------------- | IP address | sub-domain | --------------------------------------------------------------------------------------------------- | 34.238.187.50 | no | --------------------------------------------------------------------------------------------------- Total of 1 IP addresses found Wait for the next chunk... Domain is not attached to any IP address Wait for the next chunk... Domain is not attached to any IP address [Expert@gateway:0]#