topic Re: The URL list inside the Custom Application/Site object is not being applied. in Firewall and Security Management

The URL list inside the Custom Application/Site object is not being applied.

dkzndkqh — Mon, 01 Sep 2025 11:28:32 GMT

Here’s a natural English translation of what you wrote:

Currently, as seen from the nslookup, I have added the following domains to the URL list in a Custom Application/Site object:

gig-ai-g-prod-australiaeast-2-app-v4-tag.australiaeast.cloudapp.azure.com Address: 20.213.196.212

Aliases: dc.services.visualstudio.com

dc.applicationinsights.microsoft.com

dc.applicationinsights.azure.com

global.in.ai.monitor.azure.com

global.in.ai.privatelink.monitor.azure.com

dc.trafficmanager.net

australiaeast-global.in.applicationinsights.azure.com

gig-ai-prod-australiaeast-global.trafficmanager.net

For each domain, I have added three entries in the URL list, for example:
dc.services.visualstudio.com, *.dc.services.visualstudio.com, *dc.services.visualstudio.com.

Despite this, the policy containing this Custom Application/Site object is not being applied.

So I tested using regular expressions. To access gig-ai-g-prod-australiaeast-2-app-v4-tag.australiaeast.cloudapp.azure.com, I added the first domain in the chain, dc.services.visualstudio.com, to the custom object’s list as:

\/dc.services.visualstudio.com

\.dc.services.visualstudio.com

(Note: I did not remove the previous URL list entries that were not regular expressions.)

However, drop logs are still being generated starting from the first domain in the chain. If my understanding is correct, if the regular expressions for the first domain in the chain were being applied, the drop logs should appear for the second domain in the chain. Am I correct in thinking this?

current using : SG6200 R81.20SP JHT89 and management server : Smart1 5050 R81.20 JHT84

Re: The URL list inside the Custom Application/Site object is not being applied.

PhoneBoy — Tue, 02 Sep 2025 14:54:18 GMT

Can you provide a full log card via a screenshot? (Redact any sensitive details)

Re: The URL list inside the Custom Application/Site object is not being applied.

the_rock — Tue, 02 Sep 2025 23:32:11 GMT

So what exact domain fails? If you can give us a log example, like @PhoneBoy had asked, it would help.

Andy

Re: The URL list inside the Custom Application/Site object is not being applied.

dkzndkqh — Wed, 03 Sep 2025 01:22:19 GMT

It seems that, according to the logs, the URL Filtering blade detects first, and then the Firewall appears.

The URLs to be allowed are as follows. They seem to be the URLs required for using the Azure service.

gig-ai-g-prod-australiaeast-0-app-v4-tag.australiaeast.cloudapp.azure.com
gig-ai-g-prod-australiaeast-1-app-v4-tag.australiaeast.cloudapp.azure.com
gig-ai-g-prod-australiaeast-2-app-v4-tag.australiaeast.cloudapp.azure.com
dc.services.visualstudio.com

The screenshot shows how the regular expression was written, but could it be that I made a mistake in the regular expression? For now, since I don’t know what kind of subdomain might appear under the URL to be allowed, I specified it in the format like \/gig-ai-g-prod-australiaeast-0-app-v4-tag\.australiaeast\.cloudapp\.azure.com. And to allow only the main domain itself, I specified it like \.gig-ai-g-prod-australiaeast-0-app-v4-tag.australiaeast.cloudapp.azure.com.

Would the format ^gig-ai-g-prod-australiaeast-0-app-v4-tag\.australiaeast\.cloudapp\.azure.com only match the exact domain gig-ai-g-prod-australiaeast-0-app-v4-tag.australiaeast.cloudapp.azure.com?

Going further, ultimately I’m wondering why it doesn’t take effect even when I add the URLs into the existing Custom Application/Site list, and also why it doesn’t work when I use regular expressions. Could it be because the URL list inside is too large? There are about 290 entries, and most of the domains configured inside are using *.

Re: The URL list inside the Custom Application/Site object is not being applied.

the_rock — Wed, 03 Sep 2025 01:24:11 GMT

Just add *cloudapp.azure* as custom app site and it will work.

Andy

Re: The URL list inside the Custom Application/Site object is not being applied.

dkzndkqh — Wed, 03 Sep 2025 01:37:10 GMT

When accessing the domain gig-ai-g-prod-australiaeast-0-app-v4-tag.australiaeast.cloudapp.azure.com, do I also need to register the actual SNI along with it?

Re: The URL list inside the Custom Application/Site object is not being applied.

the_rock — Wed, 03 Sep 2025 01:49:06 GMT

I dont have lab access atm, will check in the morning, but either way, if you use ordered layer with appc and urlf blades on or same as network layer, just create a rule with services as custom url object and add "cloudapp.azure" and see if it lets you check regular expression (used to be able to in R81.20), but may not in R82. Regardless if it does or not, that should work.

Andy

Re: The URL list inside the Custom Application/Site object is not being applied.

Wolfgang — Wed, 03 Sep 2025 08:50:29 GMT

@dkzndkqh to use URL-Filtering on HTTPS websites you must use HTTPS inspection or the light version "Categorize HTTPS websites". With the light version the URLs have to be seen via the SNI.

Which application or service do you want to use ? Maybee a service from the "Updatable objects" can be used to allow instead of the the URL filter.

Re: The URL list inside the Custom Application/Site object is not being applied.

G_W_Albrecht — Wed, 03 Sep 2025 11:30:50 GMT

Have a look at https://regex101.com/, a good place for learning and testing RegEx !

Re: The URL list inside the Custom Application/Site object is not being applied.

PhoneBoy — Wed, 03 Sep 2025 17:35:42 GMT

The hostname part of the URL is always matched on SNI, just FYI.
That applies even with full HTTPS Inspection enabled.
If you're having matching issues, that's where I'd start looking.

Yes, ^ anchors the expression at the beginning of the URL after https://.

Re: The URL list inside the Custom Application/Site object is not being applied.

dkzndkqh — Thu, 04 Sep 2025 00:33:09 GMT

In the screenshot log I attached, the SNI value appears as dc.services.visualstudio.com. However, even if I add this URL to the URL list, it does not apply. Without using a regular expression, is there a better way to register it than using .services.visualstudio.com or dc.services.visualstudio.com?

Re: The URL list inside the Custom Application/Site object is not being applied.

dkzndkqh — Thu, 04 Sep 2025 00:33:16 GMT

At present, I can only confirm that it is related to Azure services. I also do not know exactly which specific Azure service is being used.

Re: The URL list inside the Custom Application/Site object is not being applied.

the_rock — Thu, 04 Sep 2025 01:32:25 GMT

Have you tried using updatable objects to see if that makes a difference?

Andy

Re: The URL list inside the Custom Application/Site object is not being applied.

Wolfgang — Thu, 04 Sep 2025 06:11:50 GMT

Question again..... Is HTTPS inspection and/or "categorize HTTPS websites" enabled ? URLF does not work for encrypted websites witout this.

As mentioned by @the_rock and me you can try with the "Azure Services" as destination. If you use these updatable objects you don't need HTTPS inspection.

Re: The URL list inside the Custom Application/Site object is not being applied.

dkzndkqh — Thu, 04 Sep 2025 08:12:03 GMT

Currently, I am using Categorize HTTPS, not Inspection. Is the method of using updatable objects similar to FQDN?

Re: The URL list inside the Custom Application/Site object is not being applied.

PhoneBoy — Thu, 04 Sep 2025 21:32:23 GMT

SNI is in a field called Resource, I believe (not the Destination).

Re: The URL list inside the Custom Application/Site object is not being applied.

Bob_Zimmerman — Thu, 04 Sep 2025 21:41:15 GMT

In my research, the carat anchors before the "https", so you need to include the scheme in your expression.