'Web Browser' category blocks legitimate URLs

[Yuber_Sierra_av]

Hello friends,

Would appreciate your help if you could give me some advise with this problem regarding URL Filtering/App Control blade:

Users are experiencing a very strange behavior when browsing some web pages:

First time users click a link, they get the blocked message despite the URL they are trying to access is categorized whitin the allowed categories for such users. 

Then, they go back and click the same link again, this time they are allowed access.

Review of the log shows that App control is blocking the browser (Edge, Chrome). If I allow such Apps then the policies are bypassed and users can access all categories:

Action:                    Inspect

Application Name:          Microsoft Edge

Application Description:   Microsoft Edge is a web browser developed by Microsoft.

Primary Category:          Web Browser

Matched Category:          Web Browser

Additional Categories:     Web Browser

Application Risk:          Unknown

Resource:                  https://vehiculos.tucarro.com.co/_PublishedToday_YES

Browse Time:               2

User Check:                1

UserCheck Message to User: La aplicación Microsoft Edge esta bloqueada de acuerdo a las politicas de seguridad de la compañia Category: Web Browser Para mas informacion favor comunicarse con soporte a usuarios.

UserCheck Interaction Name:Blocked Message

Access Rule Name:          Cleanup rule

Access Rule Number:        109.17

Policy Rule UID:           65566f27-e62b-4907-a52a-478888eb2780

[PhoneBoy]

Version/JHF of the gateway?
Screenshots of the precise rules in question would be helpful.
Specifically, the one that should allow traffic, the one that is blocking it, as well as the parent rule for your inline layer (109).

[Yuber_Sierra_av]

Version R81.10
JHF:  T66

Screenshot of the policy, highlight both requested rules:

In Logs I can see that URL filter blade Accepts the traffic accordingly, but then APP Control BLocks it:

Thank you.

[Wolfgang]

Screenshot of your log entry shows matching rule 109.17 but screenshot of rules shows 107.17  ?

[Yuber_Sierra_av]

Yes, that is because after I created the post had to delete two rules above the parent rule., so, rules are the same.

Thank you.

[Wolfgang]

Understand.

Rule shows a block from ApplicationControl and the rule which allow the traffic match by URLFilter. These are different blades and maybe as an idea you can change your rules. Create a new rule allowing traffic with application „edge“ or category „WebBrowser“ and as an new inline layer you can define your URLFilter rules.

[PhoneBoy]

What applications are in the group located in Rule 107.8?
Note that App Control isn't blocking it, per-se, but for some reason the (initial) traffic isn't matching whatever is listed in Rule 107.8...or any other rule in that layer.
Which means the cleanup rule for that inline layer logs the application (or URL Filtering category) that is "best match" for the traffic in question, which is probably Web Browsing.

This will probably require a TAC case to properly troubleshoot.

[Yuber_Sierra_av]

The group "Navegacion_WebLevel3" in rule 107.8 contains the URL Filering categories allowed for the corresponding AD group "Weblevel3_AD", including "Vehicles" which is the category that matches the URL that was blocked:

For http://articulo.tucarro.com.co
Categories: Vehicles

[PhoneBoy]

Generally speaking, a certain amount of traffic has to flow before we can identify a specific website/application.
Clearly it's not getting enough traffic to make that determination before it closes the connection, therefore it falls to the Cleanup Rule for the layer.
Why this is happening would need to be investigated by the TAC.

[the_rock]

I agree with @Wolfgang . The block rule match app control, but allow shows URL filtering. How is your policy configured? Do you have 2 ordered layers? Usually, what I always recommend to people is to have 1st layer as regular network layer with only fw enabled in policy settings and then 2nd layer with urlf + appc blades on.