Having a hard time redirecting Youtube requests on Gaia 80.20 VSX

[JordyMandera]

Good morning everyone,

One of my clients would like to block a small amount of websites and redirect those requests to a custom page. We implemented this last week and all website work like a charm except for one: Youtube.

Parts of the page are still allowed and I can't figure out a way to properly redirect Youtube like the other pages. I know it could have something to do with SSL certificates coming from Google.com (which is not blocked) so it's difficult to block all of them.

We are using HTTPS Inspection to inspect the traffic. If I test on my own machine I see all my traffic in my requests getting inspected but my browser keeps loading a half working Youtube website. The weird thing is that sometimes redirect works in Edge but when i go to youtube.com in Chrome and then refresh in Edge the page loads in both browsers. How is this possible?

Summarized:

- Environment is running on Gaia 80.20 (VSX)

- HTTPS Inspection is enabled

- All redirects work except for Youtube.

- We are using the "Youtube" object in the application policy to drop traffic, tested with URL as well but gave same result.

- Upgrading to 80.30 0r .40 is not an option since that failed multiple times even with TAC assistance. 

[PhoneBoy]

How about leveraging a recent R80.20 JHF with SNI support?
That should help as far as at least blocking.

Curious what were the failures to upgrade to R80.30/.40?
Perhaps start a seperate thread on this subject.

[JordyMandera]

We are running this environment on R80.20 Jumbo HF 141.

The problems with R80.30 are related to overexceptional CPU usage on the fw_worker which runs multiple software blades like Firewall, Application Control, URL Filtering, Anti-Bot and Anti-Virus. After about 30 minutes to 1 hour the CPU load freaks out and cuts internet connections. It are 12400 VSX Gateways.

TAC did provide several custom hotfixes after sending in a dozen cpinfo's but nothing helped to cure the problem and I had to rollback 2 times to R80.20 already. Looking to replace the whole stack now with something more next-gen like the 6000 series.

But for now I would like to address the Youtube problem.

[PhoneBoy]

SNI support was added in Take 117, so you're good there.

Most likely some part of the traffic is getting accepted on an earlier rule that doesn't quite classify as YouTube.
We'd have to see the rulebase for sure.
Also, what bypass rules do you have in place for HTTPS inspection?

[JordyMandera]

Posting the rulebase of one of my clients to the public internet doens't give me a great feeling to be honest. But i'll do my best to explain what's configured at the moment in HTTPS inspection and Application Control:

#1

Source: Network Group with hosts we want to block access to certain websites including Youtube

Destination: Internet

Services: HTTPS & HTTP_HTTPS_Proxy

Site Category: Custom Application/Site we created with all domains we want to block

Action: Inspect

#2

Source: RFC1918 networks

Destination: Internet

Services: HTTPS

Site Category: Financial Services & Custom Application/Site we created with domains we want to bypass inspection

Action: Bypass

#3

Source: RFC1918 Networks

Destination: Internet

Services: HTTPS

Site Category: Any

Action: Inspect

#4

Source: Any

Destination: Any

Services: HTTPS/HTTPS Proxy

Site Category: Any

Action: Bypass

Log: None

This setup works for all URLs except for Youtube. Sometimes it redirects on first attempt but when I open a second tab in the browser the website kind of opens with a lot of elements getting blocked. It's not functioning but I prefer a proper redirect page instead. All other URLs we put into the block application/site group work fine and redirect correctly on every attempt.

I've gone through the bypass groups multiple times to figure out where stuff gets bypassed and I can't find anything that would allow Youtube to open.

[PhoneBoy]

YouTube loads a lot of content from things that aren't youtube.com.
I suspect that content is being allowed and you need to include additional domains in your INSPECT rule.

[JordyMandera]

Me and our 3rd party service provider can not figure out where the issue is at. We decided to open a TAC case to do some more investigation. I'll keep you posted.

[Chris_Atkinson]

Does the behavior in Chrome change with QUIC disabled/blocked?

[JordyMandera]

Just tested this and it doesn't change the behavior when I disable QUIC in Chrome. Another test in Edge gave me a redirect to UserCheck two times and in the third tabblad I was able to open up a partly functioning Youtube. It really seems that something keeps slipping through.