topic Re: Windows Update Services with HTTPS inspection enabled in Firewall and Security Management

Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Mon, 02 Sep 2019 11:44:21 GMT

Hello,
we are having issues accessing Windows Update with HTTPs Inspection enabled (Check Point R80.20 with Take 87) and "Bypass HTTPS inspection of traffic to well-known software update services" option checked.

If, from browser, I try to surf to https://slscr.update.microsoft.com, instead of getting "403 - Forbidden: Access is denied.", I get the "ERR_CONNECTION_RESET" error.

Any advice ?

Thank you,

Luca

Re: Windows Update Services with HTTPS inspection enabled

Kim_Moberg — Mon, 02 Sep 2019 15:29:35 GMT

Hi Luca

I am seeing the similar issues while running R80.20 Take 80.

it was blocking windows update for Windows 10 Ver 1903 while doing https inspection but as soon I am using an uninspected subnet it worked.

on the working subnet I was usning wireshark to search and filter for Server HELLO messages to find domains which Windows update CDN (Content Domain Network) was being used by it.

I am though not 100% through yet.. because I have bypass the following hosts on layer 7 but also tried to bypass on ip on layer 4 in the OSI model.

The hosts are which also include some other Microsoft services:

ams15s32-in-f3\.1e100\.net
wdcp\.microsoft\.com
wns\.windows\.com
wdcpalt\.microsoft\.com
update\.microsoft\.com
download\.microsoft\.com
windowsupdate\.microsoft\.com
download\.windowsupdate\.com
wustat\.windows\.com
ntservicepack\.microsoft\.com
stats\.microsoft\.com
wns\.windows\.com
nexus\.officeapps\.live\.com
fe2\.update\.microsoft\.com
delivery\.mp\.microsoft\.com
vortex-win\.data\.microsoft\.com
cp601-prod\.do\.dsp\.mp\.microsoft\.com
geover-prod\.do\.dsp\.mp\.microsoft\.com
big\.telemetry\.microsoft\.com
ctldl\.windowsupdate\.com
audownload\.windowsupdate\.nsatc\.net
au\.download\.windowsupdate\.com\.hwcdn\.net
slscr\.update\.microsoft\.com
sfdataservice\.microsoft\.com
windowsupdate\.com
windows\.com
slscr\.update\.microsoft\.com
slscr\.update\.microosft\.com\.akadns\.net
v10\.events\.data\.microsoft\.com
v10\.event\.data\.microsoft\.com\.aria\.akadns\.net
onecollector\.cloudapp\.aria\.akadns\.net
fe2cr\.update\.microsoft\.com
fe2cr\.update\.microsoft\.com\.akadns\.net

Did you create a TAC on this issue?

Am I missing some host to get it to work?

@HeikoAnkenbrand do you have any experience with this issue?

Thanks

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Tue, 03 Sep 2019 13:09:11 GMT

Hello @Kim_Moberg,

thank you for your reply. I didn't create any TAC yet; I just shared the problem here for finding any useful information before going to open a support ticket.

We have different behavior, depending on Windows 10 build version, but in all cases, they cannot access Windows Update services. I didn't find an official Microsoft documentation listing all URLs used by Software Updates, but only:

Windows 10, version 1903, connection endpoints for non-Enterprise editions (not only SU)
Step 2: Configure WSUS (it is more related to WSUS)

I tried to create a bypass rule containing all URLs listed in the article at point 2, but it doesn't work either.
Maybe creating a bypass rule containing IP address could solve the issue, but it isn't so flexible (they could change frequently).

At this point, if no other has a solution to this, I can proceed to open a TAC.

Bye,
Luca

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Tue, 03 Sep 2019 13:07:40 GMT

Update: I opened a TAC. I'll give you more information asap.

Re: Windows Update Services with HTTPS inspection enabled

Leonardo_Tessar — Mon, 16 Sep 2019 19:59:13 GMT

Do you have any update about this issue?

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Tue, 17 Sep 2019 06:40:19 GMT

Hello @Leonardo_Tessar,
I have just a quick update regarding this issue (not yet solved).

First of all, support asked me to manual update the Trusted CA list; for some reason, it wasn't up-to-date, even if the option "Notify when a Trusted CA and Blacklist update file is available for installation" on SmartDashboard is selected:

sk132812 steps 1 - 4 to force list update (if you have doubts that the list updated).
Check the list at /opt/CPshrd-R80.20/database/downloads/TRUSTED_CA/2.0/2.3/updateFile.zip is up to date
Download the list to admin PC
Upload it manually through SmartDashboard
Install Policy

The updateFile.zip at point 2 doesn't update, file timestamp was old May 13 (a deep investigation will be needed). So they provide me the file via SFTP and after uploaded it manually on SmartDashboard (points 4 - 5), about 94 Trusted CA were updated and 3 removed.

However, it didn't resolve the issue. I'm waiting for further steps.

Bye,
Luca

Re: Windows Update Services with HTTPS inspection enabled

Kim_Moberg — Tue, 17 Sep 2019 08:24:34 GMT

Hi Luca

I also created a TAC and received the same step-by-step to update the root certificate list.

Still windows update services doesn't work after the suggested solution and waiting for R&D to update the next step.

Re: Windows Update Services with HTTPS inspection enabled

Kim_Moberg — Tue, 17 Sep 2019 09:38:25 GMT

An short update.
Check Point have managed to reproduce this issue in their labs and RnD currently working on it..
Looking forward to see an soon resolution on this issue. 🙂

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Tue, 17 Sep 2019 09:46:02 GMT

Hello @Kim_Moberg,

thank you for the update.
I didn't get any feedback yet - maybe they are waiting for feedback from R&D.

Bye,
Luca

Re: Windows Update Services with HTTPS inspection enabled

FedericoMeiners — Tue, 17 Sep 2019 12:45:59 GMT

Hello,

@Kim_Moberg When you used Wireshark did you only the client hello going and then a connection reset?

If you can please try to enable the probe bypass feature to check if that solves the issue? You can enable this on the fly, try this on non business hour to avoid possible SNI issues

None of these commands requires a reboot or cpstop/cpstart
Enable on the fly
[Expert@HostName]# fw ctl set int enhanced_ssl_inspection 1
Disable on the fly
[Expert@HostName]# fw ctl set int enhanced_ssl_inspection 0

If it's not working it's maybe because probe bypass is working on fail close, you can change this to fail open with the following:
[Expert@HostName]#fw ctl set int bypass_on_enhanced_ssl_inspection 1
To change it back to normal:
[Expert@HostName]#fw ctl set int bypass_on_enhanced_ssl_inspection 0

If you want more information you can check sk104717 - Improvements in HTTPS Inspection Bypass mechanism - Probe Bypass

Regards,

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Tue, 17 Sep 2019 13:33:13 GMT

Hello @FedericoMeiners ,

thank you for your feedback. I'll try this late evening.

Just to add this other information: before switching between probe bypass enabled/disabled on-the-fly, I usually clear the CN cache with command fw tab -t cptls_server_cn_cache -x -y.

This to avoid any "false" results. The support suggested to me that command in an old ticket (related to Check Point R80.10).

Regards,
Luca

Re: Windows Update Services with HTTPS inspection enabled

Ilya_Yusupov — Wed, 18 Sep 2019 07:52:59 GMT

Hi all,

we are replicate the issue and working with RnD to solve it, as current WA you will need to bypass the domains in the SSLi RB.

This issue only happen on Windows 10 clients.

i will update once we will have resolution.

Thanks,

Ilya

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Wed, 18 Sep 2019 08:01:07 GMT

Hello @Ilya_Yusupov,

thank you for your feedback.

I'm pretty ignorant, so, please, can you explain to me what "SSLi RB" means? I can imagine it has to do with SSL; some by-pass rule ?

Thank you again.

Regards,
Luca

Re: Windows Update Services with HTTPS inspection enabled

Ilya_Yusupov — Wed, 18 Sep 2019 08:29:12 GMT

@lucafabbri365 - i will contact you offline.

Re: Windows Update Services with HTTPS inspection enabled

Kim_Moberg — Wed, 18 Sep 2019 14:52:02 GMT

@lucafabbri365 and @Ilya_Yusupov

R&D were able to reproduce this error and found out Microsoft changed its servers to support only ECDSA cipher suites.
Firewall by default, does not propose ECDSA

You have to enabled ECDSA and update https_inspection_white_list.bin

Please contact TAC to get the solution. It worked for me on R80.20 AC

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Wed, 18 Sep 2019 15:35:10 GMT

Hello @Kim_Moberg ,
I received feedback from support with the same procedure.

I'll update you, here, asap.

Bye,
Luca

Re: Windows Update Services with HTTPS inspection enabled

Kim_Moberg — Wed, 18 Sep 2019 15:40:18 GMT

@lucafabbri365

Good to hear.

You might experience the command cpstop;cpstart fails and your cluster sync dont work.

then go once again with cpstart and you cluster will be running again..

I experienced this on both members in my cluster after the update. So cpstart is way to go 🙂

Let me know how the result is.

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Wed, 18 Sep 2019 15:49:43 GMT

Hello @Kim_Moberg,
I was thinking, instead of cpstop and cpstart, to reboot each node.

However I'll try them at first.

Bye,
Luca

Re: Windows Update Services with HTTPS inspection enabled

Peter_Lyndley — Wed, 18 Sep 2019 16:12:55 GMT

I assume there will be a public SK for this shortly ? so that everyone else knows what is going on?

Re: Windows Update Services with HTTPS inspection enabled

lucafabbri365 — Thu, 19 Sep 2019 06:56:35 GMT

Hello all,
just a quick update:

1. Into one Check Point node, we enabled ECDSA and update https_inspection_white_list.bin (as suggested by support)
2. Into the other node, we enabled ECDSA only (as discussed and suggested by @Ilya_Yusupov, privately)

Both solutions seem to work. So it shouldn't need to replace *.bin file.

I'm waiting for confirmation.

Bye,
Luca