Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
gtsanava
Contributor

R81 HTTPS Inspection

Hello

we have upgraded from R80.30 to R81 and we discovered that HTTPS Inspection has some problems there.

for example on youtube.com some video images are unable to load, these images are loaded from ytimg.com. after bypassing ytimg.com issue is resolved.

same repeats not only for youtube.com but for several websites, for example: workspace.com, admin.exchange.microsoft.com, ui.com, for some facebook.com, microsoft.com pages and many more.

In overall it looks like some content can't be inspected and is stuck somewhere.

I can't just bypass everything :))) maybe there is some fix or any experience about it? you can see my settings in attachments.

Respectfully,

George Tsanava

0 Kudos
29 Replies
_Val_
Admin
Admin

Do you actually have HTTPS enabled? How does your inspection rulebase look? Any errors/weird entries in the HTTPSi logs? Is your GW capable to resolve DNS for external servers, so SNI verification would work?

0 Kudos
gtsanava
Contributor

if you mean HTTPS Inspection then yes it is enabled, I have uploaded my inspection rule base screen too, no there are not any weird entries in logs, yes gateway can resolve any servers, it has access to internet.

0 Kudos
_Val_
Admin
Admin

Thanks. Rule 5 does not look good. Put "Internet" instead of "Any" as Destination. Also, Source field is also questionable, I would advise using "internal networks" instead of Identity Awareness user role.

How does your cleanup rule look? 

Anything related to performance bottleneck? CPU utilization on particular cores?

 

_Val_
Admin
Admin

You may also want to look into sk163595 & sk112214

the_rock
Authority
Authority

Personally and this is just my opinion, yes, @_Val_ is correct about rule 5, but I am 99% sure it wont make any difference even if you change it. TAC and myself tried playing around with rules dozens of times when customer had similar issues and it did not change the behavior at all.

0 Kudos
_Val_
Admin
Admin

Rule 5 in the current state tries decrypting all the traffic passing through the GW. It may not make a difference, but definitely needs to be changed to the best practice. It many cases it saves you a lot of CPU%

0 Kudos
the_rock
Authority
Authority

Again, I agree with you, but just speaking from my own experience with TAC :). Tried it many times, did not change cpu, memory usage or any other behavior at all.

0 Kudos
gtsanava
Contributor

rule 5 is cleanup rule and is ok because we are using that gateway as forwarding proxy server there goes traffic only from users browsers to internet so it's completely okay.

in R80.30 I even had there any source and any destination so I put there "all identified users" after upgrade to R81 but it makes no sense.

the_rock
Authority
Authority

I agree with you 100%. I can also tell you that from my personal experience, it also makes no difference what you put in there. I cant speak for other people, but having spent extended time troubleshooting https inspection with TAC escalations, that is what I experienced.

0 Kudos
_Val_
Admin
Admin

We did discuss best practices for HTTPSi in this forum a dozen of times or more. Plese look it up. If rule 5 is the last rule, you are decrypting, or trying to decrypt a lot. Also, HTTPSi happens way before the regular rulebase is matched, so user roles might be an issue there. All inspect rules for outbound should have Internet as Destination and relevant Web services as Service. A final clean-up rule should say Any-Any-Any-Bypass

 

0 Kudos
the_rock
Authority
Authority

Val, just for my own reference and sorry, not trying to be pest about this, just want to make sure I get all the FACTS correct. Is there anywhere on CP side officially documented what you stated in your response, because reading below link, I could not find it.

HTTPS inspection best practices 

And also, as I stated few times already, based on my own personal experience and speaking with 3 TAC escalations people, not one of them said that it really mattered at all the order of the rules or what was in the source and the reason why I believe it is because we tested it on customer's live environment and it absolutely made no difference whatsoever.

 

0 Kudos
Pawel_Szetela
Contributor

Hi Val,

"A final clean-up rule should say Any-Any-Any-Bypass" or should it contain services HTTPS default services?

0 Kudos
_Val_
Admin
Admin

Any for services is a better choice.

0 Kudos
Timothy_Hall
Champion
Champion

Are you sure?  The following presentation shows populated services in the final bypass rule.  Can we get a clarification on this from R&D?

HTTPS Inspection Best Practices

Edit: Please have R&D also look at my post here for accuracy: https://community.checkpoint.com/t5/Management/HTTPS-Inspection-Policy-Rule-Order/m-p/128681#M27952

 

 

New 2021 IPS/AV/ABOT Self-Guided Video Series
now available at http://www.maxpowerfirewalls.com
the_rock
Authority
Authority

Im with you on that...though I cant access the link, it would be actually nice to get a consistent/official CP recommendation on this, because so far, at least me personally, I cant seem to find one.

0 Kudos
the_rock
Authority
Authority

I had case with TAC escalations for https inspection for few months and we actually found that kernel parameter mux_enabled was causing some issues, so one thing you can try is this...if you have a cluster, run below command on whichever member is active:

fw ctl set int mux_enabled 0

Then push policy and test again. If this fixes your problem, not sure I would recommend leaving it that way, so you may want to open TAC case and have them run a debug when feature is set to 1, which is default. It supposedly has something to do with streaming and how traffic is distributed, but also plays big part on inspection as well.

I pasted below link about it:

MUX module 

0 Kudos
_Val_
Admin
Admin

Sorry to say, I cannot find a single case with "fw ctl set int mux_enabled 0" recommendations, related to R81. With all due respect, @the_rock , I believe the topic starter should follow TAC recommendations instead of playing with kernel parameters.

@gtsanava please open a support case.

0 Kudos
the_rock
Authority
Authority

I actually have maintenance window with a customer tomorrow where we will have remote with TAC, so they can take debugs for 2 really odd issues when that kernel option is set to 1. Will see how far we get...

0 Kudos
_Val_
Admin
Admin

Just saying, playing with MZX without TAC recommendations is not a good idea.

the_rock
Authority
Authority

Agree 100% and we never had...TAC escalations recommended it actually.

0 Kudos
_Val_
Admin
Admin

Even if TAC recommends something in your case, it does not mean any recommendations you received would work, let alone be safe for others. 

You can recommend troubleshooting steps and analysis techniques, also with caution. Changing kernel parameters, or any other deep engine settings - no.

Let TAC be liable to any impact those changes cause 🙂

the_rock
Authority
Authority

Well, its always sort of catch 22 situation as they say...we rely on TAC recommendations in complicated cases like that. Anyway, but I get what you are saying 🙂

 

Cheers and have a nice weekend!

0 Kudos
gtsanava
Contributor

I have opened service request and now waiting for remote session in 30 minutes so I 'll update you after, many thanks for experience you shared guys.

the_rock
Authority
Authority

Yes, please keep us posted, Im curious to see what the outcome will be...

0 Kudos
gtsanava
Contributor

Hello guys @the_rock @_Val_ @Ilya_Yusupov 

sorry for late response.

problem is resolved after we disabled new inspection for HTTP2 and instead use regular HTTPS inspection.

0 Kudos
Ilya_Yusupov
Employee
Employee

Hi @gtsanava ,

 

If you are not running JHF 42 and above so if it's possible for you to try latest take?

Asking since we have major fixes around HTTP2 that should address  to such cases.

In case it's not solving the issue please ping me and  i will engage RnD for further investigation.

 

Thanks,

Ilya  

0 Kudos
gtsanava
Contributor

I'm running JHF 36 and still didn't received JHF 42 .

we also had problem with RA VPN which is resolved from JHF44 so I think that we will skip JHF 42 and we will go for JHF 44, or if JHF42 will not cause same RA VPN problems we will not skip it 🙂

Thanks

 

0 Kudos
Ilya_Yusupov
Employee
Employee

Go ahead with JHF 44, in any case if you have any problem feel free to contact me offline and i will do my best to assist.

 

Thanks,

Ilya 

0 Kudos
Ilya_Yusupov
Employee
Employee

Hi @gtsanava ,

 

Since R80.40 we supporting HTTP2 inspection, which means in R81 you are passing through http2.

 

Please open tac case and share the number and i will push it from my sids.

 Thanks,

Ilya