80.20 Take 87 HTTPS inspection Issues

[Orlando_de_Bien]

We upgraded to take 87 this weekend from take 47.

we have 4 members in our cluster so i left 2 at take 47 and set them to down and upgraded the other 2.

Initial testing seemed ok, but Monday morning our internet basically didn't work.

We quickly figured there was an issue with HTTPS inspection and i saw several out of state blocks in the logs from my machine.

If we went to a computer that was in the bypass rule of HTTPS inspection we did not have any issues.

Since I still had 2 FW's on take 47 which worked just fine before the update i failed over to them and set the take 87 fw's to down. Still had the issue.

Few ideas we had:

1. State table was screwed up because of having different takes in the same cluster - we eliminated this by shutting down all of our fw's and only bringing up one. we still experienced the internet issue.

2. Have all members on take 87. - no change.

We are currently at a loss for what to do. I am opening a ticket with support but thought i would also post here.

We have HTTPS inspection in bypass for all our computers.

Thoughts?

[Aidan_Luby]

Hey Orlando,

I just looked and don't see any known limitations on the R80.20 Jumbo Hotfix page so that's a dead end. But have you looked through your logs for HTTPS inspection errors on your SMS? Also have you tried any debugs of wstlsd or pkxld to look for handshake or other HTTPS problems? Do you have any special kernel flags enabled for things like SNI support or do you have probe bypass enabled? I know CheckPoint has been working on properly supporting SNI categorization so maybe something has changed around that. Also do any HTTPS sites work at all? If so you can try to see if certain handshake methods or support for things like SNI is broken causing issues.

[FedericoMeiners]

Hi there,

I guess that you disabled HTTPS Inspection for the moment. Remember that even if you bypass, the firewall inspects the first packet of the connection, it's not a real bypass. Please let us know if you have any special configuration enabled for HTTPS Inspection.

Regarding the out of state packets, are they from all kinds? (Out of state sync, out of state ack, etc) or only from one type? 

How are you enforcing the SSL Inspection policy? Are you using Access role objects (Identity Awareness)? Do you have many FQDN objects on top? If possible please show us parts of the policy and the number if inspection rules, please obfuscate sensible data.

¿Did you notice an increment load on the firewalls regarding Memory/CPU after the upgrade? You can check this in system counters.

I highly suggest that you re deploy your SSL Inspection policy by only specifying which subnets do you want to inspect, don't specify a bypass for the rest. Is the only way to deploy a gradual policy and not inspect at all the rest.

Also, R80.30 works really well if you are not using probe bypass, if you can try that version.

And last but not last, wstlsd debugs take a lot of CPU, so do it on a maintenance window.

Hope it helps

[Chinmaya_Naik]

Hello Team,

We upgrade some Security Gateway to R80.20.

We see that when we upgrade the Security Management Server to R80.20 with jumbo HotFix Take_47 or Take_87 then we still not face any issue.

But when we upgrade the Security Gateway to R80.20 with Jumbo HotFix Take_47 or Take_87 then suddenly we face reboot issue, Sometimes when I run "fw unloadlocal" then Gateway going to reboot automatically.

TAC is given a custom HotFix for this issue.

NOTE: We have not faced this issue with every Security Gateway.

regards

@Chinmaya_Naik 

[Yifat_Chen]

@Chinmaya_Naik  Thanks for your inputs ,  i  would like to check this issue.  I will appreciate if you can provide me yourticket # via mail ( yifatc@checkpoint.com) 

[Yifat_Chen]

@Chinmaya_Naik Indeed there is a hotfix on top of take 87.

The Fix will be part of the upcoming R80.20 Jumbo ( plan to be released as Ongoing during Aug.) 

[Peter_Lyndley]

If you are running Proxy on the gateway then I have seen reboot issues on R80.20 JHF and R80.30. There are hotfixes available for proxy users on both versions.

hope that helps

Peter

[abihsot__]

OP, we had similar issue after pushing JHF. We were using probe bypass and following information was helpful. However I don't know if this value changed R80.20 JHF47 -> JHF87, but worth checking.

• In R80.10, before Jumbo Hotfix Accumulator for R80.10 Take 189, the probing feature is set, by default, to Fail Open.
• From Take 189, the default behavior is changed to Fail Close.
• You can return to the behavior as it was before Take 189, by setting bypass_on_enhanced_ssl_inspection 1