Solved: Re: Most URLs categorized as X-VPN this morning

Marcel_Gramalla · ‎2024-04-10

Hi,

we encountered a big issue this morning as nearly all URLs were categorized as X-VPN application which is blocked in our rulebase because of the category (Anonymizer) and it's also set to critial risk.

Did you encounter the same and is there any official statement? It's obviously an issue with the database and the first time we see such an issue.

TP_Master · ‎2024-04-11

Hi,

I can confirm that indeed from the last ~15 minutes this issue has been resolved, using the APPI package 11042401.

Gateways will be updating according to their scheduled update policy.

If you need to rush the update, please do the following on the security gateways:

- # rm $FWDIR/appi/update/Version
- # rm $FWDIR/appi/update/next_update
- Wait for 5 minutes.

A new dedicated sk182202 is now available for this issue.

An official and detailed RCA will follow.

Ofir Israel

VP, Threat Prevention Check Point Software Technologies

P.S. We are aware that the fix seems to have not worked for a limited amount of our customers. We currently suspect this as a logging behavior where old connections have new sessions and are updated on the log view. If you encounter drops please let us know through a TAC ticket, we are monitoring the situation.

View solution in original post

796570686578 · ‎2024-04-12

I noticed something that may help others as well.

Yesterday once the issue has been resolved and the new package released, I performed the Update of the Application Control & URL Filtering via SmartConsole. This resolved the issue on 2 out of 3 clusters. The 3rd cluster, although it had the same updated package version(110424_1) as the other gateways, was still dropping traffic.

So I performed the steps mentioned in https://support.checkpoint.com/results/sk/sk182202 and suddenly it worked. The package version was still the same but apparently there was a difference between updating via Mgmt and manually deleting the files.

Hope this helps anyone else

View solution in original post

kale24 · ‎2024-04-12

All gateways had the new file yet we still had to create the manual whitelist for X-VPN.
I then completed the SK article on ALL gateways (even ones not using the application blade) and only then did we stop seeing logs.
I will monitor to see if anymore appear but the trick is going through the SK regardless of the file date.

View solution in original post

TP_Master · ‎2024-04-17

Hi, let me introduce some more information.

As I wrote last week even after releasing the package we encountered cases of customers still seeing the issue. We later saw evidence on incorrect MD5 and that's why we added that section to the SK182202.

We did not tweak the signatures in the package therefore no new package was issued but we did perform manual operations to speed up the integration & deployment of package 110424_1 into the updates system, so that our customers get it faster than usual - this caused some issues with wrong MD5 for small portion of customers.

Therefore the recommendation is for anyone who still has issues with X-VPN classification to check the MD5 and if necessary force re-download of the package.

Ofir Israel

VP, Threat Prevention

Check Point

View solution in original post

Albin · ‎2024-04-10

Hi,

We have the same issue. We had this the other day and just now again on one site.

Daniel_Hainich · ‎2024-04-10

hi,

we have the same issue since today.

sascham · ‎2024-04-10

Hello,

same issue here. It started around 04:12 AM CEST.

Sebastian · ‎2024-04-10

Hi all,

We have the exact same issue! Nothing is passing by as every web request is categorized as X-VPN and therefore blocked.

Nenad_Odic · ‎2024-04-10

We have this from this morning,is there a reason why or we have to raise a TAC ticket?

tomaszek · ‎2024-04-10

Yes, we have the same issue

Stephan_Scholz · ‎2024-04-10

Same here. Any comments from Check Point?

Nenad_Odic · ‎2024-04-10

Just raised

Case

6-0003912005

so we will see if they come up with an idea ,other that remove anonymizer category from rules.

Josef_Maier · ‎2024-04-11

Hi,

we have the same problem. We registered an ticket bye Check Point Support. At the moment no reaction. On the status Checkpoint website https://status.checkpoint.com/ there are all services OK.

_Val_ · ‎2024-04-11

Hi all,

We are aware of the miscategorization by Application Control / URL Filtering blade, where legitimate traffic is categorized as X-VPN and being dropped accordingly. Right now, a joint Task force that includes both R&D and TAC experts are working together to mitigate the issue.

We will keep you posted

Nenad_Odic · ‎2024-04-11

thank you Val,

hope you will sort it out

_Val_ · ‎2024-04-11

I will update this post once we have a definitive fix.

796570686578 · ‎2024-04-11

Thank you for your quick response!

Is it possible that you provide a status update in this thread as soon as you get the news that the issue has been resolved? That would would be amazing.

Appreciate you and best regards

FedericoB · ‎2024-04-11

I'm experiencing the same issue since this mornig.

vNenad · ‎2024-04-11

Hey @_Val_.

This issue has caused disruption, and it raises a question about communication. While we understand technical problems occur, many partners rely on prompt notifications from Check Point to address such situations effectively.

I'd like to inquire why there wasn't a more immediate communication effort to partners regarding this miscategorization issue. Timely updates would have significantly aided troubleshooting and minimized disruption for many organizations. Are there plans to improve communication protocols for future occurrences?

In addition to the previous questions, I'm also interested in:

Any recommended workarounds or mitigation strategies in the meantime.
Whether there's an estimated timeframe for a resolution.

Thank you for your time and any information you can share.

Sincerely,
Nenad Vijatov

Nigel_Costar · ‎2024-04-11

Please can we have an update and ETA on the issue this is causing significant disruption for our staff and CP are being frustratingly quiet. the website reports there will be an update every 30 minutes but I do not see any posts regarding this?

GeoMal86 · ‎2024-04-11

Same here... We have spent an hour thinking that our PCs were infected and trying to find what happened.

This seems to only be happening on Harmony Connect?

Update: nvm, it's happening on all gateways.

Tom_Hinoue · ‎2024-04-11

It seems the issue is occurring for users that are using Full SSL Inspection.
We have a case open to TAC since yesterday but no updates yet.

I noticed X-VPN signature was updated on 8-Apr (Mon) in APPI package No. 070424_1.
If this is the case, then I hope a new package is released soon 🙂

Vincent_Croes · ‎2024-04-11

This is not the case. We do not use full ssl inspect and are still impacted.

Question to Check Point, when can we expect an official statement?

Tom_Hinoue · ‎2024-04-11

Thanks for sharing.
It's strange because we have many customers but don't experience this issue with only HTTPS categorization... Note we do have Anonymizers to be blocked in the policy.

Hugo_vd_Kooij · ‎2024-04-11

Not so weird at all. The detection is based on a perceived header. Without HTTPS inspection you will not be able to look into that traffic to match this.

<< We make miracles happen while you wait. The impossible jobs take just a wee bit longer. >>

Vincent_Croes · ‎2024-04-11

Not 100% what you mean by this but we have multiple environments impacted that are not decrypting one bit of SSL. In other words, SSL inspection is not a pre-requisite to experience this issue.

HTTPS categorization is enabled.

Tom_Hinoue · ‎2024-04-11

Thanks all for the insight 🙂
I was just wondering because that we have more than hundreds of customers, but only 3-4 reports from the field experiencing this issue, and it was just that all of those 3-4 customers had Full SSLINS enabled.

Btw, I just noticed there was a APPI package update No.110424_1 with just "X-VPN" updated.
Maybe the issue was addressed?🤔

Nenad_Odic · ‎2024-04-11

if you meant this one it does not help still traffic is x-vpn categorized

nfinney1878 · ‎2024-04-11

Where do you see that "X-VPN" was updated?

Tom_Hinoue · ‎2024-04-11

I recieved a mail from APPWIKI_UPDATES just around an hour ago. I haven't checked on MGMT yet... probably the package is still not distributed to the update servers?

EngineerActo · ‎2024-04-11

Seems a bit more complicated than that. Incoming traffic to our own website's webservers with HTTPS-inspection and IPS also gets frequently blocked as X-VPN, but not all, even when the same URL's are requested. Not every browser triggers this protection, but mostly Chrome and MS Edge do.
Not all users are impacted by the issue, or many hours after others were inpacted by the issue. Hopefully it is NOT something that get's distributed to pc's with (cloud) shared/synchronized browser profiles, such as google and microsoft stored browser profiles.

StefanSchmidt · ‎2024-04-11

Same problem here, seeing this since April 8 4pm CEST

Hugo_vd_Kooij · ‎2024-04-11

Stefan, Did you open a service request yet? As you seem to be seeing this 2 to 3 days before most other see it.

And this is the sort of additional insight that might help finding a root cause.

<< We make miracles happen while you wait. The impossible jobs take just a wee bit longer. >>

Are you a member of CheckMates?

Most URLs categorized as X-VPN this morning