O365 access filtering in R80.10

[Kaspars_Zibarts]

Challenge description: our user general internet access is limited to proxy only or very specific "whitelisted" IPs could be accessed directly bypassing proxy, i.e O365. Up until October we were able to script simple network group with all O365 IPv4 addresses based on XML information from MS. That has been streamedlined now and many services have only FQDNs, not IP addresses and quite a few have a wild-card in front of the FQDN (there goes domain object option..). It's all good and great with R80.20 as there you have ready made updatable objects (Updatable Objects in R80.20 ) that cover exactly this. Seems like with R80.10 the only option is these dynamic objects (Dynamic Objects in R80.10 ) that you must order from CP or enabling Application Control, that requires HTTPS inspection. Which we do not want to do as it breaks some O365 functionality and generally is not recommended by MS.

Is anyone else  facing similar challenge? Any ideas what you are going to do before R80.20 is rolled out? For us it's a simple call of SW maturity and we don't believe that R80.20 is ready to be deployed in production yet.

[Mike_A]

Kaspars,

I thought this could be accomplished in R80.10 by using the "Non-FQDN". SK (sk120633) mentions the example of checkpoint.com on a non-FQDN setup allowing access to support.checkpoint.com, community.checkpoint.com, etc. would this not solve your wildcard situation for any domain at *.domain.com? 

Domain Objects in R80.10 and above 

Snippet:

Non-FQDN mode

When FQDN mode is unchecked, traffic to the domain and its sub-domains will be matched on the rule using the non-FQDN Domain object.

Example: 
checkpoint.com as non-FQDN - all traffic to checkpoint.com and to its sub-domains, such as support.checkpoint.com, community.checkpoint.com, etc. will be matched on that object.

When upgrading domain objects from pre-R80.10, this option is enforced.

To match a rule with non-FQDN domain object, the Security gateway uses DNS reverse lookup (if the IP address is not already in cache).

Note: some DNS servers do not support DNS reverse lookups or might not be fully updated with all reverse entries.

[Kaspars_Zibarts]

Nah, non-FQDN object stops acceleration and generally is rather ineffective as it will only permit one IP in case names resolves to multiple . This option should be banned haha

[Mike_A]

Good to know, thank you!

Is there documentation on the single IP usage and stopping of acceleration when using this "Non-FQDN" objects? Or was information you obtained from TAC?

- Mike 

[Kaspars_Zibarts]

Actually it's fairly well documented

Best Practices - Working with Domain Objects (Pre R80.10) 

SecureXL Mechanism 

Couldn't find about one IP issue, but we had to learn it hard way so we left donaid objects alone pre-R80.10

[Mike_A]

Please understand I am not trying debate but better understand the documentation here and possibly help others. I to want to be able to use domain objects and possible wildcards. 

One link you sent represents a "Pre R80.10" best practice on domain objects which I have always agreed to and obeyed by, Domain objects on any version R77.30 or below was NOT a good idea. Everything I have read, starting in R80.10 this was not applicable and fixed. 

Below is from sk120633 towards the bottom. 
   

Domain objects Acceleration

Starting from R80.10, Domain objects do not disable SecureXL Accept templates anymore and support Templates Acceleration. Hence, Domain objects can be used in upper rules in the security policy with no performance impact.

The SecureXL link also references where conditions are met to not create an accepted template. Yet references "Rules that contain Domain Object" would not create an accelerated template, but states that its been resolved in R80.10 (bottom bullet)

Snippet from SecureXL Mechanism link:

All subsequent rules below such rules will not be templated as well, regardless of the rule. It is advised that all rules that can be templated, be placed at the top of the rule base (unless of course, this will violate other optimization considerations):

• Rule with service 'Any' (resolved in R75.40 and above)

• Rule with a service that has a 'handler' (where a specific protocol is chosen in 'Protocol Type' field - instead of 'None' ; go to service object - right-click - click on "Edit..." - click on "Advanced..." button - refer to "Protocol Type:" field).
  Note: This setting can be changed only in SmartDashboard R7X and lower.

• Rules that contain Port range object (resolved in R75.40 and above).

• Rules that contain Time object (resolved in R80.10).

• Rules that contain Dynamic object (resolved in R80.10).

• Rules that contain Domain object (resolved in R80.10).

[Kaspars_Zibarts]

Good point Mike! I really don't have answer for that, I guess a better clarification would be useful. But just this note alone about reverse lookups would put me off in all honesty. I don't believe you would get accurate results with this approach

Note: some DNS servers do not support DNS reverse lookups or might not be fully updated with all reverse entries.

[Mike_A]

Also a good point Kaspars! I thought in R80.10 reverse DNS lookups were removed and only forward lookups were used. Good conversation!

I think the biggest talking point here is with cloud computing. I do not believe that AWS, for example, gives you the ability to create reverse DNS entries for any of your ELB, EC2, etc instances. They rely on the user adding a CNAME to make any host name to their owned domain (domain.com) look aesthetically pleasing. How would one using Check Point (rhetorical question) for a domain.com hosted within AWS with support.domain.com, community.domain.com etc. benefit from a Non-FQDN domain object where the IP cannot be defined? If one was to allow .domain.com but not want to allow all of AWS, all the underlying reverse entries for said AWS issued IP, 100.26.82.X for example, would look like this ec2-100-26-82-X.compute-1.amazonaws.com. If security was so strict that they did not want to allow .amazonaws.com domain object, how would this be accomplished? Would we need to define FQDN objects for each sub-domain we wanted our users to hit? I can see in the example this isn't an extreme admin overhead but what about instances where various companies use then a CDN, well say Akamai, all is good if we allow .amakaitechnologies.com, until said organization decides they want to switch CDN networks... 

Timothy Hall‌ Dameon Welch-Abernathy‌ Valeri Loukine‌ any insight as to how you are seeing users overcome these "reverse DNS" hurdles? 

- Mike 

[PhoneBoy]

Best not to rely on reverse DNS lookups, which have been problematic for a couple of decades now.

It's why we eventually reworked Domain objects to support forward DNS lookups. 

[Mike_A]

Thanks Dameon!

Can we assume that a FQDN object is a forward lookup and any Non-FQDN (un-tick the box) would be subject to a reverse DNS lookup?

[PhoneBoy]

Yes, this is how Domain objects worked prior to R80.10.

Note in the mode, the objects will not generate SecureXL templates.

[Kaspars_Zibarts]

Hey Mike, thought I'll follow up on this one as one of my admins eagerly created fifty odd "old style" FQDN objects and it resulted in DNS cache completely screwed up.. Acceleration still worked but it had all sorts of weird effects on rest of the traffic

[Martin_Valenta]

Either you allow IP spaces used by O365 on r80.10 via network objects and hoping that they will not change so frequently, so you would not have to care about updating them or you will use r80.20 and updatable objects.

[Kaspars_Zibarts]

So far it's been ok based on old info but I doubt it will last too long.. we did updates daily before

[PhoneBoy]

Totally get being cautious, but we've had more than 1000 customers already upgrade to R80.20 with generally positive results.

That said you may be able to use something like this to get the IP addresses imported as objects in R80.10:

Basic script for importing IP Adress objects from feed (here office365)‌

[Kaspars_Zibarts]

Ah - that was the reason for this thread. MS has stopped XML feed (we used that ourselves before) and changed it to REST. Which in itself would not be a problem but the main issue is that whole format has changed - some services are defined by FQDN only, some just by IP, some by both and some by domains with wildcards. And that's why it became tricky to implement in R80.10. Looks like R80.20 is the only option.. Unless CP ports the same functionality to R80.10

Will have to start new thread about R80.20 MDS and VSX experiences in real world

[Astardzhiev]

You might want to check MineMeld. Yes it is product from another vendor, but it is opensource product, which doesn't require licensing and it can be integrated with other vendors. There are some articles on how to deploy the MineMeld and also some useful how to create Office365 miners. The good news is that the miners are updated to work with the new REST service and the basic setup already return all IPv4, IPv6 and URLs used for any O365 product.

[Tim_Spencer]

Having generated the feeds in Minemeld are you saying it is then possible to such these into R80.10? We have a Minemeld already and use it to feed out other firewall vendor devices but not Check Point yet as I didn't think it could.

[Tomer_Sole]

If you're the one generating the feeds, you can generate them in { "name" : "Office365Group", "members" : { "add" : ["New Host 1", "New Host 2"]   } }  format and send it via POST to https://<Management Server IP>/web-api/v1.1/set-group

I want to make the argument that solving that o365 challenge is a compelling reason to upgrade. 

Kaspars Zibarts wrote:

Will have to start new thread about R80.20 MDS and VSX experiences in real world

Care to elaborate?

[Kaspars_Zibarts]

I just wanted to hear from those running R80.20 in larger networks, how's it going so far and if they would recommend to go ahead with MDS and VSX upgrades. That was all. Since we're sort of forced to rush it out earlier than we thought. I would normally want to wait till first jumbo is released.

[Sohail_Ahmed]

Hi Kaspars

we upgrade JFH 154 to R80.10 VSX enabled gateways as advice by SE NZ here to take advantage of O365 dynamic objects but after applying the Hotfix Dynamic objects are still not supported. we open a TAC case and they informed us that Dynamic objects are only supported from R80.20 with a special Hotfix applied. that Hotfix was never applied in R80.10 environment and not GA for R80.10 yet. 

IT is really disappointment here R80.20 is recently launched and it may takes at least 6 month to becomes an stable version and fixed identified issues. Why CP R&D does not fixing this long waited issues in R80.10 which is not quite old yet.

[Dor_Marcovitch]

i also don't understand why checkpoint continue being behind the technologies security needs for such a long time.

and again they are planing to do it as an "hard coded" so internal / custom uses of it may not be achieved.

[Kaspars_Zibarts]

Not entirely sure if I follow you - dynamic objects work absolutely fine on our VSX running R80.10 JHF 142

You probably meant updatable objects ? Then the only supported version is R80.20

But I can't agree more about "being so far behind" considering hoe long O365 and other cloud solutions have existed.

[Brian_Deutmeyer]

We do this today with R80.10.  The PAC file ultimately becomes the "director" of what should go direct via firewall and what should go via proxy.

1. Subscribe to the JSON (XML previously) of published IP networks using some sort of script

2. Add/remove the networks via API

• We have a group for each subcategories (Skype, Exchange, SharePoint, Common)

3. Put the groups in firewall policy (and you need some sort of mechanism to install policy if your policies aren't already being installed on a frequent basis)

4. Implement Microsoft's proxy PAC 

• Microsoft O365 networks will go direct
• Content not hosted by Microsoft (non-Microsoft O365 IP space) will use the proxy environment

[Kaspars_Zibarts]

This was our old approach when XML existed and most services had IPs associated with them.

There are two issues now:

1) sometimes O365 client bypasses proxy settings and I have no idea why and no one has really come up with any reasonable answer. As a result you must have direct opening in the perimeter firewall

2) the new JSON list has some parts defined as FQDN with a wildcard and no IP, therefore it is impossible to cater for possible scenarios when proxy is bypassed and IP is not known upfront, for example

Yes, we still could keep scripting IP addresses from JSON lists but I believe it would make list incomplete.

[Brian_Deutmeyer]

1. This is interesting.  We were able to switch directly from XML to  new JSON published IPs without issue.

2. We are only paying attention to the IP.  With the Microsoft defined PAC file AND the IPs on the JSON, it should work just fine for normal client O365 traffic.

[Kaspars_Zibarts]

Great! Thanks for sharing that! Will try to follow up on that too! I guess the issue is that we don't use MS PAC file. Will dig into that

[Brian_Deutmeyer]

We integrated the MS PAC into ours, which says send all published MS IP space direct and everything else to the proxy.  Good luck!

[Kaspars_Zibarts]

Thought I can share my results  It seems to work nearly flawlessly as you suggested except 5 FQDNs that were meant to go via proxy (category is set to "Default" not optimized) but instead went directly from clients to MS. Still checking with our O365/client team why that would be the case

[Brian_Deutmeyer]

I was going to send you something in a DM, but apparently you need to be following me to do so...  If you want, follow me and I'll send my message.