Custom Application/Site Findings

[Bob_Zimmerman]

For years, I've been wanting to test Check Point's Custom Application/Site objects' matching logic. Finally got a chance to.

/etc/hosts on the client has these two lines added:

192.168.5.2     gooddomain.local subdomain.gooddomain.local
192.168.5.2     baddomain.local gooddomain.local.baddomain.local subdomain.gooddomain.local.baddomain.local

I built a self-signed certificate on the server and set up httpd(8) to serve a simple page via both HTTP and HTTPS using that self-signed cert. I build a small KSH script on the client with my test cases:

#!/bin/ksh
set -A domainsList "http://gooddomain.local/" \
"https://gooddomain.local/" \
"http://gOoDdOmAiN.lOcAl/" \
"https://gOoDdOmAiN.lOcAl/" \
"http://gooddomain.local:80/" \
"https://gooddomain.local:443/" \
"http://gOoDdOmAiN.lOcAl:80/" \
"https://gOoDdOmAiN.lOcAl:443/" \
"http://subdomain.gooddomain.local/" \
"https://subdomain.gooddomain.local/" \
"http://baddomain.local/" \
"https://baddomain.local/" \
"http://gooddomain.local.baddomain.local/" \
"https://gooddomain.local.baddomain.local/" \
"http://subdomain.gooddomain.local.baddomain.local/" \
"https://subdomain.gooddomain.local.baddomain.local/" \
"http://baddomain.local/gooddomain.local/" \
"http://gooddomain.local:password@baddomain.local/" \
"http://user:gooddomain.local@baddomain.local/" \
"http://baddomain.local/gooddomain.local" \
"http://baddomain.local/gooddomain.local/" \

for domain in ${domainsList[*]}; do
printf "%60s: " $domain
curl -k "$domain" >/dev/null 2>/dev/null
exitCode=$?
if [ $exitCode -eq 0 ]; then
echo "Success"
else
echo "   Fail ${exitCode}"
fi
done

And finally, I set up the policy on the firewall (standalone, R81.10 jumbo 94) in the middle. Here it is in abbreviated form:

"rulebase" : [ {
	"rule-number" : 1,
	"source" : [ "Any" ],
	"destination" : [ "Any" ],
	"service" : [ "GoodDomain" ],
	"action" : "Accept",
}, {
	"rule-number" : 2,
	"source" : [ "Any" ],
	"destination" : [ "Internet" ],
	"service" : [ "Any" ],
	"action" : "Accept",
}, {
	"rule-number" : 3,
	"source" : [ "ClientNet", "ServerNet" ],
	"destination" : [ "Outside Private (to reach my NTP server)" ],
	"service" : [ "Any" ],
	"action" : "Accept",
}, {
	"rule-number" : 4,
	"source" : [ "Any" ],
	"destination" : [ "Any" ],
	"service" : [ "SSH" ],
	"action" : "Accept",
}, {
	"rule-number" : 5,
	"source" : [ "Any" ],
	"destination" : [ "Any" ],
	"service" : [ "Any" ],
	"action" : "Drop",
} ]

"GoodDomain" is the Application/Site object I built for testing. HTTPS Inspection is not set up.

With the first rule's Application/Site object set to match "gooddomain.local" not as a regular expression:

obsd-client# ./domainTest.sh
                                    http://gooddomain.local/: Success
                                   https://gooddomain.local/: Success
                                    http://gOoDdOmAiN.lOcAl/: Success
                                   https://gOoDdOmAiN.lOcAl/: Success
                                 http://gooddomain.local:80/: Success
                               https://gooddomain.local:443/: Success
                                 http://gOoDdOmAiN.lOcAl:80/: Success
                               https://gOoDdOmAiN.lOcAl:443/: Success
                          http://subdomain.gooddomain.local/:    Fail 56
                         https://subdomain.gooddomain.local/:    Fail 35
                                     http://baddomain.local/:    Fail 56
                                    https://baddomain.local/:    Fail 35
                    http://gooddomain.local.baddomain.local/: Success
                   https://gooddomain.local.baddomain.local/: Success
          http://subdomain.gooddomain.local.baddomain.local/:    Fail 56
         https://subdomain.gooddomain.local.baddomain.local/:    Fail 35
                    http://baddomain.local/gooddomain.local/:    Fail 56
           http://gooddomain.local:password@baddomain.local/:    Fail 56
               http://user:gooddomain.local@baddomain.local/:    Fail 56
                     http://baddomain.local/gooddomain.local:    Fail 56
                    http://baddomain.local/gooddomain.local/:    Fail 56

Not great. It allows some potential phishing domains.

With the Application/Site object set to match "gooddomain.local/":

obsd-client# ./domainTest.sh                                                   
                                    http://gooddomain.local/: Success
                                   https://gooddomain.local/:    Fail 35
                                    http://gOoDdOmAiN.lOcAl/: Success
                                   https://gOoDdOmAiN.lOcAl/:    Fail 35
                                 http://gooddomain.local:80/: Success
                               https://gooddomain.local:443/:    Fail 35
                                 http://gOoDdOmAiN.lOcAl:80/: Success
                               https://gOoDdOmAiN.lOcAl:443/:    Fail 35
                          http://subdomain.gooddomain.local/:    Fail 56
                         https://subdomain.gooddomain.local/:    Fail 35
                                     http://baddomain.local/:    Fail 56
                                    https://baddomain.local/:    Fail 35
                    http://gooddomain.local.baddomain.local/:    Fail 56
                   https://gooddomain.local.baddomain.local/:    Fail 35
          http://subdomain.gooddomain.local.baddomain.local/:    Fail 56
         https://subdomain.gooddomain.local.baddomain.local/:    Fail 35
                    http://baddomain.local/gooddomain.local/:    Fail 56
           http://gooddomain.local:password@baddomain.local/:    Fail 56
               http://user:gooddomain.local@baddomain.local/:    Fail 56
                     http://baddomain.local/gooddomain.local:    Fail 56
                    http://baddomain.local/gooddomain.local/:    Fail 56

Worse, as it doesn't allow HTTPS.

Would anybody like to see other URLs tested? I plan to explore other matching expressions, and I plan to try to enable HTTPS Inspection.

Edit: I realized I should include the exit codes in case some failures are for things other than connectivity being prevented. And of course, I discovered that subdomain.gooddomain.local.baddomain.local was failing to resolve, so I added that to /etc/hosts on the client. Modifications made to the hosts file, the script, and the results above.

Edit 2: Added some phishy paths to the script and the results.

[Bob_Zimmerman]

Require at least one subdomain:

^https?://([^/]+\\.)+gooddomain.local(/|$)

As regular expression

obsd-client# ./domainTest.sh
                                    http://gooddomain.local/:    Fail 56
                                   https://gooddomain.local/:    Fail 35
                                    http://gOoDdOmAiN.lOcAl/:    Fail 56
                                   https://gOoDdOmAiN.lOcAl/:    Fail 35
                                 http://gooddomain.local:80/:    Fail 56
                               https://gooddomain.local:443/:    Fail 35
                                 http://gOoDdOmAiN.lOcAl:80/:    Fail 56
                               https://gOoDdOmAiN.lOcAl:443/:    Fail 35
                          http://subdomain.gooddomain.local/: Success
                         https://subdomain.gooddomain.local/: Success
                                     http://baddomain.local/:    Fail 56
                                    https://baddomain.local/:    Fail 35
                    http://gooddomain.local.baddomain.local/:    Fail 56
                   https://gooddomain.local.baddomain.local/:    Fail 35
          http://subdomain.gooddomain.local.baddomain.local/:    Fail 56
         https://subdomain.gooddomain.local.baddomain.local/:    Fail 35
                    http://baddomain.local/gooddomain.local/:    Fail 56
           http://gooddomain.local:password@baddomain.local/:    Fail 56
               http://user:gooddomain.local@baddomain.local/:    Fail 56
                     http://baddomain.local/gooddomain.local:    Fail 56
                    http://baddomain.local/gooddomain.local/:    Fail 56

Allow any number of subdomains (including zero):

^https?://([^/]+\\.)*gooddomain.local(/|$)

As regular expression

obsd-client# ./domainTest.sh
                                    http://gooddomain.local/: Success
                                   https://gooddomain.local/: Success
                                    http://gOoDdOmAiN.lOcAl/: Success
                                   https://gOoDdOmAiN.lOcAl/: Success
                                 http://gooddomain.local:80/: Success
                               https://gooddomain.local:443/: Success
                                 http://gOoDdOmAiN.lOcAl:80/: Success
                               https://gOoDdOmAiN.lOcAl:443/: Success
                          http://subdomain.gooddomain.local/: Success
                         https://subdomain.gooddomain.local/: Success
                                     http://baddomain.local/:    Fail 56
                                    https://baddomain.local/:    Fail 35
                    http://gooddomain.local.baddomain.local/:    Fail 56
                   https://gooddomain.local.baddomain.local/:    Fail 35
          http://subdomain.gooddomain.local.baddomain.local/:    Fail 56
         https://subdomain.gooddomain.local.baddomain.local/:    Fail 35
                    http://baddomain.local/gooddomain.local/:    Fail 56
           http://gooddomain.local:password@baddomain.local/:    Fail 56
               http://user:gooddomain.local@baddomain.local/:    Fail 56
                     http://baddomain.local/gooddomain.local:    Fail 56
                    http://baddomain.local/gooddomain.local/:    Fail 56

Exact match, no subdomains:

^https?://gooddomain.local(/|$)

As regular expression

obsd-client# ./domainTest.sh
                                    http://gooddomain.local/: Success
                                   https://gooddomain.local/: Success
                                    http://gOoDdOmAiN.lOcAl/: Success
                                   https://gOoDdOmAiN.lOcAl/: Success
                                 http://gooddomain.local:80/: Success
                               https://gooddomain.local:443/: Success
                                 http://gOoDdOmAiN.lOcAl:80/: Success
                               https://gOoDdOmAiN.lOcAl:443/: Success
                          http://subdomain.gooddomain.local/:    Fail 56
                         https://subdomain.gooddomain.local/:    Fail 35
                                     http://baddomain.local/:    Fail 56
                                    https://baddomain.local/:    Fail 35
                    http://gooddomain.local.baddomain.local/:    Fail 56
                   https://gooddomain.local.baddomain.local/:    Fail 35
          http://subdomain.gooddomain.local.baddomain.local/:    Fail 56
         https://subdomain.gooddomain.local.baddomain.local/:    Fail 35
                    http://baddomain.local/gooddomain.local/:    Fail 56
           http://gooddomain.local:password@baddomain.local/:    Fail 56
               http://user:gooddomain.local@baddomain.local/:    Fail 56
                     http://baddomain.local/gooddomain.local:    Fail 56
                    http://baddomain.local/gooddomain.local/:    Fail 56

Unless somebody else wants to suggest any other URLs to try, it looks like we have a winner.

[Bob_Zimmerman]

Important notes:

HTTPS requests appear to get normalized as https://<domain> with the scheme, but no trailing slash. HTTP requests keep their trailing slash. Thus, to ensure I'm matching to the end of the domain name for both HTTP and HTTPS, I had to add "(/|$)" to the end of the expressions.

Some other results I got along the way confirm either the matching engine is case-insensitive or the input is normalized to lowercase. At one point, I made a request for "http://baddomain.local/gOoDdOmAiN.lOcAl/", and it was allowed by an expression which was entirely lowercase. This isn't strictly correct (paths in URLs are case-sensitive), but I don't see any way it could really present a problem. If there are things you need to access at a domain and things you must not be able to access on the same domain, and the paths to these resources differ only by case, you have much bigger issues.

The "(/|$)" tail to these expressions triggers the "URLs containing characters that may affect performance" warning.

[PhoneBoy]

Great work showing how this works!

[Bob_Zimmerman]

With HTTPS Inspection enabled, HTTPS URLs get their terminal slash after the domain name!

gooddomain.com/

Not as a regular expression

obsd-client# ./domainTest.sh  
                                    http://gooddomain.local/: Success
                                   https://gooddomain.local/: Success
                                    http://gOoDdOmAiN.lOcAl/: Success
                                   https://gOoDdOmAiN.lOcAl/: Success
                                 http://gooddomain.local:80/: Success
                               https://gooddomain.local:443/: Success
                                 http://gOoDdOmAiN.lOcAl:80/: Success
                               https://gOoDdOmAiN.lOcAl:443/: Success
                          http://subdomain.gooddomain.local/:    Fail 56
                         https://subdomain.gooddomain.local/:    Fail 52
                                     http://baddomain.local/:    Fail 56
                                    https://baddomain.local/:    Fail 52
                    http://gooddomain.local.baddomain.local/:    Fail 56
                   https://gooddomain.local.baddomain.local/:    Fail 52
          http://subdomain.gooddomain.local.baddomain.local/:    Fail 56
         https://subdomain.gooddomain.local.baddomain.local/:    Fail 52
                    http://baddomain.local/gooddomain.local/:    Fail 56
           http://gooddomain.local:password@baddomain.local/:    Fail 56
               http://user:gooddomain.local@baddomain.local/:    Fail 56
                     http://baddomain.local/gooddomain.local:    Fail 56
                    http://baddomain.local/gooddomain.local/:    Fail 56

HTTPS Inspection also avoids the need for the "(/|$)" and lets us end the regular expressions with a slash.

This makes a sort of sense to me, since the path of an HTTPS request is encrypted, and the slash after the domain name is arguably part of the path. The firewall is clearly reconstructing the URL, though, since it adds a scheme to HTTPS requests. If HTTPS Inspection is disabled, it should probably add the terminal slash at the same time.

Next, I think I need to test HTTP and HTTPS on ports other than 80 and 443 respectively to see if those cause the port numbers to be included in the reconstructed URL.

[_Val_]

Nice one!