topic Re: Face recognition with R81 in General Topics

Face recognition with R81

HeikoAnkenbrand — Thu, 10 Dec 2020 08:04:58 GMT

Face recognition is not only a magic of iPhones and Android. Also with Check Point Firewall's you can do this in R81.

For this purpose I have created a software in the last weeks, which implements this face recognition function with R81. For this I used the new "sk167210: Generic Data Center feature" in R81.

The Generic Data Center feature provides the ability to enforce access to/from IP addresses defined in JSON files located in external web servers on the Security Management machine. The „Generic Data Center Objects“ are updated automatically on the Security Gateway each time the JSON file change. There is no need to install policy for the updates to take effect.

Objects created based on these files can be used as a source or a destination in the access control policy.

How does it work:

I have developed a software with OpenCV that recognises faces. When a face is detected, the IP of the detected user is written to a JSON file „face_detect.txt“. If the user is not recognised for more than 5 seconds, a dummy IP is written to this JSON file. In my example the face detection software recognised myself and the IP 10.10.52.181 of my laptop is insert in the JSON file.

User „Heiko Ankenbrand“ was recognised:	No user was recognised:

Here is an example of the JSON file „face_detect.txt“ that is created when a user is recognised.

This file is provided via a web server (nginx) on my laptop so that the Check Point SMS can read this file as "Generic Data Center Object" from the web server.

On Check Point site a "Generic Data Center Object" is created in the Smart Console. This object fetches the JSON file „face_detect.txt“ every second from the web server from my face recognition software.

What we need now for example, is a firewall rule that allows access to the Internet. The „Generic Data Center Object“ is used as source here.

This means, if the user face has been recognised, the IP of the user is added here via the "Generic Data Center Object“. If the user is not recognised via the face recognition a "dummy IP" is inserted here.

Therefore, the rule can be controlled almost in real time via face recognition. The „Generic Data Center Object“ provides the ability to enforce access to the IP address defined in JSON files located in external web servers on the Security Management machine. The „Generic Data Center Object“ is updated automatically on the Security Gateway each time the JSON file change via the face recognition. There is no need to install policy for the updates to take effect.

I will provide an improved version of the face recognition software on GIT in the next weeks.

Re: Face recognition with R81

Wolfgang — Thu, 10 Dec 2020 06:59:01 GMT

@HeikoAnkenbrand

cool 👍

Re: Face recognition with R81

_Val_ — Thu, 10 Dec 2020 08:41:46 GMT

Wow! Nice!!!

Re: Face recognition with R81

_Val_ — Thu, 10 Dec 2020 08:42:48 GMT

what do you use for the actual face recongnition?

Re: Face recognition with R81

HeikoAnkenbrand — Thu, 10 Dec 2020 09:27:18 GMT

Im use Python + OpenCV (for face recongnition)

Install:

python
pip install opencv-python
pip install opencv-contrib-python

I will publish my tool on GIT. But at the moment it is still in early alpha status.

Re: Face recognition with R81

HristoGrigorov — Thu, 10 Dec 2020 09:41:55 GMT

I see great potential here. Like receive an alert when boss is approaching my office 😄

Re: Face recognition with R81

_Val_ — Thu, 10 Dec 2020 11:25:57 GMT

Great one!

Re: Face recognition with R81

Gateway — Thu, 10 Dec 2020 11:51:53 GMT

I need this for our office. Now you can recognise the good guys from the bad guys 🙂

Re: Face recognition with R81

Magnus-Holmberg — Thu, 10 Dec 2020 12:01:17 GMT

awesome 😄 and abit scary!

Re: Face recognition with R81

Danny — Thu, 10 Dec 2020 13:00:21 GMT

@Danny is holding a printout of @HeikoAnkenbrand 's face into the camera and gets full admin access to his firewall. 🙂

Re: Face recognition with R81

HeikoAnkenbrand — Thu, 10 Dec 2020 15:06:01 GMT

oops, full admin access! 😀

Re: Face recognition with R81

masher — Thu, 10 Dec 2020 15:19:22 GMT

This is certainly a different approach to Identity Awareness. 😂

Re: Face recognition with R81

Alex- — Thu, 10 Dec 2020 15:45:51 GMT

This is absolutely impressive.

Re: Face recognition with R81

Eliot — Thu, 10 Dec 2020 16:18:49 GMT

Is a great idea. You should sell that as a product.

We have a central access control into our data center at our company. Here you could immediately enable the service ports via firewall rule.

👍

Re: Face recognition with R81

JozkoMrkvicka — Thu, 10 Dec 2020 16:21:37 GMT

next level - recognize based on my face if I am not too tired/bored/drunk 😄 Only after that I am able to open smartConsole or CLI 😄

Re: Face recognition with R81

HeikoAnkenbrand — Sat, 12 Dec 2020 09:47:37 GMT

Start Smart Console via face recognition and laugh mode.

Hi @JozkoMrkvicka,

Next level 😂. Now it gets even crazier.

Laugh mode in face recognition to start the SmartConsole automatically.

The idea of starting the SmartConsole automatically was easy to programme.

I have added the following to the face recognition tool:
- Laugh mode in face recognition: Now the Smart Console opens automatically.
- Automatic authentication at the SmartCenter (more read here: R80.20 - Portable SmartConsole + Tips and Tricks)
- Now I send the SmartConsole the login file with the parameter "-p"
("C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml)
The SmartConsoleP.xml file contains the login credentials(SMS IP, User, PW).

This is the result:

(view in My Videos)

Re: Face recognition with R81

Phil_Atkinson — Thu, 10 Dec 2020 23:39:10 GMT

wow...

Re: Face recognition with R81

HeikoAnkenbrand — Sat, 12 Dec 2020 09:50:01 GMT

That's actually all you need to programme in python for laugh recognition:-)

smile_cascade = cv2.CascadeClassifier('cascades/data/haarcascade_smile.xml') ret, frame = cap.read() gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) for (x, y, w, h) in faces: roi_gray = gray[y:y+h, x:x+w] ... subitems = smile_cascade.detectMultiScale(roi_gray, 1.4, 20) sub_1 = str(subitems).strip('[]') if sub_1 != "()": smile = 1 for (ex,ey,ew,eh) in subitems: cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(255,255,0),2) ... if smile == 1: os.system('"D:\\sc.bat"'); font = cv2.FONT_HERSHEY_SIMPLEX color = (255, 255, 255) stroke = 3 cv2.putText(frame, "Smile: Start SmartConsole", (1,90), font, 0.7, color, 1, cv2.LINE_AA) ...

With the sc.bat file I start the SmartConsole:

"C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml

And the file SmartConsoleP.xml for the SmartConsole login:

<?xml version="1.0" encoding="utf-8"?> <RemoteLaunchParemeters xmlns:xsi="http:confused_face:/www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http:confused_face:/www.w3.org/2001/XMLSchema"> <Username>admin</Username> <ServerIP>x.y.z.w</ServerIP> <DomainName></DomainName> <ReadOnly>False</ReadOnly> <CloudDemoMode>False</CloudDemoMode> <Password>this is secret</Password> </RemoteLaunchParemeters>

Re: Face recognition with R81

Reinhard_Stich — Fri, 11 Dec 2020 07:17:05 GMT

cool demo ! I love it.

but I also see real world usecases where you only allow for example access to sensitive data as long as a user is recognized by the cam, while access is immediately disabled if the user is not looking at the screen. of course this depends very much on the application...

Re: Face recognition with R81

HeikoAnkenbrand — Fri, 11 Dec 2020 08:13:22 GMT

Hi @Reinhard_Stich,

I have been programming my home automation for many years and use face recognition as an additional function (for example: door opener, separate web console profiles for each family member in the home automation,...). I don't use Siri or Alexa to recognise speech in my home automation. From my point of view, this is a security issue. I use a speech recognition system that does not access the internet.

Hence the idea of connecting this with Check Point firewall.

I would not seriously use this tool for customers and I think there is no real world use case.

I just wanted to show the potential behind R81 "Generic Data Center Objects" and it was an interesting small project. Safety must also be considered here. The security of the web server must also be considered for "sk167210: Generic Data Center feature" objects. If I can hack the web server, I can manipulate the "Generic Data Center Objects" objects and so I can manipulate the firewall access. In my view, this is also an important safety factor. Here I am waiting for the web application firewall that Check Point has been announcing for years at CPX in Barcelona.

But all in all, it was an interesting project for me.

PS:
Now a little more fun. For christmas I will add a Father Christmas recognition and see if the Father Christmas laughs when he brings the packages or if he is stressed. 😂🤣😂