Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
HeikoAnkenbrand
Champion
Champion

Face recognition with R81

Face recognition is not only a magic of iPhones and Android. Also with Check Point Firewall's you can do this in R81.

For this purpose I have created a software in the last weeks, which implements this face recognition function with R81. For this I used the new "sk167210: Generic Data Center feature" in R81.

The Generic Data Center feature provides the ability to enforce access to/from IP addresses defined in JSON files located in external web servers on the Security Management machine. The „Generic Data Center Objects“ are updated automatically on the Security Gateway each time the JSON file change. There is no need to install policy for the updates to take effect.

Objects created based on these files can be used as a source or a destination in the access control policy.

How does it work:

I have developed a software with OpenCV that recognises faces. When a face is detected, the IP of the detected user is written to a JSON file „face_detect.txt“. If the user is not recognised for more than 5 seconds, a dummy IP is written to this JSON file. In my example the face detection software recognised myself and the IP 10.10.52.181 of my laptop is insert in the JSON file.

User „Heiko Ankenbrand“ was recognised:             No user was recognised:
F1.jpg f2.png


Here is an example of the JSON file „face_detect.txt“ that is created when a user is recognised.

f3.png

This file is provided via a web server (nginx) on my laptop so that the Check Point SMS can read this file as "Generic Data Center Object" from the web server.

On Check Point site a "Generic Data Center Object" is created in the Smart Console. This object fetches the JSON file „face_detect.txt“ every second from the web server from my face recognition software.

f4.JPG

f5.JPG

What we need now for example, is a firewall rule that allows access to the Internet. The „Generic Data Center Object“ is used as source here.

f6.JPG

This means, if the user face has been recognised, the IP of the user is added here via the "Generic Data Center Object“. If the user is not recognised via the face recognition a "dummy IP" is inserted here.

Therefore, the rule can be controlled almost in real time via face recognition. The „Generic Data Center Object“ provides the ability to enforce access to the IP address defined in JSON files located in external web servers on the Security Management machine. The „Generic Data Center Object“ is updated automatically on the Security Gateway each time the JSON file change via the face recognition. There is no need to install policy for the updates to take effect.

I will provide an improved version of the face recognition software on GIT in the next weeks.

(2)
38 Replies
Wolfgang
Leader
Leader

@HeikoAnkenbrand 

cool 👍

_Val_
Admin
Admin

Wow! Nice!!!

_Val_
Admin
Admin

what do you use for the actual face recongnition?

0 Kudos
HeikoAnkenbrand
Champion
Champion

Im use Python + OpenCV (for face recongnition)

Install:

python
pip install opencv-python
pip install opencv-contrib-python

I will publish my tool on GIT. But at the moment it is still in early alpha status.

 

_Val_
Admin
Admin

Great one!

I see great potential here. Like receive an alert when boss is approaching my office 😄

Gateway
Participant

I need this for our office. Now you can recognise the good guys from the bad guys 🙂

Magnus-Holmberg
Advisor

awesome 😄 and abit scary!

https://www.youtube.com/c/MagnusHolmberg-NetSec
0 Kudos
Danny
Champion
Champion

@Danny  is holding a printout of @HeikoAnkenbrand 's face into the camera and gets full admin access to his firewall. 🙂

0 Kudos
HeikoAnkenbrand
Champion
Champion

oops, full admin access! 😀

0 Kudos
Vladimir
Champion
Champion

Combine face recognition with the app that generates daily QR code, so that printout of your face will not be sufficient to unlock the rule.

0 Kudos
masher
Employee
Employee

This is certainly a different approach to Identity Awareness. 😂 

Alex_Gilis
Advisor

This is absolutely impressive.

0 Kudos
Eliot
Explorer

Is a great idea. You should sell that as a product.

We have a central access control into our data center at our company. Here you could immediately enable the service ports via firewall rule.

👍

0 Kudos
JozkoMrkvicka
Leader
Leader

next level - recognize based on my face if I am not too tired/bored/drunk 😄 Only after that I am able to open smartConsole or CLI 😄 

Kind regards,
Jozko Mrkvicka
HeikoAnkenbrand
Champion
Champion

Start Smart Console via face recognition and laugh mode.

Hi @JozkoMrkvicka,

Next level 😂. Now it gets even crazier.

Laugh mode in face recognition to start the SmartConsole automatically.

The idea of starting the SmartConsole automatically was easy to programme.

I have added the following to the face recognition tool:
- Laugh mode in face recognition: Now the Smart Console opens automatically. 
- Automatic authentication at the SmartCenter (more read here: R80.20 - Portable SmartConsole + Tips and Tricks)
- Now I send the SmartConsole the login file with the parameter "-p"
   ("C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml)
  The SmartConsoleP.xml file contains the login credentials(SMS IP, User, PW).

This is the result:

 

(2)
HeikoAnkenbrand
Champion
Champion

That's actually all you need to programme in python for laugh recognition:-)

 

smile_cascade = cv2.CascadeClassifier('cascades/data/haarcascade_smile.xml')
ret, frame = cap.read()
gray  = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
for (x, y, w, h) in faces:
    	roi_gray = gray[y:y+h, x:x+w]
...

    	subitems = smile_cascade.detectMultiScale(roi_gray, 1.4, 20)
    	sub_1 = str(subitems).strip('[]')
    	if sub_1 != "()":
    	  smile = 1
    	for (ex,ey,ew,eh) in subitems:
    		cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(255,255,0),2)

...
    if smile == 1:
        os.system('"D:\\sc.bat"');
        font = cv2.FONT_HERSHEY_SIMPLEX
    	color = (255, 255, 255)
    	stroke = 3
    	cv2.putText(frame, "Smile:    Start SmartConsole", (1,90), font, 0.7, color, 1, cv2.LINE_AA)

... 

 

With the sc.bat file I start the SmartConsole:

 

"C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml

 

And the file SmartConsoleP.xml for the SmartConsole login:

 

<?xml version="1.0" encoding="utf-8"?>
	<RemoteLaunchParemeters xmlns:xsi="http:confused_face:/www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http:confused_face:/www.w3.org/2001/XMLSchema">
		<Username>admin</Username>
		<ServerIP>x.y.z.w</ServerIP>
		<DomainName></DomainName>
		<ReadOnly>False</ReadOnly>
		<CloudDemoMode>False</CloudDemoMode>
		<Password>this is secret</Password>
	</RemoteLaunchParemeters>

 

JozkoMrkvicka
Leader
Leader

Hat off @HeikoAnkenbrand . I am speechless 😮 

Kind regards,
Jozko Mrkvicka
Power_Support
Participant

wow, nice solution 

0 Kudos
Tamar
Participant

nice

charlie_h
Participant

Crazy solution:-)

0 Kudos
Phil_Atkinson
Ambassador
Ambassador

wow...

Reinhard_Stich
Contributor

cool demo ! I love it.

but I also see real world usecases where you only allow for example access to sensitive data as long as a user is recognized by the cam, while access is immediately disabled if the user is not looking at the screen. of course this depends very much on the application...

0 Kudos
HeikoAnkenbrand
Champion
Champion

Hi @Reinhard_Stich,

I have been programming my home automation for many years and use face recognition as an additional function (for example: door opener, separate web console profiles for each family member in the home automation,...). I don't use Siri or Alexa to recognise speech in my home automation. From my point of view, this is a security issue. I use a speech recognition system that does not access the internet.

Hence the idea of connecting this with Check Point firewall.

I would not seriously use this tool for customers and I think there is no real world use case.

I just wanted to show the potential behind R81 "Generic Data Center Objects" and it was an interesting small project. Safety must also be considered here. The security of the web server must also be considered for "sk167210: Generic Data Center feature" objects. If I can hack the web server, I can manipulate the "Generic Data Center Objects" objects and so I can manipulate the firewall access. In my view, this is also an important safety factor. Here I am waiting for the web application firewall that Check Point has been announcing for years at CPX in Barcelona.

But all in all, it was an interesting project for me.

PS:
Now a little more fun. For christmas I will add a Father Christmas recognition and see if the Father Christmas laughs when he brings the packages or if he is stressed. 😂🤣😂

 

 

G9
Explorer

WOW

A very impressive idea.

0 Kudos
FraP
Contributor

Did you already implement the "voice provisioning tool" for your speech recognition system? 😃
Probably you'll able to speak with your firewall in the next few days...

0 Kudos
JozkoMrkvicka
Leader
Leader

Just out of curious ... Would it be possible to simply say loud "Hey Firewall, show logs from source x.y.z.x" and SmartView (or SmartConsole) will be opened and will show me exact logs according my voice instructions ?

Kind regards,
Jozko Mrkvicka
0 Kudos
HeikoAnkenbrand
Champion
Champion

Start Smart Console via Voice.

Hi @JozkoMrkvicka,

Now it's getting even crazier:

Not quite your requirement, but you can use it to start the SmartConsole by voice. Here is the code with python for speech recognition. You still need to adjust your micro. You can find the USB ID in the Windows device manager. I have used Google for speech recognition here. You can also use local speech recognition.

Maybe it works even easier with "Alexa" 🤣😂😄

Install python 3.9.1 + PyAudio + SpeechRecognition

 

pip install PyAudio
pip install SpeechRecognition

 

Now the programme code for text recognition.
- You just have to say "start smartconsole".
- The USB micro still needs to be adapted to your system.

 

import speech_recognition as sr
import os;
  
#enter the name of usb microphone that you found 
mic_name = "USB Device 0xaaa:0xbbb: Audio (hw:1, 0)"

sample_rate = 48000
chunk_size = 2048 
r = sr.Recognizer() 
mic_list = sr.Microphone.list_microphone_names() 
for i, microphone_name in enumerate(mic_list): 
    if microphone_name == mic_name: 
        device_id = i 

    r.adjust_for_ambient_noise(source) 
    print "Say "
    audio = r.listen(source) 
          
    try: 
        text = r.recognize_google(audio) 
        print "you said to your R81 firewall: " + text 
	  if text == "start smartconsole":
            os.system('"D:\\sc.bat"');
             
    except sr.UnknownValueError: 
        print("Google Speech Recognition no R81 command") 
      
    except sr.RequestError as e: 
        print("Could not request results from Google Speech Recognition service; {0}".format(e)) 

 

With the sc.bat file I start the SmartConsole:

 

"C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml

 

And the file SmartConsoleP.xml for the SmartConsole login:
Change the following parameters:
- Username
- Password
- ServerIP

 

<?xml version="1.0" encoding="utf-8"?>
	<RemoteLaunchParemeters xmlns:xsi="http:confused_face:/www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http:confused_face:/www.w3.org/2001/XMLSchema">
		<Username>admin</Username>
		<ServerIP>x.y.z.w</ServerIP>
		<DomainName></DomainName>
		<ReadOnly>False</ReadOnly>
		<CloudDemoMode>False</CloudDemoMode>
		<Password>this is secret</Password>
	</RemoteLaunchParemeters>

 

JozkoMrkvicka
Leader
Leader

Starting from API 1.6.1 (R80.40 JHF 78), there is posibility to call API command show-logs

Just a hint that searching for logs is possible also via API 😉 Running mgmt_cli should be possible using mgmt_cli.exe from Windows, or ?

Kind regards,
Jozko Mrkvicka
0 Kudos