Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
HeikoAnkenbrand
Champion Champion
Champion

Face recognition with R81

Face recognition is not only a magic of iPhones and Android. Also with Check Point Firewall's you can do this in R81.

For this purpose I have created a software in the last weeks, which implements this face recognition function with R81. For this I used the new "sk167210: Generic Data Center feature" in R81.

The Generic Data Center feature provides the ability to enforce access to/from IP addresses defined in JSON files located in external web servers on the Security Management machine. The „Generic Data Center Objects“ are updated automatically on the Security Gateway each time the JSON file change. There is no need to install policy for the updates to take effect.

Objects created based on these files can be used as a source or a destination in the access control policy.

How does it work:

I have developed a software with OpenCV that recognises faces. When a face is detected, the IP of the detected user is written to a JSON file „face_detect.txt“. If the user is not recognised for more than 5 seconds, a dummy IP is written to this JSON file. In my example the face detection software recognised myself and the IP 10.10.52.181 of my laptop is insert in the JSON file.

User „Heiko Ankenbrand“ was recognised:             No user was recognised:
F1.jpg f2.png


Here is an example of the JSON file „face_detect.txt“ that is created when a user is recognised.

f3.png

This file is provided via a web server (nginx) on my laptop so that the Check Point SMS can read this file as "Generic Data Center Object" from the web server.

On Check Point site a "Generic Data Center Object" is created in the Smart Console. This object fetches the JSON file „face_detect.txt“ every second from the web server from my face recognition software.

f4.JPG

f5.JPG

What we need now for example, is a firewall rule that allows access to the Internet. The „Generic Data Center Object“ is used as source here.

f6.JPG

This means, if the user face has been recognised, the IP of the user is added here via the "Generic Data Center Object“. If the user is not recognised via the face recognition a "dummy IP" is inserted here.

Therefore, the rule can be controlled almost in real time via face recognition. The „Generic Data Center Object“ provides the ability to enforce access to the IP address defined in JSON files located in external web servers on the Security Management machine. The „Generic Data Center Object“ is updated automatically on the Security Gateway each time the JSON file change via the face recognition. There is no need to install policy for the updates to take effect.

I will provide an improved version of the face recognition software on GIT in the next weeks.

➜ CCSM Elite, CCME, CCTE
(2)
39 Replies
Wolfgang
Authority
Authority

@HeikoAnkenbrand 

cool 👍

_Val_
Admin
Admin

Wow! Nice!!!

_Val_
Admin
Admin

what do you use for the actual face recongnition?

0 Kudos
HeikoAnkenbrand
Champion Champion
Champion

Im use Python + OpenCV (for face recongnition)

Install:

python
pip install opencv-python
pip install opencv-contrib-python

I will publish my tool on GIT. But at the moment it is still in early alpha status.

 

➜ CCSM Elite, CCME, CCTE
_Val_
Admin
Admin

Great one!

HristoGrigorov

I see great potential here. Like receive an alert when boss is approaching my office 😄

Gateway
Participant

I need this for our office. Now you can recognise the good guys from the bad guys 🙂

Magnus-Holmberg
Advisor

awesome 😄 and abit scary!

https://www.youtube.com/c/MagnusHolmberg-NetSec
Danny
Champion Champion
Champion

@Danny  is holding a printout of @HeikoAnkenbrand 's face into the camera and gets full admin access to his firewall. 🙂

0 Kudos
HeikoAnkenbrand
Champion Champion
Champion

oops, full admin access! 😀

➜ CCSM Elite, CCME, CCTE
0 Kudos
Vladimir
Champion
Champion

Combine face recognition with the app that generates daily QR code, so that printout of your face will not be sufficient to unlock the rule.

0 Kudos
masher
Employee
Employee

This is certainly a different approach to Identity Awareness. 😂 

Alex-
Advisor
Advisor

This is absolutely impressive.

0 Kudos
Eliot
Explorer

Is a great idea. You should sell that as a product.

We have a central access control into our data center at our company. Here you could immediately enable the service ports via firewall rule.

👍

0 Kudos
JozkoMrkvicka
Mentor
Mentor

next level - recognize based on my face if I am not too tired/bored/drunk 😄 Only after that I am able to open smartConsole or CLI 😄 

Kind regards,
Jozko Mrkvicka
HeikoAnkenbrand
Champion Champion
Champion

Start Smart Console via face recognition and laugh mode.

Hi @JozkoMrkvicka,

Next level 😂. Now it gets even crazier.

Laugh mode in face recognition to start the SmartConsole automatically.

The idea of starting the SmartConsole automatically was easy to programme.

I have added the following to the face recognition tool:
- Laugh mode in face recognition: Now the Smart Console opens automatically. 
- Automatic authentication at the SmartCenter (more read here: R80.20 - Portable SmartConsole + Tips and Tricks)
- Now I send the SmartConsole the login file with the parameter "-p"
   ("C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml)
  The SmartConsoleP.xml file contains the login credentials(SMS IP, User, PW).

This is the result:

 

➜ CCSM Elite, CCME, CCTE
(3)
HeikoAnkenbrand
Champion Champion
Champion

That's actually all you need to programme in python for laugh recognition:-)

 

smile_cascade = cv2.CascadeClassifier('cascades/data/haarcascade_smile.xml')
ret, frame = cap.read()
gray  = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
for (x, y, w, h) in faces:
    	roi_gray = gray[y:y+h, x:x+w]
...

    	subitems = smile_cascade.detectMultiScale(roi_gray, 1.4, 20)
    	sub_1 = str(subitems).strip('[]')
    	if sub_1 != "()":
    	  smile = 1
    	for (ex,ey,ew,eh) in subitems:
    		cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(255,255,0),2)

...
    if smile == 1:
        os.system('"D:\\sc.bat"');
        font = cv2.FONT_HERSHEY_SIMPLEX
    	color = (255, 255, 255)
    	stroke = 3
    	cv2.putText(frame, "Smile:    Start SmartConsole", (1,90), font, 0.7, color, 1, cv2.LINE_AA)

... 

 

With the sc.bat file I start the SmartConsole:

 

"C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml

 

And the file SmartConsoleP.xml for the SmartConsole login:

 

<?xml version="1.0" encoding="utf-8"?>
	<RemoteLaunchParemeters xmlns:xsi="http:confused_face:/www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http:confused_face:/www.w3.org/2001/XMLSchema">
		<Username>admin</Username>
		<ServerIP>x.y.z.w</ServerIP>
		<DomainName></DomainName>
		<ReadOnly>False</ReadOnly>
		<CloudDemoMode>False</CloudDemoMode>
		<Password>this is secret</Password>
	</RemoteLaunchParemeters>

 

➜ CCSM Elite, CCME, CCTE
JozkoMrkvicka
Mentor
Mentor

Hat off @HeikoAnkenbrand . I am speechless 😮 

Kind regards,
Jozko Mrkvicka
Power_Support
Participant

wow, nice solution 

0 Kudos
Tamar
Participant

nice

charlie_h
Participant

Crazy solution:-)

0 Kudos
Ralf_Schirmer
Participant

Crazy idea, but very impressive,
Congratulations for the " Toolbox Author of the Year ".

0 Kudos
Phil_Atkinson
Employee Alumnus
Employee Alumnus

wow...

Reinhard_Stich
Contributor

cool demo ! I love it.

but I also see real world usecases where you only allow for example access to sensitive data as long as a user is recognized by the cam, while access is immediately disabled if the user is not looking at the screen. of course this depends very much on the application...

0 Kudos
HeikoAnkenbrand
Champion Champion
Champion

Hi @Reinhard_Stich,

I have been programming my home automation for many years and use face recognition as an additional function (for example: door opener, separate web console profiles for each family member in the home automation,...). I don't use Siri or Alexa to recognise speech in my home automation. From my point of view, this is a security issue. I use a speech recognition system that does not access the internet.

Hence the idea of connecting this with Check Point firewall.

I would not seriously use this tool for customers and I think there is no real world use case.

I just wanted to show the potential behind R81 "Generic Data Center Objects" and it was an interesting small project. Safety must also be considered here. The security of the web server must also be considered for "sk167210: Generic Data Center feature" objects. If I can hack the web server, I can manipulate the "Generic Data Center Objects" objects and so I can manipulate the firewall access. In my view, this is also an important safety factor. Here I am waiting for the web application firewall that Check Point has been announcing for years at CPX in Barcelona.

But all in all, it was an interesting project for me.

PS:
Now a little more fun. For christmas I will add a Father Christmas recognition and see if the Father Christmas laughs when he brings the packages or if he is stressed. 😂🤣😂

 

 

➜ CCSM Elite, CCME, CCTE
G9
Explorer

WOW

A very impressive idea.

0 Kudos
FraP
Contributor

Did you already implement the "voice provisioning tool" for your speech recognition system? 😃
Probably you'll able to speak with your firewall in the next few days...

0 Kudos
JozkoMrkvicka
Mentor
Mentor

Just out of curious ... Would it be possible to simply say loud "Hey Firewall, show logs from source x.y.z.x" and SmartView (or SmartConsole) will be opened and will show me exact logs according my voice instructions ?

Kind regards,
Jozko Mrkvicka
0 Kudos
HeikoAnkenbrand
Champion Champion
Champion

Start Smart Console via Voice.

Hi @JozkoMrkvicka,

Now it's getting even crazier:

Not quite your requirement, but you can use it to start the SmartConsole by voice. Here is the code with python for speech recognition. You still need to adjust your micro. You can find the USB ID in the Windows device manager. I have used Google for speech recognition here. You can also use local speech recognition.

Maybe it works even easier with "Alexa" 🤣😂😄

Install python 3.9.1 + PyAudio + SpeechRecognition

 

pip install PyAudio
pip install SpeechRecognition

 

Now the programme code for text recognition.
- You just have to say "start smartconsole".
- The USB micro still needs to be adapted to your system.

 

import speech_recognition as sr
import os;
  
#enter the name of usb microphone that you found 
mic_name = "USB Device 0xaaa:0xbbb: Audio (hw:1, 0)"

sample_rate = 48000
chunk_size = 2048 
r = sr.Recognizer() 
mic_list = sr.Microphone.list_microphone_names() 
for i, microphone_name in enumerate(mic_list): 
    if microphone_name == mic_name: 
        device_id = i 

    r.adjust_for_ambient_noise(source) 
    print "Say "
    audio = r.listen(source) 
          
    try: 
        text = r.recognize_google(audio) 
        print "you said to your R81 firewall: " + text 
	  if text == "start smartconsole":
            os.system('"D:\\sc.bat"');
             
    except sr.UnknownValueError: 
        print("Google Speech Recognition no R81 command") 
      
    except sr.RequestError as e: 
        print("Could not request results from Google Speech Recognition service; {0}".format(e)) 

 

With the sc.bat file I start the SmartConsole:

 

"C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml

 

And the file SmartConsoleP.xml for the SmartConsole login:
Change the following parameters:
- Username
- Password
- ServerIP

 

<?xml version="1.0" encoding="utf-8"?>
	<RemoteLaunchParemeters xmlns:xsi="http:confused_face:/www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http:confused_face:/www.w3.org/2001/XMLSchema">
		<Username>admin</Username>
		<ServerIP>x.y.z.w</ServerIP>
		<DomainName></DomainName>
		<ReadOnly>False</ReadOnly>
		<CloudDemoMode>False</CloudDemoMode>
		<Password>this is secret</Password>
	</RemoteLaunchParemeters>

 

➜ CCSM Elite, CCME, CCTE

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events