Face recognition with R81

[HeikoAnkenbrand]

Face recognition is not only a magic of iPhones and Android. Also with Check Point Firewall's you can do this in R81.

Easy Face Recognition

Video Player is loading.

Play Video

Play

Mute

Current Time 0:00

/

Duration 0:00

Loaded: 0%

0:00

Stream Type LIVE

Seek to live, currently behind liveLIVE

Remaining Time -0:00

1x

Playback Rate

Chapters

• Chapters

Descriptions

• descriptions off, selected

Captions

• captions settings, opens captions settings dialog
• captions off, selected

Audio Track

• en (Main), selected

Picture-in-PictureFullscreen

This is a modal window.

Beginning of dialog window. Escape will cancel and close the window.

Text

ColorWhiteBlackRedGreenBlueYellowMagentaCyanTransparencyOpaqueSemi-Transparent

Background

ColorBlackWhiteRedGreenBlueYellowMagentaCyanTransparencyOpaqueSemi-TransparentTransparent

Window

ColorBlackWhiteRedGreenBlueYellowMagentaCyanTransparencyTransparentSemi-TransparentOpaque

Font Size

50%75%100%125%150%175%200%300%400%

Text Edge Style

NoneRaisedDepressedUniformDropshadow

Font Family

Proportional Sans-SerifMonospace Sans-SerifProportional SerifMonospace SerifCasualScriptSmall Caps

Reset restore all settings to the default valuesDone

Close Modal Dialog

End of dialog window.

Close Modal Dialog

This is a modal window. This modal can be closed by pressing the Escape key or activating the close button.

For this purpose I have created a software in the last weeks, which implements this face recognition function with R81. For this I used the new "sk167210: Generic Data Center feature" in R81.

The Generic Data Center feature provides the ability to enforce access to/from IP addresses defined in JSON files located in external web servers on the Security Management machine. The „Generic Data Center Objects“ are updated automatically on the Security Gateway each time the JSON file change. There is no need to install policy for the updates to take effect.

Objects created based on these files can be used as a source or a destination in the access control policy.

How does it work:

I have developed a software with OpenCV that recognises faces. When a face is detected, the IP of the detected user is written to a JSON file „face_detect.txt“. If the user is not recognised for more than 5 seconds, a dummy IP is written to this JSON file. In my example the face detection software recognised myself and the IP 10.10.52.181 of my laptop is insert in the JSON file.

User „Heiko Ankenbrand“ was recognised:	No user was recognised:

Here is an example of the JSON file „face_detect.txt“ that is created when a user is recognised.

This file is provided via a web server (nginx) on my laptop so that the Check Point SMS can read this file as "Generic Data Center Object" from the web server.

On Check Point site a "Generic Data Center Object" is created in the Smart Console. This object fetches the JSON file „face_detect.txt“ every second from the web server from my face recognition software.

What we need now for example, is a firewall rule that allows access to the Internet. The „Generic Data Center Object“ is used as source here.

This means, if the user face has been recognised, the IP of the user is added here via the "Generic Data Center Object“. If the user is not recognised via the face recognition a "dummy IP" is inserted here.

Therefore, the rule can be controlled almost in real time via face recognition. The „Generic Data Center Object“ provides the ability to enforce access to the IP address defined in JSON files located in external web servers on the Security Management machine. The „Generic Data Center Object“ is updated automatically on the Security Gateway each time the JSON file change via the face recognition. There is no need to install policy for the updates to take effect.

I will provide an improved version of the face recognition software on GIT in the next weeks.

[Wolfgang]

@HeikoAnkenbrand 

cool 👍

[_Val_]

Wow! Nice!!!

[_Val_]

what do you use for the actual face recongnition?

[HeikoAnkenbrand]

Im use Python + OpenCV (for face recongnition)

Install:

python
pip install opencv-python
pip install opencv-contrib-python

I will publish my tool on GIT. But at the moment it is still in early alpha status.

[_Val_]

Great one!

[HristoGrigorov]

I see great potential here. Like receive an alert when boss is approaching my office 😄

[Gateway]

I need this for our office. Now you can recognise the good guys from the bad guys 🙂

[Magnus-Holmberg]

awesome 😄 and abit scary!

[Danny]

@Danny  is holding a printout of @HeikoAnkenbrand 's face into the camera and gets full admin access to his firewall. 🙂

[HeikoAnkenbrand]

oops, full admin access! 😀

[Vladimir]

Combine face recognition with the app that generates daily QR code, so that printout of your face will not be sufficient to unlock the rule.

[masher]

This is certainly a different approach to Identity Awareness. 😂 

[Alex-]

This is absolutely impressive.

[Eliot]

Is a great idea. You should sell that as a product.

We have a central access control into our data center at our company. Here you could immediately enable the service ports via firewall rule.

👍

[JozkoMrkvicka]

next level - recognize based on my face if I am not too tired/bored/drunk 😄 Only after that I am able to open smartConsole or CLI 😄 

[HeikoAnkenbrand]

Start Smart Console via face recognition and laugh mode.

Hi @JozkoMrkvicka,

Next level 😂. Now it gets even crazier.

Laugh mode in face recognition to start the SmartConsole automatically.

The idea of starting the SmartConsole automatically was easy to programme.

I have added the following to the face recognition tool:
- Laugh mode in face recognition: Now the Smart Console opens automatically. 
- Automatic authentication at the SmartCenter (more read here: R80.20 - Portable SmartConsole + Tips and Tricks)
- Now I send the SmartConsole the login file with the parameter "-p"
   ("C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml)
  The SmartConsoleP.xml file contains the login credentials(SMS IP, User, PW).

This is the result:

Laugh mode in face recognition to start the SmartConsole automatically.

Video Player is loading.

Play Video

Play

Mute

Current Time 0:00

/

Duration 0:00

Loaded: 0%

0:00

Stream Type LIVE

Seek to live, currently behind liveLIVE

Remaining Time -0:00

1x

Playback Rate

Chapters

• Chapters

Descriptions

• descriptions off, selected

Captions

• captions settings, opens captions settings dialog
• captions off, selected

Audio Track

Picture-in-PictureFullscreen

This is a modal window.

Beginning of dialog window. Escape will cancel and close the window.

Text

ColorWhiteBlackRedGreenBlueYellowMagentaCyanTransparencyOpaqueSemi-Transparent

Background

ColorBlackWhiteRedGreenBlueYellowMagentaCyanTransparencyOpaqueSemi-TransparentTransparent

Window

ColorBlackWhiteRedGreenBlueYellowMagentaCyanTransparencyTransparentSemi-TransparentOpaque

Font Size

50%75%100%125%150%175%200%300%400%

Text Edge Style

NoneRaisedDepressedUniformDropshadow

Font Family

Proportional Sans-SerifMonospace Sans-SerifProportional SerifMonospace SerifCasualScriptSmall Caps

Reset restore all settings to the default valuesDone

Close Modal Dialog

End of dialog window.

Close Modal Dialog

This is a modal window. This modal can be closed by pressing the Escape key or activating the close button.

[HeikoAnkenbrand]

That's actually all you need to programme in python for laugh recognition:-)

smile_cascade = cv2.CascadeClassifier('cascades/data/haarcascade_smile.xml')
ret, frame = cap.read()
gray  = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
for (x, y, w, h) in faces:
    	roi_gray = gray[y:y+h, x:x+w]
...

    	subitems = smile_cascade.detectMultiScale(roi_gray, 1.4, 20)
    	sub_1 = str(subitems).strip('[]')
    	if sub_1 != "()":
    	  smile = 1
    	for (ex,ey,ew,eh) in subitems:
    		cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(255,255,0),2)

...
    if smile == 1:
        os.system('"D:\\sc.bat"');
        font = cv2.FONT_HERSHEY_SIMPLEX
    	color = (255, 255, 255)
    	stroke = 3
    	cv2.putText(frame, "Smile:    Start SmartConsole", (1,90), font, 0.7, color, 1, cv2.LINE_AA)

... 

With the sc.bat file I start the SmartConsole:

"C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml

And the file SmartConsoleP.xml for the SmartConsole login:

<?xml version="1.0" encoding="utf-8"?>
	<RemoteLaunchParemeters xmlns:xsi="http:confused_face:/www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http:confused_face:/www.w3.org/2001/XMLSchema">
		<Username>admin</Username>
		<ServerIP>x.y.z.w</ServerIP>
		<DomainName></DomainName>
		<ReadOnly>False</ReadOnly>
		<CloudDemoMode>False</CloudDemoMode>
		<Password>this is secret</Password>
	</RemoteLaunchParemeters>

[JozkoMrkvicka]

Hat off @HeikoAnkenbrand . I am speechless 😮 

[Power_Support]

wow, nice solution 

[Tamar]

nice

[charlie_h]

Crazy solution:-)

[Ralf_Schirmer]

Crazy idea, but very impressive,
Congratulations for the " Toolbox Author of the Year ".

[Phil_Atkinson]

wow...

[Reinhard_Stich]

cool demo ! I love it.

but I also see real world usecases where you only allow for example access to sensitive data as long as a user is recognized by the cam, while access is immediately disabled if the user is not looking at the screen. of course this depends very much on the application...

[HeikoAnkenbrand]

Hi @Reinhard_Stich,

I have been programming my home automation for many years and use face recognition as an additional function (for example: door opener, separate web console profiles for each family member in the home automation,...). I don't use Siri or Alexa to recognise speech in my home automation. From my point of view, this is a security issue. I use a speech recognition system that does not access the internet.

Hence the idea of connecting this with Check Point firewall.

I would not seriously use this tool for customers and I think there is no real world use case.

I just wanted to show the potential behind R81 "Generic Data Center Objects" and it was an interesting small project. Safety must also be considered here. The security of the web server must also be considered for "sk167210: Generic Data Center feature" objects. If I can hack the web server, I can manipulate the "Generic Data Center Objects" objects and so I can manipulate the firewall access. In my view, this is also an important safety factor. Here I am waiting for the web application firewall that Check Point has been announcing for years at CPX in Barcelona.

But all in all, it was an interesting project for me.

PS:
Now a little more fun. For christmas I will add a Father Christmas recognition and see if the Father Christmas laughs when he brings the packages or if he is stressed. 😂🤣😂

[G9]

WOW

A very impressive idea.

[FraP]

Did you already implement the "voice provisioning tool" for your speech recognition system? 😃
Probably you'll able to speak with your firewall in the next few days...

[JozkoMrkvicka]

Just out of curious ... Would it be possible to simply say loud "Hey Firewall, show logs from source x.y.z.x" and SmartView (or SmartConsole) will be opened and will show me exact logs according my voice instructions ?

[HeikoAnkenbrand]

Start Smart Console via Voice.

Hi @JozkoMrkvicka,

Now it's getting even crazier:

Not quite your requirement, but you can use it to start the SmartConsole by voice. Here is the code with python for speech recognition. You still need to adjust your micro. You can find the USB ID in the Windows device manager. I have used Google for speech recognition here. You can also use local speech recognition.

Maybe it works even easier with "Alexa" 🤣😂😄. 

Install python 3.9.1 + PyAudio + SpeechRecognition

pip install PyAudio
pip install SpeechRecognition

Now the programme code for text recognition.
- You just have to say "start smartconsole".
- The USB micro still needs to be adapted to your system.

import speech_recognition as sr
import os;
  
#enter the name of usb microphone that you found 
mic_name = "USB Device 0xaaa:0xbbb: Audio (hw:1, 0)"

sample_rate = 48000
chunk_size = 2048 
r = sr.Recognizer() 
mic_list = sr.Microphone.list_microphone_names() 
for i, microphone_name in enumerate(mic_list): 
    if microphone_name == mic_name: 
        device_id = i 

    r.adjust_for_ambient_noise(source) 
    print "Say "
    audio = r.listen(source) 
          
    try: 
        text = r.recognize_google(audio) 
        print "you said to your R81 firewall: " + text 
	  if text == "start smartconsole":
            os.system('"D:\\sc.bat"');
             
    except sr.UnknownValueError: 
        print("Google Speech Recognition no R81 command") 
      
    except sr.RequestError as e: 
        print("Could not request results from Google Speech Recognition service; {0}".format(e)) 

With the sc.bat file I start the SmartConsole:

"C:\Program Files (x86)\CheckPoint\SmartConsole\R80.40\PROGRAM\SmartConsole.exe" -p D:\SmartConsoleP.xml

And the file SmartConsoleP.xml for the SmartConsole login:
Change the following parameters:
- Username
- Password
- ServerIP

<?xml version="1.0" encoding="utf-8"?>
	<RemoteLaunchParemeters xmlns:xsi="http:confused_face:/www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http:confused_face:/www.w3.org/2001/XMLSchema">
		<Username>admin</Username>
		<ServerIP>x.y.z.w</ServerIP>
		<DomainName></DomainName>
		<ReadOnly>False</ReadOnly>
		<CloudDemoMode>False</CloudDemoMode>
		<Password>this is secret</Password>
	</RemoteLaunchParemeters>