topic Re: Dynamic Block Lists for Check Point firewalls in API / CLI Discussion

Dynamic Block Lists for Check Point firewalls

Daniel_Husand — Tue, 07 Feb 2017 15:41:21 GMT

I have cooked together some further improvements on Check Points 'block TOR' scripts and built a small service around it. This is not an official Check Point function/product and is provided by me in my spare time.

At this moment the following blocklists are implemented:

OpenBL
Emerging Threats: Known Compromised Hosts
TOR exit nodes
BruteforceBlocker
Blocklist.de All
Talos
Dshield

The feeds are downloaded, sanity checked and then published on cpdbl.net for free. I am currently running all lists on two separate clusters without any noticeable performance hit. Of course ymmv so all feedback is appreciated. If you want to try it out go to: https://cpdbl.net

Screenshot of the interface:

Gateway details:

These scripts utilize the rate limiting policy in SecureXL. Therefore blocking is done in fastpath and should not impact performance noticably.

Connections from IPs listed in the activated blocklists are only blocked INBOUND. Outgoing communications are currently allowed. I have roadmapped a toggle for this.

VSX is not supported for now.

Workflow:

The server(cpdbl.net) downloads all the lists nightly and

Validates that all entries are valid IPs.
Baselines the lists, makes sure a list does not suddenly grow enormously.
Publishes the lists for the clients to download.

The client:

Downloads fresh lists every 12 hours
Times out entries in the block-table after 12 hours, hence if cpdbl.net is unavailable all entries will be removed at this time.
Validates that only entries containing numbers and "-" are read into the system. (to stop possible code injection)
Installs validated entries into blocking tables and waits for 12 hours before starting over again.

To monitor the blocked IP addresses:

R77.30:

In SmartView Tracker, search for "SecureXL message: Quota violation".

R80:

In SmartLog, search for "blade:Firewall Alert".

Re: Dynamic Block Lists for Check Point firewalls

Christian_Hurta — Thu, 20 Sep 2018 14:05:46 GMT

Is there a way to use this with a proxy or does it need to have direct access from the gateway? Talking about R77.30

CP Dynamic Block Lists is a killer, and I will be adding a customized internal url feed as well in addition to the existing ones on the script, and will modify the script to call that additional feed.

Re: Dynamic Block Lists for Check Point firewalls

Dilian_Chernev — Thu, 29 Nov 2018 15:37:06 GMT

Scripts need to be touched a bit to work on R80.20.

/opt/CPshrd-R80/bin/ path in all files should be replaced with /opt/CPshrd-R80.20/bin/

Re: Dynamic Block Lists for Check Point firewalls

Fire_Verse — Wed, 29 May 2019 06:00:03 GMT

This is now opendbl.net

Re: Dynamic Block Lists for Check Point firewalls

Kaspars_Zibarts — Wed, 29 May 2019 07:58:21 GMT

Absolutely awesome work Daniel! We love it! As we love dynamic objects 🙂

I "stole" your code (sorry!) and made VSX version that can be executed via crontab or manually if desired and all required protections listed in config file as follows

[Expert@vsx1-ext:0]# cat blacklist.conf
blocklistde-all
bruteforce
etknown
malwaredomain
sslblock
talos
tor-exit
zeustracker

It has hard coded directory set in /home/admin/dynamic_objects as all our dynamic objects are handled there but you can change it yourself of course. Not as pretty from UI point of view.

You will need to supply VS number when running script, i.e ./blacklist.sh 3

#!/bin/bash # VSX version of the opendbl tool https://opendbl.net/ # based on version R80-0.5 . /opt/CPshared/5.0/tmp/.CPprofile.sh source /etc/profile.d/vsenv.sh fwv=`fw ver | awk {'print $7'}` opendblv="0.5" vsid=$1 # Update log cd /home/admin/dynamic_objects echo "`date` *** Starting update ***" >> blacklist.log echo " VS-$vsid" >> blacklist.log # Download all lists via VS0 vsenv 0 while read line; do url=`echo "https://opendbl.net/lists/${line}.list"` curl_cli -s --cacert opendbl.crt --user-agent "$fwv $opendblv" --retry 10 --retry-delay 60 $url | dos2unix > ${line}.blacklist done < blacklist.conf # Implement all lists on desired VS vsenv $vsid while read line; do # Create arrays with max size of 2000 of IP pairs y=0; z=0; todo=() while read ip; do if ! [[ "$ip" =~ [^0-9.-] ]]; then todo[$y]+=" $ip $ip" if [ $z -eq 2000 ]; then z=0 let y=$y+1 else let z=$z+1 fi fi done < ${line}.blacklist # Purge fully existing dynamic objects listname=`echo $line | awk -F- '{print $1}'` dynamic_objects -do dynob_blacklist_${listname} dynamic_objects -n dynob_blacklist_${listname} # Update with new IP lists from each array for i in "${todo[@]}" ; do dynamic_objects -o dynob_blacklist_${listname} -r $i -a done # Update log let x=y*2000+z echo -e " $x \t - $listname IPs set" >> blacklist.log done < blacklist.conf echo "*** Update finished ***" >> blacklist.log echo >> blacklist.log rm -f *.blacklist

Re: Dynamic Block Lists for Check Point firewalls

mickrose555 — Thu, 11 Jul 2019 20:20:53 GMT

Hello Daniel,

I don't have the skills to implement this safely but I would really want such dynamic IP blacklist on my cluster of R77.30 (with the management already upgraded to 80.X, the rest will follow ).

Can someone do it with me like a freelance or prof services? The alternative is my vendor but it's summer and everything is slower here in Switzerland.

thanks a lot,

mike

Re: Dynamic Block Lists for Check Point firewalls

Kurtis_Johnson — Mon, 22 Jul 2019 20:33:42 GMT

Would anyone be able to comment if these Lists not already be monitored by Anti-Bot or Anti-Virus reputation feeds?

Re: Dynamic Block Lists for Check Point firewalls

mickrose555 — Tue, 23 Jul 2019 08:56:24 GMT

Here is the Forti URL to check such malicious IP:

https://fortiguard.com/learnmore#botnet

These 2 malicious IPs are not in there but in some of the block lists, like:

https://opendbl.net/lists/blocklistde-all.list

141.98.80.67

185.211.245.198

Someone can share other search engines maybe.

Re: Dynamic Block Lists for Check Point firewalls

Ted_Serreyn — Mon, 10 Feb 2020 22:46:50 GMT

OK, so is this still the way to do things now that we have the ability to use external IOC feeds?

Open for discussion either way, just trying to think about improving things, and handling multiple feeds.

I believe that I am going to either an IOC or a dynamic block list for stuff that I am seeing with my clients to share data among them anonymously.

Ted Serreyn

Re: Dynamic Block Lists for Check Point firewalls

Luis_Miguel_Mig — Wed, 20 Jan 2021 19:44:01 GMT

The scripts at http://opendbl.net/opendbl-r80-0.52.tar seem not to be finished for dshield, right?
Has anybody configured dshield with dynamic objects?
It seems that opendbl.sh doesn't contain the option for dshield and then ip-opendbl-include.sh doesn't parse ip ranges.

Re: Dynamic Block Lists for Check Point firewalls

Kaspars_Zibarts — Thu, 21 Jan 2021 07:52:49 GMT

It's because dshield list is published as a range instead of single IP and update script cannot handle a range

194.147.140.0-194.147.140.255 92.63.196.0-92.63.196.255 45.155.205.0-45.155.205.255 167.248.133.0-167.248.133.255 92.63.197.0-92.63.197.255 78.128.113.0-78.128.113.255 46.161.27.0-46.161.27.255 94.102.51.0-94.102.51.255 193.27.229.0-193.27.229.255 192.241.224.0-192.241.224.255 89.248.168.0-89.248.168.255 138.99.216.0-138.99.216.255 92.118.160.0-92.118.160.255 92.118.161.0-92.118.161.255 192.241.223.0-192.241.223.255 83.97.20.0-83.97.20.255 185.153.197.0-185.153.197.255 45.146.165.0-45.146.165.255 80.82.77.0-80.82.77.255 89.248.165.0-89.248.165.255

this part needs re-writing to accept ipFirst-ipLast format in the list. Might look later today. Meetings right now 🙂

while read ip; do if ! [[ "$ip" =~ [^0-9.-] ]]; then todo[$y]+=" $ip $ip" if [ $z -eq 2000 ] then z=0 let y=$y+1 else let z=$z+1 fi fi done

Re: Dynamic Block Lists for Check Point firewalls

Kaspars_Zibarts — Thu, 21 Jan 2021 08:29:07 GMT

This one did the trick for me:

while read ip; do if ! [[ "$ip" =~ [^0-9.-] ]]; then if ! [[ "$ip" =~ "-" ]]; then # For single IP list todo[$y]+=" $ip $ip" else # For a range firstIP-lastIP format ip=`echo $ip | sed 's/-/ /'` todo[$y]+=" $ip" fi if [ $z -eq 2000 ]; then z=0 let y=$y+1 else let z=$z+1 fi fi echo "${todo[@]}" done

So dynamic object contents look ok now:

object name : dynob_blacklist_dshield range 0 : 45.146.165.0 45.146.165.255 range 1 : 45.155.205.0 45.155.205.255 range 2 : 46.161.27.0 46.161.27.255 range 3 : 78.128.113.0 78.128.113.255 range 4 : 80.82.77.0 80.82.77.255 range 5 : 83.97.20.0 83.97.20.255 range 6 : 89.248.165.0 89.248.165.255 range 7 : 89.248.168.0 89.248.168.255 range 8 : 92.63.196.0 92.63.197.255 range 9 : 92.118.160.0 92.118.161.255 range 10 : 94.102.51.0 94.102.51.255 range 11 : 138.99.216.0 138.99.216.255 range 12 : 167.248.133.0 167.248.133.255 range 13 : 185.153.197.0 185.153.197.255 range 14 : 192.241.223.0 192.241.224.255 range 15 : 193.27.229.0 193.27.229.255 range 16 : 194.147.140.0 194.147.140.255

Re: Dynamic Block Lists for Check Point firewalls

Luis_Miguel_Mig — Thu, 21 Jan 2021 11:48:57 GMT

I added a couple of sanity checks. You may also want to include your public ip addresses.

fwv=`fw ver | awk {'print $7'}`
opendblv="0.52"
y=0
x=0
z=0

function validateIP()
{
local ip=$1
local ip2=$1
local stat=0
local privip=1
local badip=1
if [[ $ip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
OIFS=$IFS
IFS='.'
ip=($ip)
IFS=$OIFS
[[ ${ip[0]} -le 255 && ${ip[1]} -le 255 \
&& ${ip[2]} -le 255 && ${ip[3]} -le 255 ]]
badip=$?
fi

if ! [[ "$ip2" =~ ^(192\.168|10\.[0-9]|172\.1[6789]\.|172\.2[0-9]\.|172\.3[01]\.) ]];
then
privip=0
fi

if [ $privip -eq 0 ] && [ $badip -eq 0 ]; then
stat=1
fi
return $stat
}

function convert {
while read ip; do
if ! [[ "$ip" =~ [^0-9.-] ]];
then

if ! [[ "$ip" =~ "-" ]]; then
# For single IP list
validateIP $ip
vip=$?
line=" "$ip" "$ip
else
# For a range firstIP-lastIP format
ip1=`echo $ip|cut -d '-' -f 1`
ip2=`echo $ip|cut -d '-' -f 2`
validateIP $ip1
vip1=$?
validateIP $ip2
vip2=$?
vip=$((vip1*vip2))
line=" "$ip1" "$ip2
echo "ip $ip ip1 $ip1 ip2 $ip2 vip1 $vip1 vip2 $vip2 vip $vip line $line" > /var/log/secfeed.2.log
fi
if [ $vip -eq 1 ]; then
todo[$y]+="$line"
if [ $z -eq 2000 ]
then
z=0
let y=$y+1
else
let z=$z+1
fi
fi
fi
done

dynamic_objects -do openDBL_$type
dynamic_objects -n openDBL_$type

for i in "${todo[@]}" ;
do
dynamic_objects -o openDBL_$type -r $i -a
done
}

function update {
curl_cli -s --cacert $CPDIR/conf/opendbl.crt --user-agent "$fwv $opendblv" --retry 10 --retry-delay 60 $url | dos2unix | convert
echo "$(date): Security Feed $type update" > /var/log/secfeed.log
}

Re: Dynamic Block Lists for Check Point firewalls

Luis_Miguel_Mig — Thu, 21 Jan 2021 14:00:04 GMT

by the way, does anybody know if a dynamic object is persistent / survives a reboot?

Re: Dynamic Block Lists for Check Point firewalls

Kaspars_Zibarts — Thu, 21 Jan 2021 19:51:47 GMT

I know that I have rebooted standby cluster member and dynamic objects survived there 🙂 I think clusterxl syncs them via tables but not 100% sure if you rebooted both cluster members

Re: Dynamic Block Lists for Check Point firewalls

Luis_Miguel_Mig — Fri, 22 Jan 2021 12:45:29 GMT

Thanks.
Interesting idea about the clusterxl. I guess they are not in sync lookin at this:

[Expert@fw1:0]# ls -l /opt/CPsuite-R80.40/fw1/database/dynamic_objects.db
-rw-rw---- 1 admin root 1084018 Jan 22 11:43 /opt/CPsuite-R80.40/fw1/database/dynamic_objects.db

[Expert@fw2:0]# ls -l /opt/CPsuite-R80.40/fw1/database/dynamic_objects.db
-rw-rw---- 1 admin root 1102273 Jan 22 04:23 /opt/CPsuite-R80.40/fw1/database/dynamic_objects.db

Re: Dynamic Block Lists for Check Point firewalls

Freco_Wong — Wed, 11 Aug 2021 03:00:11 GMT

It really useful and I succeed to setup in non-production with self-defined dynamic blocklist, thanks!

However, I encounter other issue, could anyone can help?

How to ignore errors if incorrect IP address in the IP blocklist?

for example:

-------

1.1.1.1

2.2.2.2

3.3.3

-------

I found the dynamic_objects will clear all data if one incorrect IP address inside instead of ignore the incorrect IP address.

Regards

Freco

Re: Dynamic Block Lists for Check Point firewalls

Kaspars_Zibarts — Mon, 16 Aug 2021 08:48:58 GMT

Interesting - which of the public lists had incorrect IP included? Just curious. I thought they would have made sure that IPs are correct before pushing out lists.

Re: Dynamic Block Lists for Check Point firewalls

Freco_Wong — Mon, 25 Oct 2021 07:34:06 GMT

It is possible if update by human, so I am asking how to minimize the impact instead of clear all data once problem triggered