cancel
Showing results for 
Search instead for 
Did you mean: 
Post a Question

Log rate calculator for MDS / MLM

There are two great SKs out there, so this is just re-inventing the wheel

sk120341 - How to monitor the Log Receive Rate on Management Server

sk88681 - How to calculate/count the total amount of FireWall Logs per second that arrive to Security Management Server

I simply bastardised the script from the second article to make it faster (it runs parallel instead of sequential through all CMAs) and output is more easy to read. But all the credits go to CP!

log rate calculator

Here's your script, a you can see you can supply time that you want to run it for, by default 5secs

#!/bin/sh
# Print log rate data on all Domains

# execute the script that defines Check Point environment variables
source /opt/CPshared/5.0/tmp/.CPprofile.sh
total=0

if [ "x$1" = "x" ]; then
   SLEEP_TIME=5
else
   SLEEP_TIME=$1
fi

echo
echo "Log rate per second, measuring for ${SLEEP_TIME}s"
echo "----------------------------------"
ls -l /var/log/mds_logs/*/log/fw.logptr > BEFORE
sleep $SLEEP_TIME
ls -l /var/log/mds_logs/*/log/fw.logptr > AFTER

for DOMAIN in $(ls -1 $MDSDIR/customers)
do
   SIZE_BEFORE=$(cat BEFORE | grep "/$DOMAIN/" | awk '{print $5}')
   SIZE_AFTER=$(cat AFTER | grep "/$DOMAIN/" | awk '{print $5}')
   rate=`echo "scale=0 ; ($SIZE_AFTER - $SIZE_BEFORE) / ( 4 * $SLEEP_TIME )" | bc`
   let total=total+rate
   echo -e -n "${DOMAIN}:\t"
   if [ ${#DOMAIN} -le 15 ]; then echo -e -n "\t"; fi
   echo "$rate"
done

echo "----------------------------------"
echo -e "Total rate:\t\t$total"
echo

5 Replies
Admin
Admin

Re: Log rate calculator for MDS / MLM

Going to move this to the ‌ space

Thanks for sharing!

0 Kudos

Re: Log rate calculator for MDS / MLM

Log rate per second, measuring for 5s
----------------------------------
dcwclm01: 272
dcwclm02: 220
dcwclm03: 1788
dcwclm04: 2120
dcwclm05: 364
dcwclm06: 652
dcwclm07: 3860
dcwclm08: 4
dcwclm09: 2328
dcwclm10: 2144
dcwclm11: 2280
dcwclm12: 884
----------------------------------
Total rate: 16916

Running on physical server with 28 CPU cores and 14 TB storage of with Raid 6..

Is there any recommendation how much logs should CLM receive?

Re: Log rate calculator for MDS / MLM

I haven't seen exact recommendations - it really depends on how fast is your IO to disks I guess. As long as you don't have massive backlog ( ) and rest of the resources (CPU/RAM/disks) are under control you should be good.

Re: Log rate calculator for MDS / MLM

@Kaspars_Zibarts  I've modified script to get a little bit different output format in order to graph those data, but having issues that sometimes it's throwing negative values, check attached image. What could be the cause ? I suspect this:

rate=`echo "scale=0 ; ($SIZE_AFTER - $SIZE_BEFORE) / ( 4 * $SLEEP_TIME )" | bc`

 

 

 

Re: Log rate calculator for MDS / MLM

I would say when log is rotated then before value would end up higher than after. So you would need to add a check if log file has changed whilst waiting for calculation 🙂 and then ignore results. Or simply ignore negative results as they are caused (most likely) by log file rotation
0 Kudos