A distributed denial-of-service (DDoS) attack is a malicious attempt to disrupt normal traffic of a targeted server, service or network by overwhelming the target or its surrounding infrastructure with a flood of Internet traffic. DDoS attacks achieve effectiveness by utilizing multiple compromised computer systems as sources of attack traffic. Exploited machines can include computers and other networked resources such as IoT devices. From a high level, a DDoS attack is like a traffic jam clogging up with highway, preventing regular traffic from arriving at its desired destination.
Well, nothing except the fact that in DDoS attack the incoming traffic flooding the victim originates from many different sources.
This effectively makes it harder to stop the attack because we cannot simply block a single source.
They build an army!
Army of botnets around the world that are waiting for a command.
That command will usually include info such as:
These botnets could be any type of a computer that is connected to the internet and is currently infected with a piece of malware that allows the attacker to control the device in order to launch the DDoS attack.
Usually when an attacker decides that they want to attack a specific target they won’t start to build an army of botnets for the specific attack.
For that there are already services that ran by hackers which offer the use of their already prepared infrastructure of botnets.
A nice example is of two young Israeli hackers that were able to compromise millions of IoT devices all around the world and used them to launch DDoS attack on specific targets.
You can read more about it here:
https://krebsonsecurity.com/2016/09/israeli-online-attack-service-vdos-earned-600000-in-two-years/
The easiest and quickest way to prove or disprove if a DDoS attack occurred on one of CheckPoint’s Firewalls, would be to use cpview tool to go back in time to the point of the attack and look for a spike in the CPU utilization and packet rate.
If you are not familiar with cpview tool, check out sk101878
In version R77.10, cpview got the History Mode feature that automatically keeps all cpview’s records for 30 days backwards.
With this feature we can go back in time to the point of the suspected DDoS attack and check what exactly happened during this time on the GW.
This can be done by running cpview with the following syntax:
# cpview -t <timestamp>
where <timestamp> format is: [Jan...Dec] [01...31] [4-digit Year] [hh:mm:ss]
IMPORTANT NOTE: Starting R80.30 the syntax of CPView history mode has changed, this guide was generated using older versions and the old syntax. please review sk163804 for the new syntax.
For example, if you believe a DDoS attack occurred on April 30, 2019 during 13:00 – 15:00 we will execute cpview with the following timestamp:
# cpview -t Apr 30 2019 12:30:00
Note that we set the timeframe to 30 minutes prior to when the attack started, this will later help us compare between during the attack and normal operation.
#2 – Network section
We need to keep in mind the following values of a normal operation:
CPU utilization
Packets/sec
Now using the ‘+’ character we will go forward in time, minute-by-minute and keep monitoring these values.
We will look for a massive increase or spikes.
In case we are dealing with a DDoS attack the CPU utilization and packets per second rate should be correlated to each other.
In case we only see the CPU utilization rise during the mentioned time frame, it means that we are most likely not dealing with a DDoS attack and instead we might be facing a performance issue on the GW that should be investigated.
Solution
Technical Support (TAC) should investigate performance issue.
In case we see an increase of packets/sec value without CPU levels increasing, it could indicate a DDoS attack that our GW is not affected by and in that case it could be that the end device is not capable of handling the traffic.
The next step should be to find out which traffic is causing such behavior, it can be done with the following actions:
SmartView time frame filter
SmartView Top Sources viewSolution
In this scenario we are most likely dealing with a DDoS attack, but same as scenario #2, we need to let the customer review the source of the traffic to make sure it’s not an internal server or other source that they are familiar with.
Use steps from scenario #2 to verify with the customer.
Solution
In VSX mode, cpview is supported only from the context of VS 0.
The network section in the main view may show us irrelevant data, the CPU utilization section will show us the correct information.
The approach in this case is to view the Packets/sec value on the interface level instead of the summary of all interfaces.
To do so we need to verify which interface is experiencing the DDoS (usually the internet facing interface), and then go to Network --> Interfaces --> Traffic:
In the RX Traffic (Received) section, we should focus on the “pps” column and look for the spike/increase.
In case you suspect a DDoS attack and a traffic capture from the time of the attack has been collected, the following will describe how we should determine if the packet capture contains any signs of a DDoS attack
Let’s review a few general key points that we should look for inside the traffic capture file:
All of the above key points seem hard to answer, but luckily Check Point has a tool that can help us answer all of them.
The tool name is CPMonitor and can be found inside sk103212
CPMonitor can work in two different modes:
When investigating a suspected DDoS attack inside a packet capture file, we should start with the “Complete Mode” in order to verify points 1-4 from the above list.
The command to run is:
# cpmonitor /Path_To_pcap_File/traffic.pcap
The output should like this:
=====================================
| Dump Info |
=====================================
| Name: SYN_Defender_on_April_11.pcap |
| Start time: 04/12/19 02:34:35.41 |
| End time: 04/12/19 02:36:43.10 |
=====================================
=================================================================
| Total Usage |
=================================================================
| Capture duration: 127.688 seconds |
| Total packets: 18273340 |
| Total packets size: 9660 MB |
| Avg. PPS: 143109.30 [packets/second] |
| Max PPS: 151940 [packets/second] at 02:35:21.41 |
| Avg. CPS: 4231.74 [conns/second] |
| Max CPS: 10530 [conns/second] at 02:34:36.41 |
| Avg. throughput: 605 Mb [bits/second] |
| Max throughput: 663 Mb [bits/second] at 02:35:30.41 |
| Avg. concurrent conns: 17010.09 [conns/second] |
| Max concurrent conns: 17576 [conns/second] at 02:35:01.41 |
| Sort method: packets rate |
| Unsupported entries: 827 |
=================================================================
Top Connections
========================================================================================================================
| source | port | dest | port | ipp | packets (%) [c2s/s2c] | size (%) [c2s/s2c] | avg | desc. |
========================================================================================================================
| 10.24.129.44 | 39375 | 157.240.14.52 | 443 | TCP | 49246 (0%) [45%/55%] | 27 MB (0%) [3%/97%] | 42/1038 | https |
| 10.26.187.252 | 43638 | 200.7.241.12 | 443 | UDP | 25748 (0%) [32%/68%] | 23 MB (0%) [2%/98%] | 68/1376 | https |
| 10.26.215.106 | 54760 | 200.7.241.13 | 80 | TCP | 24960 (0%) [24%/76%] | 26 MB (0%) [0%/100%] | 42/1477 | http |
| 10.24.107.79 | 37846 | 190.98.133.161 | 443 | TCP | 22769 (0%) [43%/57%] | 13 MB (0%) [3%/97%] | 42/1045 | https |
| 10.197.229.143 | 57768 | 200.7.241.14 | 443 | TCP | 22001 (0%) [35%/65%] | 20 MB (0%) [1%/99%] | 42/1472 | https |
| 10.24.212.89 | 60172 | 200.7.241.13 | 443 | TCP | 17848 (0%) [33%/67%] | 12 MB (0%) [1%/99%] | 42/1049 | https |
| 10.24.248.44 | 54129 | 157.240.14.52 | 443 | TCP | 16619 (0%) [63%/37%] | 5919 KB (0%) [95%/5%] | 552/40 | https |
| 10.24.128.188 | 34548 | 200.7.241.12 | 443 | TCP | 16331 (0%) [32%/68%] | 11 MB (0%) [1%/99%] | 42/1049 | https |
| 10.26.195.81 | 49283 | 200.7.241.13 | 443 | TCP | 16040 (0%) [32%/68%] | 11 MB (0%) [1%/99%] | 42/1051 | https |
| 10.209.49.116 | 43608 | 200.7.241.12 | 443 | TCP | 15915 (0%) [33%/67%] | 10 MB (0%) [1%/99%] | 42/1050 | https |
========================================================================================================================
Top destinations
==========================================================================
| IP | packets (%) [c2s/s2c] | size (%) [c2s/s2c] | avg |
==========================================================================
| 157.240.14.52 | 812157 (4%) [51%/49%] | 418 MB (4%) [4%/96%] | 357/733 |
| 190.98.133.225 | 759828 (4%) [39%/61%] | 519 MB (5%) [2%/98%] | 50/1149 |
| 157.240.14.15 | 731021 (4%) [49%/51%] | 260 MB (2%) [15%/85%] | 231/514 |
| 200.7.241.12 | 626071 (3%) [35%/65%] | 507 MB (5%) [2%/98%] | 62/1283 |
| 200.7.241.13 | 610489 (3%) [34%/66%] | 485 MB (5%) [2%/98%] | 56/1242 |
| 168.195.56.82 | 600580 (3%) [38%/62%] | 440 MB (4%) [2%/98%] | 53/1212 |
| 200.7.241.14 | 549053 (3%) [32%/68%] | 475 MB (4%) [2%/98%] | 66/1306 |
| 190.98.133.161 | 533195 (2%) [38%/62%] | 368 MB (3%) [2%/98%] | 49/1152 |
| 157.240.14.53 | 484829 (2%) [49%/51%] | 68 MB (0%) [32%/68%] | 96/199 |
| 168.195.56.97 | 445217 (2%) [38%/62%] | 296 MB (3%) [3%/97%] | 55/1103 |
==========================================================================
Top Services
==============================================================================================
| service | ipp | packets (%) [c2s/s2c] | size (%) [c2s/s2c] | avg | desc. |
==============================================================================================
| 443 | TCP | 13898938 (76%) [42%/58%] | 7681 MB (79%) [0%/100%] | 129/915 | https |
| 443 | UDP | 1300254 (7%) [34%/66%] | 1105 MB (11%) [1%/99%] | 234/1229 | https |
| 80 | TCP | 783018 (4%) [43%/57%] | 440 MB (4%) [6%/94%] | 88/978 | http |
| 53 | UDP | 477804 (2%) [51%/49%] | 46 MB (0%) [33%/67%] | 67/140 | DNS |
| 5222 | TCP | 400813 (2%) [48%/52%] | 54 MB (0%) [32%/68%] | 93/187 | xmpp-client |
| 3478 | UDP | 211739 (1%) [45%/55%] | 85 MB (0%) [0%/100%] | 450/398 | stun |
| 7008 | UDP | 135954 (0%) [25%/75%] | 8907 KB (0%) [24%/76%] | 64/67 | afs3-update |
| 5228 | TCP | 128290 (0%) [44%/56%] | 36 MB (0%) [14%/86%] | 94/456 | hpvroom |
| 10004 | UDP | 80584 (0%) [32%/68%] | 7393 KB (0%) [20%/80%] | 59/110 | emcrmirccd |
| 10003 | UDP | 56965 (0%) [27%/73%] | 5067 KB (0%) [17%/83%] | 57/104 | documentum-s |
==============================================================================================
=============================
| TCP Stat | Count |
=============================
| SYN | 293174 |
| SYN|ACK | 290635 |
| SYN|ACK|PUSH | 2 |
| SYN|PUSH | 2 |
| ACK | 12327501 |
| ACK|PUSH | 2053247 |
| ACK|URG|PUSH | 1 |
| FIN|ACK | 216722 |
| FIN|ACK|PUSH | 59617 |
| FIN|ACK|URG | 1 |
| FIN|ACK|URG|PUSH | 1 |
| RST | 219186 |
| RST|ACK | 20819 |
| Invalid | 25 |
=============================
IMPORTANT NOTE: Traffic capture should be taken without any filters. Most of the tests we do depend on anomaly detection and therefore we need the DDoS traffic to “stand out” from the rest of the traffic.
As we can see in the above example, the number of SYN-ACKs is almost the same as the number of total SYNs (no DDoS this time).
During SYN Flood DDoS attack the server in question won’t be able to answer all the SYN requests that are coming towards the server due to the high volume of requests (DDoS).
The important part is the “Avg. PPS:” value, should be significantly higher when comparing the traffic captures from the time of DDoS and the normal operation time.
In this section we can detect the top connections directions, it is very important to identify the directions for the top connections that are involved in the DDoS attack so we can determine if this is caused by an internal/external resource.
In addition using this section we can also tell how much traffic in size was sent by the server and how much was sent by the client, see “size (%) [c2s/s2c]” column.
When dealing with normal traffic, the server is usually the one sending larger volumes of traffic to the client and not the other way around (except for uploads and probably other edge cases).
If we notice the client is sending more traffic than the server, this might indicate a DDoS attack.
Key points #5 & #6 can be detected by running CPMonitor in “Navigate” mode.
Using this mode we can go through the traffic capture second by second and see how the attack behaves, we can still review the above mentioned point while running in Navigate mode. It’s important to look for the destination IPs that are being communicated the most and in case this is an internal resource like a web server for example, we need to make sure that it is active and responding (at least to some of the requests).
One of the most common scenarios in which customer claim there is a DDoS attack when there isn't, is when an internal resource like a web server becomes unavailable/not responding and then all the clients are trying to connect/reconnect at the same time.
In the traffic logs we see many attempts to open new connections to the website and it may look like a DDoS attack, but if we see that the web server is not replying to any of the requests at all it means that these are probably legitimate requests to the website which is not responding.
Final Notes & Recommendations
