How This Open Source Security Tool Halted Significant DDos Attacks

Configuration changes to CrowdSec stopped a 7,000-machine botnet in less than a minute.

Philippe Humeau

In 2020, our ways of living and working were turned completely upside down in a matter of days. As COVID-19 began to spread across the globe, we brought our companies home, and staying connected to our colleagues, friends, and family online became a critical necessity. This opened the door for hackers to cause disruption; for example, distributed denial of service (DDoS) attacks around the world were up 151% in the first half of the year, according to Neustar.

CrowdSec is an open source security engine that analyzes visitor behavior and provides an adapted response to all kinds of attacks. It parses logs from any source and applies heuristic scenarios to identify aggressive behavior and protect against most attack classes. It then shares that intelligence with other CrowdSec installations; every time an internet protocol (IP) address is blocked, it informs the entire user community. This creates a real-time, collaborative IP reputation database that leverages the crowd's power to make the internet safer.

How CrowdSec works: a case study

Sorf Networks, a Turkey-based technology company that provides high-configuration managed servers and DDoS protection solutions for its clients, offers an example of how CrowdSec works. One of Sorf's customers was experiencing daily DDoS attacks from 10,000+ machine botnets and struggled to find a solution that would meet technical requirements to deal with them in a timely manner.

While the customer took general precautions to mitigate those attacks, such as introducing JavaScript (JS) challenges, rate-limiting, and so on, they weren't viable on the entire attack surface. Some URLs needed to be consumed by very basic software that didn't support JS challenges. Hackers being hackers, this was exactly what they targeted every single day: the weakest link in the chain.

Sorf Networks first set up a DDoS mitigation strategy for its customer using Fail2ban (which inspired CrowdSec); it helped to some degree, but it was too slow. It required 50 minutes to process logs and deal with 7,000- to 10,000-machine DDoS attacks—which rendered it ineffective in this situation. Also, logs continued to stack because it did not ban IPs, and it needed to process several thousand logs per second, which was not possible.

In DDoS testing using a rented botnet, an attack reached around 6,700 requests per second from 8,600 unique IPs. This is a capture of one server's traffic.

Crowdserve server traffic (©2020, CrowdSec)

Although CrowdSec technology can cope with huge attacks, its default setup can process only around 1,000 endpoints per second. Sorf needed a tailor-made configuration to deal with this much traffic on a single machine.

Sorf's team made changes in CrowdSec's configuration to significantly improve its throughput to absorb the log volume. First, it removed expensive and non-crucial enrichment parsers, such as GeoIP enrichment. It also increased the default number of allowed go-routines from one to five. Afterward, the team did another live test with 8,000 to 9,000 hosts, averaging between 6,000 and 7,000 requests per second. This solution came at a cost, as CrowdSec was eating 600% CPU during the operation, but its memory consumption stayed around 270MB.

The results, however, showed remarkable success:

• In one minute, CrowdSec was able to ingest all the logs
• 95% of the botnet was banned and the attack efficiently mitigated
• 15 domains are now protected from DDoS attacks

According to Sorf Networks' director Cagdas Aydogdu, CrowdSec's platform enabled the team "to deliver a world-class and efficient defense system … in an incredibly short timeframe."

This article has been adapted from How to stop a 7k machine botnet in 1 minute with CrowdSec, originally published on CrowdSec's website.

About the Author

Philippe Humeau graduated in 1999 as IT security engineer from EPITA (Paris, France). He founded his first company at the same time and quickly oriented it towards penetration testing and high security hosting. He was also deeply involved in Magento’s community creation & animation in France and versed into eCommerce (wrote 4 books on the topic). The company (NBS) was sold in 2016 and Philippe founded CrowdSec in 2019, gathering all his experience to create a new Open-source security…More…

Attribution

This article was published in Opensource.com. It is republished by Open Health News under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0). The original copy of the article can be found here.