Varnish Proxy: How to detect & block malicious bots?

Varnish is a great reverse HTTP proxy, but how does it manage protection against malicious bots?

In this article, we’ll explore the capabilities of the Varnish web application firewall, and discuss what defenses it can and cannot provide. We will also cover how the DataDome Varnish module works, its benefits, and how to start using it to protect your website from malicious bot activity.

Varnish Bot Protection: The Varnish WAF

Varnish is first and foremost a caching reverse HTTP proxy (the caching server stores content from origin web servers so that Varnish can respond to client requests without even sending the request to the web server), but it has many other features, including a web application firewall (WAF).

The Varnish WAF is based on the open-source ModSecurity project and can stop well-known, immediately identifiable bots. But like all WAFs, is rules-based, relying primarily on signatures. It does have minimal heuristics rules, but they tend to generate a lot of false positives. Due to the false positives, the heuristics rules are often not enabled.

How the DataDome Module for Varnish Works

DataDome is a purpose-built bot detection solution that pools data from multiple sites and uses machine learning to continuously update its algorithm, which enables it to detect both known bots and new, unfamiliar visitors.

There are no changes required to your architecture due to the seamless integration of the DataDome Varnish module into your web infrastructure.

The DataDome logic is processed before the regular Varnish process starts. The module makes a call to the DataDome API, and depending on the API’s response, determines whether or not Varnish should continue the caching process. DataDome’s points of presence are located in different regions across the globe, to ensure quick response times and provide high availability.

There is a timeout for the API calls to detect possible errors in the process. If the timeout is reached, the blocking process will be disabled. This ensures that there is no negative impact to the users. There is also a timeout for opening a new connection. If the default timeouts are not adequate for your particular environment, they can easily be customized.

Key Benefits

By leveraging the DataDome bot protection module, you can expect:

Expert bot detection fully integrated into Varnish and optimized for this reverse HTTP proxy.
AI and machine learning-driven intelligent bot detection, differentiating human behavior from bots and taking the proper action to protect against malicious bots.
Improved site performance, thanks to the elimination of high volumes of illegitimate traffic.
Real-time protection (< 2 milliseconds) from price scraping, DDoS attacks, credential stuffing, and all OWASP automated threats.
Unmatched customization options, thanks to a powerful custom rules engine.
More reliable analytics with data that reliably distinguishes humans from bots.

Getting Started With the DataDome Module for Varnish

It takes less than a minute to start testing DataDome with Varnish today. Click the FREE TRIAL button below and create your account, no credit card needed.

Then, follow the Varnish module installation instructions in our technical documentation. It takes just a few simple steps:

Ensure you have libraries installed as specified in the documentation.
Run the script provided in the documentation.
Set the license key and API server endpoint in the datadome.vcl configuration file.

Give it a try and see the Varnish module in action! You can instantly go to your personal DataDome dashboard, and observe all bot traffic to your website in real time.