CAPTCHA Farms & Challenges of CAPTCHA Bot Detection

Detecting CAPTCHA Farms

Detecting bots that leverage CAPTCHA farms is challenging. In fact, many bot management providers accept solved CAPTCHAs as proof of the visitor’s humanity. (We know otherwise.)

Other bot management providers will give the user a certain “credit” of allowed requests, based on the session cookie, after passing a traditional CAPTCHA. In this case, the bot just needs to send its trusted session cookie along with malicious requests in order to avoid being challenged for a while.

Solved CAPTCHAs can also be used as a feedback mechanism for false positives (we do this at DataDome). If a human is blocked by mistake, our detection system corrects the error by letting the user continue to browse the website after solving the CAPTCHA. Since our bot detection system uses machine learning, the algorithm uses any mistakes to self-correct.

The DataDome dashboard shows the number of CAPTCHAs passed, which enables users to monitor detection quality.

CAPTCHA farms make the CAPTCHA feedback loop less reliable. If the system accepts solved CAPTCHAs as absolute proof of humanity, CAPTCHA farms will increase the number of false negatives (bots passing as humans).

On the other hand, if the bot protection system invalidates CAPTCHAs too diligently, it increases the risk of false positives (hard-blocking legitimate human users).

In order to efficiently stop bad bots while preserving the user experience for humans, accurate detection of CAPTCHA farms is essential.

Although we can’t give away too much detail about how we detect CAPTCHA farms, below are a couple examples of the high-level approaches that DataDome’s real-time detection engine relies on to detect bots that submit CAPTCHAs solved by CAPTCHA farm workers.

Fingerprinting

Our detection engine always conducts a deep analysis of visitors’ browser fingerprints. If the engine is 100% sure that the user is a bot (e.g. if it detects a modified Selenium, Puppeteer, or Playwright bot,) it will automatically invalidate the CAPTCHA passed.

Besides detecting well-known browser automation frameworks and headless browsers, our engine analyzes countless other fingerprinting signals in order to make its decision.

Solve Speed

Alongside the browser fingerprints of visitors, we also look at how quickly a visitor solves a CAPTCHA. The graph below shows two cumulative distribution functions: one for how quickly a CAPTCHA farm worker solves a CAPTCHA, another for how quickly other users solve them.

The blue line grows significantly faster than the orange line. This means that CAPTCHA farm workers are much faster at solving CAPTCHAs than regular users. Indeed, CAPTCHA farm workers solved around 50% of CAPTCHAs in less than 5 seconds where normal users only solved around 30%.

AI Outlier Detection

DataDome also leverages AI outlier detection to identify suspicious CAPTCHA-solving traffic.

For example, if our detection engine detects a sudden increase in outdated browsers (based on the user agent) coming from unusual countries for a protected website, it may flag these CAPTCHA responses as potentially coming from CAPTCHA farms. DataDome recently did an analysis of 2 million CAPTCHAs passed over 3 months to better understand which countries tend to have the most CAPTCHA farm workers.

We did looked at the IP addresses of the specific workers of CAPTCHA farms and map those IPs to particular countries (anything more granular than a country has a much higher chance of being inaccurate). Here’s what we found:

This was further broken down into the ISPs and telecom providers of CAPTCHA farm workers.

Using the data above, along with more advanced heuristics and statistical techniques based on different signals (like IP score and device fingerprint), DataDome’s detection engine determines whether or not it should validate each CAPTCHA attempt.

If the solved CAPTCHA is invalidated, the engine generates a pattern to automatically block similar CAPTCHA attempts in the future.

CAPTCHAs are not an anti-bot silver bullet.

While traditional CAPTCHAs still have their uses, they won’t keep your website or your mobile app safe from malicious bots. Even CAPTCHAs that claim to be smarter, such as reCAPTCHA v2 vs v3, have their downsides:

• The difficulty of setting proper block/allow thresholds.
• The need to monitor false positives and false negatives (feedback loop).
• The risk of blocking legitimate good bots (search engine bots, SEO bots, technical partners, etc.).

As we have seen, for very small fees, motivated bot developers can easily leverage CAPTCHA farms to bypass security systems that put too much trust in CAPTCHAs or use them as a first line of defense. An effective bot protection solution, on the other hand, will block bots—including bots that rely on CAPTCHA farms—while remaining invisible to human users.

The DataDome bot protection software—now with its own integrated CAPTCHA—analyzes 100% of the requests that hit our customers’ applications. We process 1 trillion signals per day, collecting and analyzing more than 250 different events for each and every request in order to accurately distinguish between humans and bots. We protect 99.99% of your users without ever presenting them with a CAPTCHA.

New threats are identified via statistical and behavioral detection, using data from server-side fingerprints, a JS rendering engine, SDK inputs, and session tracking. We make extensive use of online machine learning, and detect a new bad bot pattern every 10 milliseconds. Our false positive rate is below 0.01%, meaning that of 10,000 CAPTCHAs served, less than one is seen by a human.

If you’d like to see the real-time bot activity on your own website, you can set up a free DataDome trial yourself in less than an hour (no commitment, no credit card). All you have to do is create your free account and follow the installation instructions. Then, you can access your personal dashboard for a full overview of good bot, bad bot, and human traffic to your site.

Ready? Start here.