9 Questions to Ask Your Bot Management Provider About False Positive Rates [2024 Update]

False positives are a good indicator of the balancing act between security and user experience. On one hand, you want to fully protect your mobile apps, websites, and APIs against all bot-driven online threats. On the other hand, you don’t want your users to notice the protection—it should be a frictionless user experience (UX). The false positive rate tells you how well your bot protection walks the security/UX tightrope.

• What is a false positive?
• Security that’s not too tight, but not too permissive.
• 9 questions to ask your bot management provider about false positives.

What is a false positive?

A false positive is when a program or test believes something is positive when it is actually negative. A false positive in the bot protection world means that your bot management tool flags a request as coming from a bot when it actually comes from a legitimate human user.

Different bot protection tools have different responses to false positives. Some programs hard-block any request suspected to be a bot request, leaving them blind to false positives. Others serve a CAPTCHA challenge to all users and those who pass are identified as legitimate (human) requests and those who fail are identified as bots—this is when the UX suffers for legitimate users.

An example of a more balanced approach is how DataDome processes requests—only showing a CAPTCHA when a request is suspected to be bot generated (based on many diverse detection signals), then using the challenge results as an additional signal to fully understand whether the request is from a human or not, and responding accordingly. This provides a solid feedback loop on false positives to learn and optimize protection moving forward.

Security that’s not too tight, but not too permissive either.

Whether it’s on an application or a website, a seamless user experience is the pinnacle of today’s digital age. Users have come to expect easy, fast browsing. Therefore, business leaders don’t want a security solution that adds any friction to their UX, which is why a low false positive rate is critical. At the same time, effective security can’t allow advanced bots through in increasing numbers, risking your business and customers’ security.

The CTO of Pneus Online, an online shop for car and motorcycle tires, contacted DataDome because half of the business’ traffic was coming from web scrapers. The Pneus team had tried solving their bot problem themselves, but the dynamic IPs of the scrapers made it very hard to understand which requests were human and which weren’t.

It’s important that your solution is not too permissive, because allowing advanced bots to bypass security due to the fear of blocking humans will often lead to performance issues, account takeovers, and many damaging problems. Unwanted bot traffic on your site will hurt the user experience, put sensitive user data at risk, and ultimately affect your bottom line.

DataDome has a false positive rate of less than 0.01%—out of 10,000 DataDome CAPTCHAs served, less than one is seen by a human. We know our false positive rate because, even when a user passes the CAPTCHA, we continue to monitor their activity using invisible signals to ensure we always identify and block malicious bots while letting real users through. That’s how we ensure accuracy without compromise.

9 questions to ask your bot management provider about false positives.

When evaluating a bot and online fraud management provider, consider the following questions. (Not all questions are directly related to false positives, but they all relate to either accuracy or the impact on your UX.)

1. What is your false positive rate? Ask the vendor for a percentage and how they measure it. Because there’s no strict, industry-standard definition of a false positive, different vendors might measure it differently.
2. How do you handle false positives? Do you show the visitor a CAPTCHA and monitor how “human-like” the visitor solves the challenge, based on various invisible signals? Do you keep monitoring the request after a CAPTCHA has been solved? DataDome does all this to ensure we never hard-block a human.
3. How do you deal with CAPTCHA farms? Bots often connect to CAPTCHA farms, where human workers solve CAPTCHAs on their behalf. If your bot protection does not have a way to identify CAPTCHA farms, many bots will bypass it and reach your platform.
4. How easily does your technology integrate with my infrastructure? The right security solution must smoothly integrate with your current and future technical infrastructure, including your mobile apps and APIs, because bots attack those too.
5. How do you detect bots that show human-like behavior? Advanced bots can now mimic human behavior, such as wandering mouse movements, human-like clicks, and credible browsing speed. Those bots shouldn’t be able to fool your security solution.
6. How do you stay ahead of the latest bad bot trends? Bots evolve at such rapid speeds that a security solution won’t ever be able to keep up unless they have a dedicated team that monitors open-source libraries, forums, and other places where bot trends may come from.
7. How do you distinguish good bots from bad bots? Not all bots are bad. You wouldn’t want to block the Googlebot that helps visitors find your website. But many bad bots masquerade as good bots, and they often do so incredibly well. Ask your security solution provider how they tackle this challenge.
8. Will your solution slow down my digital properties? Everyone hates lag. Because response times are crucial for good UX, your security solution should not noticeably slow down any of your digital properties.
9. How easily can I speak to a person on your team once I’ve installed your solution? Once you’ve chosen a security solution, you’re certain to have questions. Make sure your vendor is readily available to help you post-purchase, whether that’s through email, Slack, chat, or a phone call.

In an ideal world, bot and online fraud protection fully protects your websites, mobile apps, and APIs without impacting the user experience. False positive rates indicate how well bot protection balances key performance indicators, such as accuracy, speed, and UX. The lower the false positive rate, the better. On websites and apps protected by DataDome, less than one in 10,000 CAPTCHAs is seen by a human, and we are constantly working hard to keep that number even lower.

Are you looking for a bot detection solution that minimizes false positives? Try out DataDome’s free, 30-day trial today.