Comparis Automates Scraping Protection & Saves a Day Per Week With DataDome

The Problem: Content Theft, Time Spend, and Degraded User Experience

“The last couple of years were a nightmare for our operations team,” says Styv Mermod, CTO at Comparis. “Every evening during prime time, the on-call service had to intervene at least two or three times to try to block scraping attacks. Sometimes, blocking an IP address would stop it; sometimes, it was an all-night battle.”

As Switzerland’s leading marketplace and comparison service for competitive sectors like real estate, Comparis is an obvious target for people wanting to piggyback on their success. For some time, the engineering team had observed a major increase in traffic from bots scraping the website to fetch newly published information. 

“They were getting smarter and smarter,” says Styv. “Initially, we could easily manage it with a set of simple rules on our load balancer, but with time it turned into a massive amount of manual work.”

It was difficult for the team to tell whether the unwanted visitors were just lazy scrapers unwittingly abusing bandwidth, or more malevolent attackers aiming to destroy their business. But either way, the volume of automated traffic was having a negative effect on the website’s real users and business partners.

“Human users would sometimes be blocked because they happened to match the set of rules we were implementing to prevent the massive attacks,” Styv explains. “Friendly partners that are allowed to crawl us were sometimes also blocked. And finally, we have partners that are feeding information to our website, and who were not happy to find their data on sites they are not affiliated with.”

The Solution: Easy to Test, Easy to Integrate Bot Protection

At one point, Styv took a long, hard look at the real cost of continuing to rely on manual prevention. When he presented his findings to the board, they agreed that implementing a bot protection solution was a no-brainer. But which bot protection?

“I’m a big fan of recommendations,” Styv smiles. “Switzerland is a small country, where most CTOs know each other. So I asked for advice in our Slack channel, and quickly had a shortlist including DataDome and a couple of other providers which we ran through our usual evaluation process.”

A fact that tipped the scale in DataDome’s favor early on was the ease of testing and integrating the solution. 

“It was very easy to integrate on just one or two servers, so we could confirm that it didn’t have any impact on performance or processing time,” he comments. “And in terms of integration, the other tools we evaluated required DNS rerouting, which means that you lose control over parts of your infrastructure. DataDome offers different levels of integration which are not only easy to implement, but also to remove, should we want to.”

The DataDome dashboard, which provides complete and detailed traffic data (even in free trial mode), confirmed Styv’s suspicions: more than half of Comparis’ traffic was automated.

“Personally, I wasn’t surprised. But when I presented the first report to the board, they definitely were,” he says. “With the tools we had before, I wasn’t able to really show the severity of the bad bot problem. The DataDome dashboard is very easy to use and understand even for non-techies, and it’s great to have that visibility that was missing.”

The Results: Restored User Experience, Significant Time Savings

DataDome now protects Comparis from scraping and all other bot-driven threats. Human users can find information and make comparisons at optimum speed without being blocked, and Comparis’ partners can trust that their data will only be used as intended. 

For Styv and especially for the operations team, the main benefit is that they no longer need to spend their evenings playing cat-and-mouse with attackers.

“DataDome is very close to a fire-and-forget solution,” Styv affirms. “Once set up and configured, you can just rely on it to run in the background. Nothing needs to be done; we spend maybe 10 minutes a day on monitoring, but that’s all. The time savings are difficult to quantify, but quite significant: I would estimate that we save about a full day’s manual work per week.”

In conclusion, he praises the ease of use of the DataDome solution and support channel.

“We never needed a two-day crash course for the techies to understand how it works or where to look if such-and-such happens; we have a simple Confluence page, and that’s all. It’s also nice and quite unusual to be able to use Slack for support, instead of the typical ticketing systems. Using Slack feels much more human, and closer to the modern way of working. It’s a small thing that I and the whole team really appreciate.”