LV Digital Protects its Classified Ads & Generates New Revenue with DataDome

The Problem: Aggressive crawlers were stealing data and causing site downtime

For any marketplace or classified ads portal, the most valuable assets are the listings. However, since the listings must be publicly available on the web to attract buyers, it’s all too easy for unscrupulous actors to scrape and steal these precious data.

“Our main classified ads portal is called Traktorpool,” explains Michael Romer, Head of Product and IT at LV digital. “It’s a marketplace for second-hand agricultural machinery and spare parts, which operates in more than hundred countries and twenty languages. We also run another very similar portal, Baupool, which targets the construction machinery market.”

“We saw our content popping up in different places on the web”, Michael continues. “So we started to trace the scrapers, for example by adding identifiers in the product descriptions, so that we could locate them and ask them to stop. However, taking legal action against scraper bot operators is a hassle. It’s not very effective, and definitely no fun.”

What made the issue even more pressing was that the scrapers crawled both Traktorpool and Baupool so aggressively that LV digital’s infrastructure was constantly under duress. Although it runs in a virtualized environment, the on-premise infrastructure doesn’t dynamically scale up and down, so there’s a limit to the load it can absorb. As a result, aggressive scrapers—even good bots—would regularly take the portals offline.

The third issue LV digital experienced was form spam. The classified ads portals enable potential buyers to get in touch with sellers via a contact form, and fraudsters targeted this form as well. Michael and his team did their homework, adding Captchas and moderating form submissions before they were sent to the seller, but it was a constant battle.

“Legal enforcement and spam moderation isn’t our core business, and it doesn’t create any value,” Michael points out. “We’re really good at bringing together buyers and sellers of machinery, that’s our job and that’s where we should put all our energy. We’re not in the business of preventing web scraping, and we learned the hard way that it’s not an easy task. No matter what we came up with, the fraudsters would find a way to work around it. We needed to find a smarter way to protect our data.”

The Solution: DataDome efficiently blocks scraper bots

To find the best solution to their bot traffic problems, Michael and his team initiated a very academic selection process, with scorecards and all the bells and whistles. Or perhaps not.

“A year or so ago, we met some people who happen to be in the web scraping business,” Michael grins. “We developed a friendly relationship with them, so they were leaving our sites alone. One day, over a beer, our scraper-operating friends told us that if we really wanted to protect our websites, we should go with DataDome. The DataDome bot protection was making their lives very hard!”

The team reasoned that if the scraper bot operators themselves were saying DataDome offered the most effective barrier, they couldn’t be wrong. “So we just went ahead without any more fuss, and we haven’t regretted it,” Michael laughs.

They quickly installed the DataDome module for Varnish, and tested the solution in free trial mode. The dashboard confirmed their suspicions: around 25% of their infrastructure resources were drained by scraper bots. Armed with hard proof, it wasn’t too difficult to secure the budget and move on to activating the protection.

“The onboarding was super smooth, an absolute joy,” Michael says. “In fact, I kept telling my boss and my colleagues how impressed I was with the process. When we set up new systems that are critical for us, it usually takes some effort to get them up and running and go through the first steps. With DataDome, it was really straightforward and fast. It took us just a couple of days to go from testing to production.”

The Results: 100% uptime and positive ROI

The benefits were immediate and tangible.

“As I mentioned, one of our problems was that we had these regular situations where crawlers were actually taking down our site,” Michael elaborates. “Since we activated the DataDome protection, it hasn’t happened once. Knock on wood, we now have 100% uptime, which was never the case before.”

Even though the team had always been aware of the aggressive scraping activity on their websites, the DataDome dashboard provided new and interesting insights.

“It’s quite eye-opening,” Micheal confirms. “Even without the protection, just seeing the figures and getting a feeling for what’s going on has a value of its own.”

At the time of writing, LV digital’s hosting plan is still adapted to pre-DataDome traffic levels. However, the next opportunity to scale down the data center capacity should enable sufficient cost savings to fully offset the DataDome subscription fees. And even without these savings, the solution has already generated a positive ROI.

“We’ve been able to come up with a new source of revenue, let me call it a data business,” Micheal reveals. “Now that web scrapers can’t just pull the information they want from our sites, we’ve been approached by several market intelligence companies asking to buy the data they need for their reports and analyses. It’s a very interesting new business model for us, and this additional revenue alone is covering the expenses we have with DataDome. So all in all, it’s been a very good investment!”

Finally, Michael and his team no longer need to spend their time protecting their portals from spiders: monitoring traffic, blocking IP addresses, and developing in-house security systems.

“We could stop doing these things from one day to another,” he notes. “DataDome freed up at least 50% of the time of one of my engineers, so from this perspective as well it pays off. That time is much better invested now: we have more bandwidth to work on new features that our customers are looking for, and that enable us to do more business.”

One person on the team still checks the DataDome dashboard on a regular basis, but mostly out of curiosity and to keep an eye on what’s going on.

“Once DataDome was set up, we could more or less forget about it,” Michael confirms. “And that’s another reason why I really like it: I don’t want to have to go there every day and check things and make tweaks and tunes and whatnot. We configured it once to meet our needs, then it’s fire and forget. It just does its job.”

He still receives the daily automated email from the system, which gives a brief summary of the previous day’s bot activity on the protected sites.

“It’s always fun to see how many bad guys we’ve been able to keep out,” he smiles. “And there’s no to-do connected to it. It’s the “good news” email of the day!”