View Ends Data Theft & Reduces Infrastructure Costs With DataDome

Efficient protection of listings and attribution

15-20% infrastructure cost reduction

“Set it and forget it” protection on autopilot

Marketplaces & classifieds Scraping

View is one of Australia’s leading property portals. First launched in 2001, the website lists hundreds of thousands of properties across the country. Like most data-rich portals, it was plagued with intensive scraper bot activity and content theft. Today, the DataDome bot protection solution efficiently blocks scraper bots, keeping View’s valuable listings safe and reducing infrastructure costs in the process.

The Problem: Intensive Scraping of Real Estate Listings

Snehal Tiwari is Tech Lead/Solutions Architect at View. His first indication of malicious activity on the website was that the number of requests in his web server logs didn’t match the data he saw in his analytics tools—there was a significant gap.

“When we investigated that discrepancy, we discovered an intensive scraping activity on our site”, says Snehal. “There’s a lot of information in our listings, and certain people wanted to collect that information and host it on their own websites.”

Unlike many other targets of website scraping, View rarely experienced poor website performance and downtime.

“We provision our infrastructure with spare capacity for peaks”, Snehal explains. “Most of the scraping was also happening late at night, when there is minimal human traffic. There wasn’t generally any intention to harm our site; we didn’t have denial of service attacks or anything like that. Nonetheless, if I had to make an estimate, I’d say that bots represented somewhere between 20 and 40 percent of our total traffic.”

Like any portal bringing together buyers and sellers, View’s most important assets are its listings.

“Our data has a lot of value,” Snehal points out. “If someone steals a listing and promotes it as their own, we can lose the attribution for marketing that property. It’s a real business risk. And when we signed an agreement with a new data partner who requested scraping protection as part of the deal, the time had come to put an end to it for good.”

We needed a solution up and running fast, and DataDome’s onboarding team was extremely helpful. It took less than a week all in all to get the solution online.

Snehal Tiwari

Tech Lead/Solutions Architect at View

The Solution: Easy Integration & No Traffic Rerouting

When Snehal and his team started looking for a bot protection solution, performance was the number one priority.

“Obviously, the solution had to efficiently defend us against scraper bots”, Snehal observes. “Once that requirement was met, the next criterion on our list was ease of integration: what effort we would have to do on the hosting infrastructure on our end. And finally, service reliability is key for us. With new data partners and stricter contractual agreements, we now have a very low tolerance threshold for scraping. We simply can’t have a situation where the service isn’t available.”

After thoroughly researching the bot management software market, Snehal had pared his shortlist down to two solutions: DataDome and one other leading bot protection solution vendor.

“We needed a solution up and running fast, and DataDome’s onboarding team was extremely helpful,” he says. “It took less than a week all in all to get the solution online, and it could probably have been done in two or three days. We just took a little extra time to do some additional testing; for instance, we tried to scrape our site ourselves. But it’s a very well designed solution, which was quick to integrate with our platform.”

Another feature he liked about the DataDome solution is that it doesn’t require any traffic redirection. The DataDome Apache module analyzes every request to View’s servers, making a call to the nearest DataDome Edge Point of Presence (there are 25+ PoPs worldwide) before the Apache process runs. If the algorithm determines that the user is a human, it lets Apache continue the regular process. If it’s a bot, it’s either presented with a CAPTCHA challenge or hard blocked.

“The traffic hits our servers first, and we control which traffic we ask DataDome to filter,” Snehal comments. “The other solution we evaluated didn’t have that as a feature, it needed more of an integration effort.”

Thanks to DataDome, I would estimate that we’re saving around 15-20% infrastructure cost.

Snehal Tiwari

Tech Lead/Solutions Architect at View

The Results: Protected Listings & Peace of Mind

Today, Snehal no longer worries about View’s listings being stolen. “That’s my number one metric for success: the thing works, our listings are protected and we get the attribution which is due,” he says.

“It’s also fast”, he continues. “When a request hits our servers, the DataDome module’s server call and response happen in real time, and it’s completely transparent for our human users. Our listings are protected, and so is the customer experience.”

Even though it wasn’t an objective in itself, reducing the infrastructure cost by removing unwanted bot traffic has been an interesting side benefit.

“It’s not linear of course; 40 percent bot traffic doesn’t mean 40 percent extra cost. But I would estimate that we’re saving around 15-20 percent infrastructure cost”, Snehal observes.

To perfect the protection, he has taken advantage of the solution’s Custom Rules feature. “We need to share data from certain specific endpoints with some of our clients and partners”, he explains. “There should be no human traffic there, and the custom rules function makes it very easy to exclude that traffic to those endpoints.”

But on a daily basis, Snehal no longer needs to think about bot traffic at all. “I go to the DataDome dashboard about once a month to get some statistics for our partner, and very occasionally to change a configuration”, he notes. “And I have an inbox rule which brings the daily email reports to my attention if the subject line or the content contains specific keywords. But other than that, DataDome works without me having to do anything. It really is a ‘set it and forget it’ solution, a great product with great support, and the DataDome team is a joy to work with.”