The online landscape is shifting dramatically, with data or content scraping ascending to the top of the list of bots inflicting damage on legitimate businesses. Scraped data is being taken and used for competitive or other motives without the scraped company benefitting in any way. In fact, the impact on companies targeted by scraping can be devastating.
High-profile companies like JP Morgan Chase, LinkedIn, Facebook, YouTube, Venmo and others are seeking legal recourse to stop it. While your company may not be the size of these household names, don’t be fooled into thinking that scraping isn’t hurting your business right now.
If you’re a business leader with significant online-driven revenue — financial services, retail, travel and hospitality, gaming, real estate and other industries — take a few minutes to read these questions and answers to make sure you’re up to speed on the rampant problem of scraping and the potential repercussions for your business.
Question #1: Who is scraping content and why?
The short answer is everyone. Chances are good that more than half of your traffic is non-human, with content and price-scraping bots making up a significant portion of that traffic.
Content scraping software (do-it-yourself) and services (software-as-a-service or data-as-a-service) are big business today. Most vendors in this space offer what’s known as “web data extraction.” These companies extract web data for everything from clothing to electronics, job postings to hotel listings, cars to real estate, and many other types of content. The data, images, prices and more on your website are fair game.
What do companies use scraped data for? The list of potential uses is lengthy, but some of the malicious uses include share price manipulation, price undercutting, search engine manipulation, data theft, and brand damage.
Question #2: How are they scraping my data?
Automated bots interact with your web applications to “scrape” or extract the data that your website presents to a human user in a browser. Sometimes they fill shopping carts to discover total pricing, including shipping and discounts. Then they abandon those shopping carts. Imagine thousands of bot-driven shopping carts full of your merchandise just standing idle, with purchases never to be completed.
Question #3: Isn’t content scraping illegal?
Most companies are not okay with their data being extracted and monetized by third-party companies, nor do they want to pay the additional computing costs incurred by the constant scraping activities of automated bots on their website.
However, content scraping remains a legal gray area. The blockbuster decision in hiQ v. LinkedIn seemed to side with the scrapers, but the fight isn’t over yet, with LinkedIn vowing to take the case to the Supreme Court if necessary.
Question #4: Why should we worry about scraping bots on our site?
Aside from ethical and legal concerns, the bottom line is that being a victim of scraping can hurt your bottom line.
If competitors are using your data to gain a competitive advantage, you’re losing customers and revenue. When scraper bots fill shopping carts and then abandon them, it ties up your inventory and prevents legitimate customers from purchasing your products. If your competitors publish your content, they can potentially rank higher on search engine results, reducing your volume of search-generated traffic and limiting your ability to convert those shoppers into customers. Lost ad revenue is another major financial loss caused by scrapers stealing your data.
Even companies that claim to obey a website’s terms of service, relevant copyright laws, and otherwise act in an ethical way often have a negative impact on the websites they target. They can create excessive load on the websites, slowing down response times and negatively affecting the experience of legitimate customers.
Question #5: How can we prevent scrapers on our site?
While updating your terms and conditions to prohibit scraping is a good start, it won’t stop most automated bots from stealing your data anyway. And unless you’re willing to dedicate an army of people to constantly monitor and defend your website, you need an advanced bot mitigation solution that is easy to use and automates as much of the effort as possible to reduce the burden and lower your cost of ownership.
Want to learn more about the dynamics of automated attacks? Check out our blog article on the secrets of mitigating against bots and detailed steps organizations can take to safeguard their web applications. Read article here.
If you're ready to start fighting malicious automation and want to see the Kasada solution in action, schedule a demo here.