What is a bot?
So, what is a bot anyway? Personally, I think it’s an overused, often misunderstood and mostly useless term.
Technically I guess a bot “runs automated tasks by executing scripts over the internet”?
But that’s really only telling half the story. It only tells you what it is technically, but we really care about it's purpose. The true focus should be on why there is a human that actually bothered to build this bot.
Humans, code and an ecosystem
A better definition: "A bot is a combination of technologies that humans use to automate interactions with applications". But it is more than that. The human characteristics need to be considered in more detail: the technical skills, the motivation/intent, the reward/result.
Why are these people doing this? Let's take two examples at polar opposite ends of the spectrum:
- Google crawler (a good bot) that has changed the way we use the internet and consume knowledge .
- The “enumeration attack” against the Australian PayID platform (a very bad bot).
Two different pieces of automation, two very different human motives.
Good bots, bad bots, I’m not sure bots.
Good bots: Automation plays a huge role in our daily lives. Google, Siri and Alexa are great examples of software tools automating processes that improve our daily lives.
Side note: Stop reading this blog right now and try asking Siri if she is lonely. Bots can also be humourous! 😁🤖
Bad bots: But not all bots are good. Just like in the real world, bad people leverage technology to achieve bad things. There are well established tools and ecosystems to help bad people achieve their nefarious actions. Validating stolen credit cards, validating stolen user credentials, e-commerce fraud, checkout fraud, fake account creation or even denial of service. The options are only limited by a fraudsters’ creativity, access to tech skills and funding.
I'm not sure bots: There are also bots that are neither good nor bad.
- Legitimate businesses tracking their competitors.
- Industry analysts and investment firms collecting alternate data sources.
- Aggregators logging into rewards programs or banking applications
- Individuals automating personal projects
In these examples, and thousands of others, the entity generating the automation is doing so for their gain. The subject of the automation is most likely not okay with their data being taken, or the infrastructure costs incurred with providing that data. In many cases, the owner of the data has developed a broader understanding of its value and/or has monetised access to it. Thus they view the automated access to their data as being a significant business issue.
Who are these humans and what are they doing with their lives?
Let’s consider a few bot builder personas:
Sophie is well educated in the world of automation and enjoys leveraging her professional skills to automate personal tasks. Sophie’s to-do list in 2020 includes buying a car, renting a new house and conducting some "innocent arbitrage" to ensure she gets the best odds when she places a bet.
Gill's job is to analyse the global property market. Whilst Gill’s business has acquired data via many traditional sources, they have asked her to source additional “alternative data” sources. Gill has a list of the main real estate listings websites in her target markets and is building a web scraper to extract her data on a weekly basis.
Ilovesneakers is the online handle for an anonymous sneaker lover. Ilovesneakers runs a bot that helps people buy limited edition sneakers from the big brand names. Ilovesneakers has built an entire ecosystem to provide a premier service to his subscribers.
Aleksei – sells stolen credit cards from a website known as Cardplanet. Aleksei increases the value of his dataset by running a bot that validates each card. This allows Aleksei to increase the return on his investment by a factor of 5-10.
Understanding a bot builders barrier to entry
Whilst all four of our persona’s need to go through a similar process, the planning phase is significantly more complex for Aleksei, ilovesneakers and Gill.
Aleksei needs to cover his tracks, his project is illegal and he doesn't want to end up in jail. (spoiler alert, Aleksei ends up in jail).
Ilovesneakers knows that the major sneaker brands will try to prevent his bot from accessing their limited edition auctions.
Gill also knows that some of her primary sources of alternate data will not take kindly to her actions.
This increases the effort required to successfully launch their respective projects. The bigger and more complex the project, the more organised these bot builders need to be.
This process follows the same methodologies as other any software development process. Define, Design, Implement, Test, Launch, evolve.
Tasks on a bot builder's to-do list:
- Define the project
- Build the set of requirements
- Select coding language or packaged software solution of choice
- Build scripts that are specific for the required task
- Integrate in the necessary bolt on evasion solutions (proxy, header rotation, captcha evasion, etc)
- OR: invest in an ecosystem (eg: Linken Sphere), outsource code (www.freelancer.com), etc
- Identify target organisations and perform reconnaissance
- Customise the bot for each target application
- Test each deployment ensuring that the bot can extract the necessary data
- Move to production
- Maintain the operation
The process of building this project is no different to a normal software project. Whislt Sophie can simply get a script from github, our other bot builders need to be far more sophisticated.
So, what is a bot?
A bot is a set of tactical tools, used by people, that achieves a strategic goal. The strategic goal is the most part important part of this definition. Businesses need to truly understand and study the mindset of these bot builders in order to properly defend against their attacks. Staying head of their next step is also key, bot builders will 'retool' on a regular basis if they are not meeting their strategic goal. They come back more intelligent, tactical and more sophisticated to bypass the barriers.
In next week's blog we will compare Sophie and Aleksei's experience as they set about launching and running their bots.
To learn more about advanced bot detection, read our blog post covering the most important components of non-static bot protection. Read full post here.