About a fifth of all web traffic (20.4 percent) comes from bad bots, which continue to attack daily in automated offensives on websites, mobile apps and APIs. That’s worse for some verticals, like the banking and finance sector, which was hit the hardest last year.
That’s according to the Distil Research Lab’s latest Bad Bot report, released Wednesday, which also found, in an analysis of hundreds of billions of 2018 bot requests, that 73.6 percent of bad bots can now be classified as “advanced persistent bots” (APBs).
While the levels of bad-bot traffic have actually slightly decreased year-over-year, the rise of greater sophistication should be of note, the report said.
Advanced Persistent Bots
Bad bots are endemically used by malicious competitors, hackers and fraudsters to carry out all forms of internet cyberattacks – including account takeovers, brute-force attacks and hijacking, web scraping, competitive data mining, transaction fraud, data theft, spam, digital ad fraud and distributed denial-of-service (DDoS) attacks.
APBs kick the story up a notch by adding the ability to cycle through random IP addresses and switch user agents, which renders simple IP blacklisting wholly ineffective; enter through anonymous proxies; and change their identities and mimic human behavior in believable and craft ways.
“Bot operators and bot defenders are playing an incessant game of cat and mouse, and techniques used today, such as mimicking mouse movements, are more human-like than ever before,” said Tiffany Olson Kleemann, CEO of Distil Networks, in a media statement.
Known as “low and slow,” APBs also carry out significant assaults using fewer requests and can even delay requests, all the while staying below request rate limits. This method reduces the “noise” generated by many bad bot campaigns.
Meanwhile, the increasing volume of stolen credentials from data breaches (14.7 million in the last five years) is also creating a worsening bot problem for any online business having a login page.
“Bots are used by criminals to test the viability of stolen credentials,” according to the report. “Every new data breach sees an increased availability of credentials and leads to higher volumes of bad bot traffic. With over 14 billion credentials stolen since 2013, the problem is already significant—and only getting worse.”
Vertically Inclined
And as sophistication increases, so does the breadth of industries impacted by bad bots. Activities vary by vertical: In the hard-hit financial services realm, where 42.2 percent of traffic originates from bad bots, the activity consists mostly of credential-stuffing to access or take over user accounts.
In the world of events and ticketing, (39.3 percent of traffic is bad-bot related), scalping bots, seat inventory checkers and credential stuffing were observed. Education (37.9 percent of traffic) suffers from scraping efforts for research papers, class inventory and user account access attacks; and government (29.9 percent of traffic) sees voter registration account interference and the scraping of business registration listings.
Other industries aren’t as heavily impacted by bad bots. In gambling and gaming, where about a quarter of traffic is bad-bot driven (25.9 percent), criminals are scraping ever-changing betting lines and carrying out account takeovers in an effort to gain loyalty points.
Airlines also see 25.9 percent of traffic from bad bots, which are scraping pricing information and attempting account takeovers to empty airline mile balances.
And finally, e-commerce is a surprisingly low-targeted realm, with only 18 percent of traffic coming from bad bots. There, price scraping, content scraping, account takeovers, credit-card fraud and gift-card abuse are the main activities. “While bot activity on industries like airlines and ticketing are well-documented, no organization – large or small, public or private – is immune,” said Kleemann. “When critical online activity, like voter registration, can be compromised as a result of bad bot activity, it no longer becomes a challenge to tackle tomorrow. Now is the time to understand what bots are capable of and now is the time to act.”
Interestingly, nearly half (49.9 percent) of bad bots are using the Chrome browser, according to the analysis. Use of mobile browsers, such as Safari Mobile, Android and Opera meanwhile increased from 10.4 percent last year to 13.9 percent. And, Amazon is the leading ISP for originating bad bot traffic, with its “market share” spiking last year.
The use of data centers meanwhile reduced in 2018 with 73.6% of bad bot traffic emanating from them—down from 82.7% in 2017.
In terms of geography, a little over half of bot traffic (53.4 percent) originates from the United States. Also, Russia and Ukraine combined make up nearly half (48.2 percent) of country-specific IP block requests.
Meanwhile, the activity of good bots, used by search engines to crawl websites, check links, retrieve content and update their indexing, decreased slightly to make up 17.5 percent of traffic.
Don’t miss our free Threatpost webinar, “Data Security in the Cloud,” on April 24 at 2 p.m. ET.
A panel of experts will join Threatpost senior editor Tara Seals to discuss how to lock down data when the traditional network perimeter is no longer in place. They will discuss how the adoption of cloud services presents new security challenges, including ideas and best practices for locking down this new architecture; whether managed or in-house security is the way to go; and ancillary dimensions, like SD-WAN and IaaS.
