Over the past two decades, many websites have tried several solutions to block bots that advertise click-fraud, harvest information or download files. But the most widely used weapon against bot activity, the CAPTCHA, has arguably created more problems than it solves.
Classic CAPTCHAs use mangled text or image-matching to establish the "humanity" of website visitors. They may annoy or frustrate users and, in the case of e-commerce sites, lead to lower conversion rates. They also don't work well on mobile devices, a major hurdle when a large share of online shopping is done on smartphones and tablets.
Some of the more recent iterations, such as the current version of Google's reCAPTCHA, use hidden cookies that may raise privacy issues. Phony CAPTCHAs have been used in malware campaigns.
CAPTCHAs are also less useful than they once were. Bots have gotten better at solving them, while humans toiling in overseas "CAPTCHA farms" are paid fractions of a cent for each CAPTCHA solved. AI that can recognize text and images now makes CAPTCHAs even less of an obstacle to bots.
That's why it's important to move beyond CAPTCHAs and use a human-verification solution that neither baffles users nor invades their privacy. The Dynamic Challenges feature of content-delivery network Fastly's Bot Management tool uses a three-tiered framework to achieve this.
How Dynamic Challenges works
Based on the context of a client device's request for a web-based resource, Dynamic Challenges chooses the appropriate response.
If the request comes from a late-model (2017 or later) Apple device, Dynamic Challenges uses the Private Access Tokens protocol to ask Apple to verify the device's legitimacy. If Apple responds affirmatively, Fastly grants the access request and also issues a verification token that the device can use when visiting other Fastly-protected sites. This entire process takes less than a second and is invisible to the user.
If the access request comes from a browser that supports JavaScript but is not running on an Apple device, then Dynamic Challenges uses a Proof-of-Work test that requires the browser to solve a math problem. To the average user, this creates a delay of a couple of seconds, but to bots that are rapidly trying to access thousands of websites at once, it's unsustainable.
Finally, if the request comes from a browser that neither supports JavaScript nor runs on an Apple device, or if Fastly's Dynamic Challenges system thinks the request is coming from a bot, the user will be asked to revert to the old ways of doing things and solve a CAPTCHA.
"When challenges have been initiated and solved, cookies are issued from the customer domain in which the challenges are issued," granting access to the domain, states a Fastly technical document regarding Bot Management.
How these methods stop bots
These methods effectively screen out bot traffic while presenting minimal friction to the user. The Private Access Token protocol relies on Apple only to verify the client device's legitimacy and communicates no other details to the website being accessed.
"When you put this together, no one entity can link client identity to website activity," according to a Fastly blog post detailing Private Access Tokens. "And yet, this authorizes access to a website — all while eliminating human interactions."
Likewise, the proof-of-work challenges presented by Dynamic Challenges' second option don't "fingerprint" browsers and pose no threat to user privacy.
Operators of websites using Fastly's Bot Management can adjust the parameters of Dynamic Challenges to suit their own needs. The website operator may want to screen requests coming from a specific range of IP addresses, for example, or may want to present more challenges during periods of unusually high traffic.
In the words of a Fastly worksheet, Dynamic Challenges "prioritizes the least intrusive traffic validation for legitimate users while posing the most difficult challenges to clear bots."
