Bots: Separating the Good from the Bad
Do you feel like you are playing a game of cat and mouse by blocking ranges of IP addresses hitting your servers more than they should? Sometimes the culprit is an attacker implementing a DDoS attack, sometimes it’s a scraper downloading your content, and other times it’s a script sending search or auto-login requests. These harmful automated scripts called bad bots degrade the performance of your website and ruin the perfect experience you want for your real users. They are messing up your user analytics data, sometimes to the point where you can no longer trust the numbers.
Today we are launching “Bot or Not,” a new product that differentiates bad bots from good bots and legitimate visitors. Bot or Not complements our AppShield offering by fighting against fraudulent and malicious bots, and our AppSpeed Performance offering by reducing unwanted traffic and increasing your website’s performance.
A long time ago, when changing IPs was pretty costly for bots, and proxies were not so widely available, IP blocking was the best tool for fighting these unwanted visitors. Today it is easy and cheap to switch an IP, so blocking IP addresses no longer suffices as a way to fight bad bots.
Frequent requests from bots not only consume a lot of bandwidth, but also overwhelm origin servers, causing performance degradation. Performance degradation results in lost users and lost revenue, but beefing up your infrastructure to respond to unwanted requests can be costly. Bot or Not is a less expensive, more robust solution that helps stop these unwanted visitors in an automated fashion.
Bot or Not is built on top of our unique endpoint-aware technology that does behavioral fingerprinting of the client to help categorize requests as coming from a bot or coming from a human. The fingerprints are fed into algorithms to determine if a request to your origin server is a normal user request or if it originated from a bot, either good or a bad. The end-to-end system then allows you to block or throttle bad bot traffic before it gets to your origin servers.
What has been your experience with bots? What are some methods you have used that worked well and what methods did you use that failed? Let us know if you want to get started with our Bot or Not technology. With no needed code changes on your part, it is so easy to deploy.