Crawler Detect
Crawler Detect Crawler Detect Crawler Detect Crawler Detect

The open-source standard for bot detection. Identify crawlers, spiders, and bots from any user-agent string. Three methods, zero dependencies, one Composer install.

CrawlerDetect is an open-source PHP library created by Mark Beech (JayBizzle) that serves as a standard for identifying crawlers, spiders, and bots from user-agent strings.

The main features and capabilities of CrawlerDetect include:

  • Zero Network Overhead: Performs pure regular expression matching against a local signature database on your server, eliminating external API calls, latency penalties, or third-party network dependencies.

  • Extensive & Updated Signature Database: Maintains a community-driven repository of thousands of known bot, spider, and crawler signatures—including modern AI agents—that is continuously updated.

  • Streamlined Three-Method API: Features an intuitive, minimal surface area consisting of just three methods: isCrawler(), getMatches(), and setUserAgent().

  • Zero-Configuration & Broad Ecosystem: Installs quickly via Composer without requiring setup or configuration files. It operates standalone, integrates natively with frameworks like Laravel, Symfony, and Yii, and offers community ports for Node.js, Python, Ruby, Go, .NET, and the JVM.

  • Lightweight & Battle-Tested: Built as a single PHP class focused specifically on accurate bot detection, supported by over 114 million total downloads and active open-source maintenance.