Crawlbase introduces Web Bot Auth as an identity layer for bots, coinciding with pay-per-crawl pricing

Crawlbase ships Web Bot Auth, a specification that treats bot identity as a verifiable signature rather than a self-declared claim, on the same day it launches pay-per-crawl pricing.

Markdown twin JSON

Anti-bot & Blocking Primary source launch / significance 3

Briefing

Crawlbase: Crawlbase ships Web Bot Auth, a specification that treats bot identity as a verifiable signature rather than a self-declared claim, on the same day it launches pay-per-crawl pricing.

Why it matters

By shifting bot identity from a claim to a cryptographic signature, Web Bot Auth could make it harder for malicious actors to impersonate legitimate crawlers, raising the bar for anti-bot systems. The simultaneous launch of pay-per-crawl pricing suggests Crawlbase is betting that verified identity will become a premium feature in the extraction ecosystem. For practitioners, this means bot identity may soon be baked into infrastructure rather than handled ad hoc, potentially simplifying compliance with site policies.

Sources

Primary source · Crawlbase · 2026-08-28 — The Web Is Growing an Identity Layer for Bots

Watch next

Will major anti-bot vendors adopt Web Bot Auth as a standard, or will it remain a Crawlbase-specific feature?

Topics: Crawlbase, bot-identity, web-bot-auth, pay-per-crawl, cryptographic-signature, anti-bot