Scrapy 2.18.0 ships with httpx2-based download handler and stable HTTP/2 support
Scrapy 2.18.0 replaces the Twisted-based HTTP/2 handler with httpx2, makes brotli and Zstandard compression always available, and changes late crawler attribute access to raise RuntimeError.
Briefing
- Scrapy: Scrapy 2.18.0 replaces the Twisted-based HTTP/2 handler with httpx2, makes brotli and Zstandard compression always available, and changes late crawler attribute access to raise RuntimeError.
Why it matters
The move to httpx2 for the download handler modernizes Scrapy's HTTP stack, potentially improving performance and maintainability. Making HTTP/2 non-experimental signals production readiness for a feature that many scrapers will rely on. The RuntimeError change for late crawler attributes is a breaking change that may affect custom extensions or middlewares, so developers should review their code before upgrading.
Sources
- Primary source · Scrapy · 2026-08-20 — scrapy/scrapy 2.18.0
Watch next
Will the httpx2 migration introduce any compatibility issues with existing Scrapy middlewares or download handlers?