Scrapling

Use it when

• turnstile captcha blocking scraper
• adaptive stealth fetch with cloudflare solve
• JS render and bot detection scrape
• managed challenge interrupts crawler sessions
• cloudflare challenge page kills scrape
• client-rendered pages that need JS render scrape
• logged-in scrape sessions when auth walls block anonymous fetch

What it solves

Not the fit when

• auth-wall-session without cookies
• official-API-available-prefer-API

Install

pip install scrapling

Invoke

from scrapling.fetchers import Fetcher, DynamicFetcher, StealthyFetcher
# static: Fetcher.get(url)
# JS: DynamicFetcher.fetch(url, network_idle=True, wait_selector='.card')
# CF/Turnstile: StealthyFetcher.fetch(url, solve_cloudflare=True)

Alternatives

curl-impersonate, camoufox, cloudflare-bypass-for-scraping, playwright