Scrapling: Collect data from websites with Python

What it does

Scrapling is a Python library that fetches web pages with fast plain requests or a hidden (headless) browser that can render JavaScript and handle Cloudflare Turnstile checks. You then pick out parts of the page with CSS or XPath patterns, text search or regex. Crawlers called spiders visit many pages at once with retries, proxy rotation, pause and resume, and exports to JSON, CSV or XML.

How to use it

Install the Scrapling Python library, then call Fetcher.get with a page address and use page.css to pull out text. For harder sites, use StealthyFetcher to open a headless browser. It also works from the terminal, an interactive shell, or an AI agent through its MCP server, a plug-in connection for AI assistants.

Common use cases

• Give it the quotes.toscrape.com page address and a pattern for quote text, and get back a printed list of every quote on that page.
• Run the example quotes spider on a multi-page site and get every quote and its author saved together in a quotes.json file.
• Point the ShopifySpider template at a public Shopify store and get its products collected, one item per variant, ready to export as CSV.