crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy ro
- apify
- automation
- beautifulsoup
- crawler
- crawling
- headless
- headless-chrome
- pip
- playwright
- python
- scraper
- scraping
- web-crawler
- web-crawling
- web-scraping
- Stars
- 9,581
- Forks
- 808
- + today
- +1
- Created
- 2y
Ranking data as of October 7, 2026 (UTC).
Star History
Today, hour by hour
Overview
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy ro It ranks #1464 on GitTiger, gaining +1 star on October 7, 2026 (UTC).
The project is written in Python and has 808 forks. It was created 2y ago.
git clone https://github.com/apify/crawlee-python.git
cd crawlee-python
# see README for setup