Free
7,628 uses
Build a reliable web scraper with proper rate limiting and error handling.
Prompt
Write a web scraper in [Python/Node.js] to extract [data type] from [website type]. Data to extract: [list of fields]. Technology: [BeautifulSoup/Scrapy/Playwright/Puppeteer]. Requirements: handle pagination (up to [X] pages), rate limiting ([X] requests/second), retry on failure with exponential backoff, handle JavaScript-rendered content (if needed), rotate user agents, save data to [CSV/JSON/database], handle IP blocking gracefully, and include a progress bar. Respect robots.txt and include ethical scraping comments.
Paste directly into ChatGPT · Replace text in [brackets] with your own info
Enter your topic — AI fills the prompt and shows a live sample
See what others created with this prompt
No results yet — be the first to share!