Free
7,628 uses
Build a reliable web scraper with proper rate limiting and error handling.
Prompt
Write a web scraper in [Python/Node.js] to extract [data type] from [website type]. Data to extract: [list of fields]. Technology: [BeautifulSoup/Scrapy/Playwright/Puppeteer]. Requirements: handle pagination (up to [X] pages), rate limiting ([X] requests/second), retry on failure with exponential backoff, handle JavaScript-rendered content (if needed), rotate user agents, save data to [CSV/JSON/database], handle IP blocking gracefully, and include a progress bar. Respect robots.txt and include ethical scraping comments.
Paste directly into ChatGPT · Replace text in [brackets] with your own info
Free template suggestion
If this prompt is close but not exactly what you need, leave a topic suggestion. We use repeated requests to expand the free prompt library.
Best for readers who want the free library to cover a specific workflow or niche.
Before you use this prompt
This free prompt is a starting point, not a finished deliverable. Review the result, remove sensitive information, and adapt the wording before you publish, send, or rely on it.
✓ Replace every bracketed placeholder with your own context before running it.
✓ Remove private customer data, passwords, personal addresses, and unpublished business details.
✓ Check facts, numbers, claims, and links before publishing or sending the output.
✓ Edit the tone so it matches your brand, audience, and local market.
✓ Do not present AI output as legal, medical, financial, or guaranteed professional advice.
✓ Save the best version so you can reuse and improve it next time.
Enter your topic — AI fills the prompt and shows a live sample
See what others created with this prompt
No results yet — be the first to share!
Explore related templates
If this prompt is useful, the fastest next step is to open a few related templates, compare the wording, and keep the version that best matches your workflow.