/automation--web-scraper

/automation--web-scraper is a Claude Code skill in the Code section. Builds a scraper that checks robots.txt and terms of service, throttles requests, handles pagination and saves results so runs can resume.

Author's description

Build a polite web scraper — robots.txt, rate limits, selectors, pagination, storage

How to ask for it

/automation--web-scraper build a scraper that respects this site's robots.txt
/automation--web-scraper extract prices from this site across paginated results
/automation--web-scraper store the scraped data in a database

Install

curl -fsSL https://raw.githubusercontent.com/sgomez-dev/claude-skills/main/install.sh | bash

After installing with the script, type /automation--web-scraper. Using Cursor, Windsurf or Codex? Platform guides

Demo coming soon

/automation--workflow-automationn8n, Zapier and Make workflows
Designs an n8n, Zapier or Make workflow with trigger, steps and error handling, delivered as an n8n JSON export or a build sheet.
Built here
/cloud--autoscaling-strategyAutoscaling strategy
Designs a service's autoscaling: scaling metric, bounds, cooldowns and cold-start handling, validated with a load test.
Built here
/cloud--aws-architectAWS architecture design
Designs an AWS architecture for your app with a minimal set of services, a Mermaid diagram and a Terraform starter.
Built here
/cloud--azure-architectAzure architecture design
Designs an Azure architecture for your app with a minimal set of services, a Mermaid diagram and a Terraform or Bicep starter.
Built here
/cloud--cloud-cost-auditCloud cost audit
Analyses billing exports and infrastructure code to find wasted spend and returns a prioritised savings plan with estimated monthly savings.
Built here
/cloud--cloud-migrationCloud migration plan
Creates a phased migration plan with an inventory, a 6 Rs strategy per component, cutover and rollback steps, and estimated costs.
Built here

How to ask for it

Install

More from this section