Skip to content
Get packs

Data Scraper Agent

Build a scheduled agent that scrapes public sources, enriches data with a free LLM, and stores results.

Source Download skill
Repository
affaan-m/ECC
Stars
295,127
Licence
MIT
Files
1 file

What it does

This skill teaches an AI agent to build a small automated data-collection bot. The bot visits public websites or APIs (such as job boards, prices, news, or GitHub repos) on a schedule, uses a free Gemini Flash language model to summarise and classify what it finds, and saves the results to Notion, Google Sheets, or Supabase. The skill also describes adding a feedback loop so the bot learns from the user's choices over time.

Great for

  • Developers who want to monitor public pages or APIs without paying for hosting
  • Analysts or researchers tracking job postings, prices, or news on a recurring basis
  • Users who want a simple automated watcher that stores results in a spreadsheet or database

Skip it if

  • Scraping sites that prohibit automated access in their terms or robots.txt, since the skill does not address legal or ethical checks
  • Sensitive or private data, because results are passed to a third-party LLM and stored in external services
  • Non-technical users who need a finished app, since the skill only guides an agent through building one

Try asking

  • “Build a bot that checks a job board daily and summarises new postings into a Google Sheet”
  • “Monitor GitHub repository releases each week and classify them with Gemini Flash”
  • “Track product prices on a public store and save changes to Supabase”
  • “Set up a GitHub Actions schedule that collects news headlines and stores them in Notion”
Step-by-step guide: how to use Data Scraper AgentInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 1 original files, the MIT licence and an attribution note.

    data-scraper-agent.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip data-scraper-agent.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip data-scraper-agent.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Check each target site's terms of service and robots.txt before building a scraper
  • Store API keys and database credentials as GitHub Actions secrets, not in code
  • Start with one small source and a single storage target before scaling up

What's inside

  • SKILL.md

Redistributed unmodified from affaan-m/ECC under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.