Guide · Agent skill
How to use the Job Scraper agent skill (scrape)
Updated · written with AI from the project's own documentation and checked against our catalogue data
The Job Scraper is an agent skill, a packaged set of instructions and scripts that an AI agent such as Claude can load and follow. This guide is for job seekers who want their agent to look for new postings on a regular basis and show only jobs they have not seen before. By the end you will know what the skill does, how to install it, how to ask for it in plain language, and what it needs to run.
Job Scraperai-job-search-dev/ai-job-search · MIT Download the skill Quick start
- 1Download the skill. Download the ZIP from the skill's page on this site and unzip it.
- 2Place the skill folder. Move it into ~/.claude/skills/ for personal use, or into the project's .claude/skills/ folder.
- 3Prepare the repository. Make sure Python 3, the portal-search CLI skills under .agents/skills/, tools/job_key.py, tools/scrape_state.py, your profile and search-queries.md are in place.
- 4Set up search queries. Edit search-queries.md so it reflects the roles, skills and languages you want.
- 5Ask for new jobs. Say "Find new jobs" or type /scrape in your agent.
- 6Review the results. Read the new matches and the fit note on each one; expired postings are kept out of the list.
What the Job Scraper skill makes the agent do
When you ask your agent to find new jobs, this skill has it search job portals such as LinkedIn and other local job boards. It uses the search terms in your profile and looks only at postings from roughly the last 14 days.
The skill keeps a memory of jobs already seen and of your application tracker, so it shows only new matches. Each match comes with a short note on how well it fits you. Text returned by web search is treated as untrusted content, not as instructions.
- Searches portals through their installed CLI skills, with web search as a fallback
- Deduplicates against earlier runs and your application tracker
- Adds a quick fit assessment to each new result
What you need before installing
The skill depends on parts of the ai-job-search project. Check these before you start, because the skill falls back to web search or stops without them.
Python 3 must be available as python or python3. bun is needed for the portal CLI tools; if it is missing, the skill uses web search instead. The repository also needs the tools tools/job_key.py and tools/scrape_state.py, a profile, a search-queries.md file, and portal-search CLI skills installed under .agents/skills/.
- Python 3 (python or python3)
- bun, for portal CLI tools (optional; web search fallback if missing)
- Portal-search CLI skills installed under .agents/skills/
- tools/job_key.py and tools/scrape_state.py from the repository
- A profile and search-queries.md in the repository
Install the skill
Download the ZIP from the skill's page on this site and unzip it. Then put the skill folder in one of two places. For all your projects, use ~/.claude/skills/. For one project only, use the .claude/skills/ folder inside that project.
If you use the Claude apps instead, open Settings, go to Capabilities, and upload the skill under Skills.
- Personal install: unzip into ~/.claude/skills/
- Project install: unzip into the project's .claude/skills/ folder
- Claude apps: Settings → Capabilities → Skills, then upload
Trigger the skill with natural requests
You do not need special syntax. Ask in plain language and the agent will recognise the request. The skill also responds to the slash command /scrape.
You can narrow a run with a focus area, or ask for a broad run that covers all search categories. A health check can be run on its own to see whether portals are responding, without searching or presenting jobs.
- "Find new jobs"
- "Scrape for jobs"
- "Any new positions?"
- "/scrape"
- "/scrape data science" (focus area)
- "/scrape broad" (all search categories)
- "/scrape health" (portal health check only)
- "/scrape health jobnet" (probe one portal even if disabled)
What a run looks like
By default the skill runs the top three priority query categories from search-queries.md. It checks the job against seen jobs and your tracker, fetches details for promising hits, and then presents only the new matches.
Some portal results carry an expired flag when the posting page shows it is no longer accepting applications. These are recorded but not shown to you. Deadlines and dead links are handled by the separate /rank step.
- Default: top 3 priority query categories
- Focus area: prioritises queries in that category
- Broad: runs all categories
Safety notes: scripts and network access
This skill runs scripts and uses the network. It runs the Python helpers tools/job_key.py and tools/scrape_state.py, and it calls the portal CLI tools and web search. Job titles, companies, descriptions and URLs from these sources are treated as untrusted data. Links inside a description are not followed.
Scraping job sites may breach some portals' terms of use. Check the terms of each portal you enable.
- Runs local Python scripts
- Makes network requests to job portals and web search
- Portal text is untrusted; embedded links are not followed
Tips for better results
Keep your profile and search-queries.md up to date, since the skill searches with those terms. Write queries in every language listed in your profile's language table, because postings requiring an undeclared language are excluded.
Use a focus area when you want a narrower list, and a health check when results seem thin.
- Update search-queries.md when your target roles change
- Declare all working languages in your profile
- Use a focus area for narrower runs
- Run a health check if a portal returns nothing
What people use it for
- Getting a recurring list of new postings without searching each portal by hand
- Searching a focus area such as data science in one run
- Checking whether portals are responding with a health-only run
- Avoiding repeat listings across several days of searching
Frequently asked questions
How do I install the Job Scraper agent skill?
Download the ZIP from the skill's page and unzip it into ~/.claude/skills/ for personal use or into a project's .claude/skills/ folder. In the Claude apps, you can instead upload it under Settings → Capabilities → Skills.
What do I need to run the job scraper skill?
You need Python 3 (as python or python3), the portal-search CLI skills under .agents/skills/, the tools tools/job_key.py and tools/scrape_state.py from the repository, and a profile with search-queries.md. bun is needed for the portal CLIs; without it the skill uses web search.
Is the Job Scraper skill free?
The skill is licensed under the MIT licence. The source material does not mention pricing, so check the project page for any other terms.
Does the job scraper apply to jobs for me?
No. The skill only finds and assesses postings. It does not submit applications.
Why does the skill show no results?
Common causes are missing portal CLI skills, a missing profile or search-queries.md, or no postings in the last 14 days. Running /scrape health can show whether portals are responding.
Is scraping job sites allowed?
Scraping job sites may breach some portals' terms of use. Check the terms of each portal before enabling it.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
This skill suits job seekers who already use the ai-job-search project and want a recurring, de-duplicated list of new postings. It is not suited to people without the portal CLI skills or a profile with search queries, or to anyone who wants applications sent automatically.
Download the skill