PluginEval Evaluation Methodology
Explains how skill quality is scored and how to interpret and improve those scores
- Repository
- wshobson/agents
- Stars
- 42,095
- Licence
- MIT
- Files
- 3 files
What it does
This skill teaches the agent how the PluginEval tool measures the quality of agent skills and plugins. It explains the scoring layers, what each dimension means, how composite scores and badges are calculated, and how to read low scores. When a user asks why a skill scored poorly or how to improve its triggering accuracy, the agent can give grounded, specific guidance using the bundled rubric and fix references.
Great for
- Plugin authors who want to understand why their skill received a low PluginEval score
- Marketplace maintainers setting quality thresholds or explaining badges to partners
- Developers who want to improve a skill's triggering accuracy or orchestration fit
Skip it if
- People who want to write a new skill from scratch, since this only covers evaluating existing ones (the rubric is about measurement, not authoring)
- Anyone relying on the judge or Monte Carlo scores as validated quality proof, because the files state those layers are experimental and not checked against human labels
Try asking
- “Why did my skill get a low triggering_accuracy score in PluginEval?”
- “What does the Gold badge mean and how is the composite score calculated?”
- “How do I fix the OVER_CONSTRAINED anti-pattern flagged in my skill?”
- “Explain the difference between quick, standard, and deep evaluation depths.”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 3 original files, the MIT licence and an attribution note.
evaluation-methodology.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip evaluation-methodology.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip evaluation-methodology.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Ask about a specific dimension or anti-pattern name to get the matching fix guidance from the references
- Remember that the static layer is the only fully deterministic part; treat judge and Monte Carlo numbers as rough signals
- Run the quick depth first for fast structural feedback before using standard or deep evaluation
What's inside
- references/improving-scores.md
- references/rubrics.md
- SKILL.md
Redistributed unmodified from wshobson/agents under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.