Skip to content
Get packs

Writing Evals for Axiom

Scaffolds AI evaluation suites, scorers, flag schemas and config for the Axiom AI SDK.

Source Download skill
Repository
openclaw/clawhub
Stars
9,546
Licence
MIT
Files
13 files

What it does

When installed, the agent can set up automated tests for AI features built with the Axiom AI SDK. It reads your code, picks a suitable eval type and scorers, and generates .eval.ts files, flag schemas and an axiom.config.ts with starter test data including adversarial and edge cases. This lets you check whether an AI capability still behaves correctly after changes.

Great for

  • Developers building AI features with the Axiom AI SDK who want repeatable quality checks
  • Teams that want a consistent structure for eval files, scorers and test data
  • Engineers setting up flag schemas to compare models or temperatures without code changes

Skip it if

  • Projects that do not use the Axiom AI SDK, since the templates and API assumptions will not match (the skill tells the agent to check installed docs first)
  • Non-technical users looking for a general AI writing or testing tool, since it requires a codebase and a running SDK setup
  • Use with production credentials carelessly: eval runs send traces and data to an Axiom account via AXIOM_URL, AXIOM_TOKEN and AXIOM_DATASET, so keep tokens in environment variables and review what gets exported

Try asking

  • “Write evals for my categorizeMessages function in src/lib/capabilities/support-agent”
  • “Set up a flag schema so I can sweep models and temperatures for my summarizer”
  • “Create an axiom.config.ts for my project with the Axiom eval settings”
  • “Add a tool-use eval for my support agent that checks which tools get called”
Step-by-step guide: how to use Writing Evals for AxiomInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 13 original files, the MIT licence and an attribution note.

    writing-evals.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip writing-evals.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip writing-evals.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Let the agent read your function first and check node_modules/axiom/dist/docs for current API signatures before generating files
  • Review generated scorers and test data, since the templates contain TODO placeholders that need real expected values
  • Keep Axiom tokens in environment variables and never commit them to the repository

What's inside

  • README.md
  • reference/api-reference.md
  • reference/flag-schema-guide.md
  • reference/scorer-patterns.md
  • reference/templates/app-scope.ts
  • reference/templates/axiom.config.ts
  • reference/templates/classification.eval.ts
  • reference/templates/instrumentation.ts
  • reference/templates/minimal.eval.ts
  • reference/templates/retrieval.eval.ts
  • reference/templates/structured-output.eval.ts
  • reference/templates/tool-use.eval.ts
  • SKILL.md

Redistributed unmodified from openclaw/clawhub under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.