Guide · Agent skill
How to use the Forecasting Expert Knowledge agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The Forecasting Expert Knowledge skill (predictor-hand-skill) is a set of written forecasting guidelines and templates that an AI agent follows when you ask it to predict an outcome. It is based on superforecasting research by Philip Tetlock and the Good Judgment Project. After this guide you will know what the skill changes about the agent's answers, how to install it, how to ask for a forecast in a way that triggers the method, and how to record and score past predictions.
Forecasting Expert KnowledgeRightNow-AI/openfang · Apache-2.0 Download the skill Quick start
- 1Download the ZIP. Download the predictor-hand-skill ZIP from this page. It contains HAND.toml and SKILL.md.
- 2Install the skill. Unzip it into ~/.claude/skills/ for personal use or into a project's .claude/skills/ folder, or upload it under Settings, then Capabilities, then Skills in the Claude apps.
- 3Ask a forecasting question. Name the event, give a time frame and ask for a probability with the reasoning and assumptions.
- 4Provide evidence. Add any reports or figures you have so the agent can weigh them as strong, moderate or weak signals.
- 5Check the assumptions. Read the key assumptions and the resolution criteria so you know how and when the forecast can be judged.
- 6Record and score. Log the prediction in a ledger and compute its Brier score once the outcome is known.
What does the Forecasting Expert Knowledge skill make the agent do?
Without the skill, an agent answering a 'will this happen?' question often gives a confident answer built from a plausible story. This skill changes the method. The agent starts from a historical base rate for similar events, then adjusts that starting point using specific evidence, and finally states its answer as a probability.
The skill also asks the agent to rate each piece of evidence as strong, moderate or weak, to list the assumptions behind the answer, and to avoid 0% and 100%. It is a playbook of principles, signal types, a probability scale and templates. It does not fetch live data.
- Starts from a reference class base rate (the outside view)
- Breaks big questions into smaller sub-questions
- Rates signals as strong, moderate or weak and adjusts the probability for each
- Gives a final probability plus a list of key assumptions
- Never uses 0% or 100%
How do you install the predictor-hand-skill ZIP?
Download the ZIP file from this page. The skill package contains two files: HAND.toml and SKILL.md. The main instructions are in SKILL.md.
There are two ways to install it. For Claude Code, unzip the package into your user skills folder ~/.claude/skills/, or into a project's .claude/skills/ folder if you want the skill only for that project. In the Claude apps, you can instead upload the skill under Settings, then Capabilities, then Skills.
- Option A (personal): unzip into ~/.claude/skills/
- Option B (single project): unzip into the project's .claude/skills/ folder
- Option C (Claude apps): Settings → Capabilities → Skills, then upload the skill
How do you trigger the skill with a natural request?
The skill is used when you ask the agent to forecast or predict something. The source does not publish a fixed list of trigger phrases, so use plain language that makes the task clear: name the event, give a time frame, and ask for a probability. Asking for the reasoning as well as the number helps the agent follow the template.
Adding the information you already have (for example a recent report or a figure you looked up) gives the agent specific evidence to weigh against the base rate.
- 'Using a base rate and evidence, estimate the probability that X happens before a given date.'
- 'Break this question into sub-questions and give me a calibrated probability with assumptions.'
- 'Here is a recent report. How much should it move your estimate for Y?'
How do you read the output of a forecast?
The skill's reasoning template has five parts: the prediction, the reference class, the evidence for and against, a synthesis, the key assumptions and a resolution plan. The resolution part says when the prediction can be checked, the exact criteria for being right, and where to check the outcome.
Check the key assumptions first. The skill says that if an assumption is wrong, the probability should shift to a stated value, so you can see which facts matter most.
- Prediction: a specific, falsifiable claim
- Reference class: base rate and historical analogues
- Evidence: each signal with its strength and adjustment
- Synthesis: starting probability, net adjustment, final probability
- Key assumptions and resolution date, criteria and data source
How do you track and score past predictions?
The skill includes a prediction ledger format that records each prediction's confidence, domain, time horizon, key signals, status and any updates. Once a prediction resolves, you score it with the Brier score: the squared difference between the predicted probability and the outcome, where the outcome is 1 if the event happened and 0 if not.
A score of 0.0 is perfect, 0.25 is what you get by saying 50% on everything, and 1.0 is the worst. The skill gives rough bands: below 0.15 is a good forecaster, 0.20 to 0.30 is average, and above 0.35 is weak. Its accuracy report template summarises total, resolved, and active predictions. The source's template is cut off after the first few lines, so build the rest of the report yourself.
- Record each prediction with a unique id, confidence and resolution criteria
- Update the confidence as new evidence arrives and log the reason
- After resolution, compute the Brier score for each prediction
- Review misses with a post-mortem, not just the hits
Brier Score = (predicted_probability - actual_outcome)^2What does the skill need, and what are the safety limits?
The skill needs nothing else to be installed. It does not run scripts and does not use the network, so it cannot check today's news or statistics by itself. The skill names sources such as FRED, SEC filings and news, but it only tells the agent where to look. If the agent has no data or search tools, it may rely on older training knowledge.
The skill's own guidance says probabilities are estimates and can be wrong. Do not use its output alone for high-stakes financial, medical or legal decisions.
- Requirements: none listed
- Runs scripts: no
- Uses network: no
- Give the agent current data or search tools when facts may have changed
- Treat outputs as estimates, not advice
What people use it for
- Estimating whether a technology feature or product release will happen before a deadline
- Structuring a recurring forecast on market, policy or adoption questions so the method stays consistent
- Teaching students how base rates, calibration and scoring work with worked examples
- Tracking a team's forecast accuracy over time using the ledger and Brier score
Frequently asked questions
How do I install the predictor-hand-skill agent skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder. In the Claude apps, you can upload it under Settings, then Capabilities, then Skills.
Is the Forecasting Expert Knowledge skill free to use?
The source lists the skill under the Apache-2.0 licence. Check the licence text in the source repository before you reuse or redistribute it.
Does this skill need any other software or API keys?
The source lists no requirements, the skill runs no scripts, and it uses no network access.
Will the skill give me up-to-date predictions from live data?
No. The skill is written guidance and templates. It names sources such as FRED, SEC filings and news, but it does not retrieve them, so provide current data or use an agent with search tools.
How is this different from just asking an AI for a prediction?
The skill makes the agent start from a base rate, rate evidence by strength, avoid 0% and 100%, and list assumptions. It also gives a Brier score method so you can measure accuracy over time.
How do I measure whether my forecasts are accurate?
After each prediction resolves, compute the Brier score: the squared difference between the probability you gave and the outcome, where the outcome is 1 for yes and 0 for no. Lower is better, and 0.25 is the score for always saying 50%.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you want structured, probability-based reasoning about future events, want a consistent method for recurring predictions, or want to learn calibration and accuracy tracking. Pick something else if you need live data, guaranteed answers, or authoritative advice for high-stakes financial, medical or legal decisions.
Download the skill