Skip to content
Get packs

Guide · Agent skill

How to use the Stable Baselines3 RL skill for agents

Updated · written with AI from the project's own documentation and checked against our catalogue data

The stable-baselines3 skill gives an AI agent tested guidance and code templates for running single-agent reinforcement learning experiments with the Stable Baselines3 library. It is aimed at people who already write some Python and want working training scripts rather than an API reconstructed from memory. By the end of this guide you will know how to install the skill, ask for RL work in plain language, and what you need on your machine to run the code it produces.

Stable Baselines3 RL TrainingK-Dense-AI/scientific-agent-skills · MIT

Quick start

  1. 1Download and unzip the skill. Download the ZIP from the item page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder. For the Claude apps, upload the ZIP under Settings → Capabilities → Skills instead.
  2. 2Install Python dependencies. Make sure Python 3.10+ and PyTorch 2.8+ are available, then install stable-baselines3 2.9.0 with uv, adding the extras form if you need TensorBoard or Atari.
  3. 3Confirm the library version. Run a short Python check that imports stable_baselines3 and prints its __version__ to confirm 2.9.0 is installed.
  4. 4Describe your RL task. Ask the agent in plain language for the algorithm, environment and goal, for example training PPO on CartPole-v1 with a fixed seed.
  5. 5Review the generated code. Read the scripts the agent produces before running them, and check the environment with check_env if it is custom.
  6. 6Run training and save the model. Run the training script, save the checkpoint, and reload it with the same device setting for evaluation.
  7. 7Evaluate the saved model. Use the evaluation script only on model files you created or trust, since loading serialized objects can run arbitrary code.

What the skill makes the agent do

Reinforcement learning means training a program to improve through trial and reward. Once this skill is installed, the agent knows how to set up Stable Baselines3 (SB3) experiments: choosing an algorithm such as PPO, SAC, DQN, TD3, DDPG or A2C, building custom Gymnasium environments, running vectorized environments in parallel, adding callbacks for monitoring and early stopping, and saving or evaluating checkpoints.

  • Training and evaluating single-agent RL with SB3
  • Creating custom Gymnasium environments and checking them with check_env
  • Running vectorized environments such as DummyVecEnv and SubprocVecEnv
  • Adding callbacks such as EvalCallback and CheckpointCallback
  • Guidance for SB3-Contrib recurrent or masked policies

How to install the stable-baselines3 skill

Skills are folders of instructions and scripts that an agent loads when a task matches them. This skill ships as a ZIP from the item page. You can install it in one of two ways.

For Claude Code, download the ZIP from our page and unzip it into ~/.claude/skills/ for your personal use, or into a project's .claude/skills/ folder to share it with a repository. For the Claude apps, upload the ZIP under Settings → Capabilities → Skills.

  • Personal install: unzip into ~/.claude/skills/
  • Project install: unzip into .claude/skills/ inside your project
  • Claude apps: Settings → Capabilities → Skills, then upload the ZIP

Install the Stable Baselines3 library the skill expects

The skill targets stable-baselines3 2.9.0 and requires Python 3.10 or newer and PyTorch 2.8 or newer. Installation uses network access unless packages are already cached. The skill's own instructions use uv to install the library, and recommend the extras form if you want TensorBoard or Atari (ale-py) support.

On zsh, quote the brackets. For MuJoCo continuous-control benchmarks, the source also lists gymnasium[mujoco] as an optional install. You can confirm the version with a short Python check.

  • Python 3.10 or newer
  • PyTorch 2.8 or newer
  • Gymnasium for environments (MuJoCo optional)
  • Optional extras: TensorBoard and ale-py for Atari
uv pip install "stable-baselines3==2.9.0" uv pip install "stable-baselines3[extra]==2.9.0" uv pip install "gymnasium[mujoco]" import stable_baselines3 print(stable_baselines3.__version__)

How to trigger the skill with natural requests

You do not need special syntax. Describe the RL task in plain language and mention the algorithm or environment if you know them. The agent will load the skill when the request matches its description. Good requests name the goal, the environment and any constraints such as reproducibility or continuous actions.

Vague requests work less well than specific ones. If you are unsure which algorithm fits, ask the agent to compare options for your action space first.

  • "Train a PPO agent on CartPole-v1 with a fixed seed and save the model."
  • "Write a custom Gymnasium grid-world environment and check it with check_env."
  • "Set up SAC for a continuous control task and save the best checkpoint with EvalCallback."
  • "Run PPO on 4 parallel environments and evaluate the saved model."

Basic training pattern the agent will follow

The skill's core pattern creates a Gymnasium environment, initializes an algorithm with a policy, trains it, saves the model, and reloads it for later use. Two details matter: total_timesteps is a lower bound, so training may run slightly longer, and the replay buffer is not saved with the model to save space.

Algorithm choice follows the action space. PPO and A2C handle Box, Discrete, flat MultiDiscrete and MultiBinary actions. SAC and TD3 target continuous control. DQN targets discrete actions.

  • PPO and A2C: general-purpose, support multiprocessing
  • SAC and TD3: continuous control, off-policy
  • DQN: discrete actions, off-policy
  • HER: replay-buffer strategy for goal-conditioned tasks
import gymnasium as gym from stable_baselines3 import PPO env = gym.make("CartPole-v1") model = PPO("MlpPolicy", env, verbose=1, device="cpu", seed=0) model.learn(total_timesteps=10000) model.save("ppo_cartpole") model = PPO.load("ppo_cartpole", env=env, device="cpu") env.close()

Build a custom environment and check it

A custom environment must inherit from gymnasium.Env and implement __init__, reset(seed, options), step(action), and close, with render optional. Before training, the skill recommends running check_env to catch API problems. The check looks at API consistency only; it does not confirm that your observations are sufficient, that rewards are correct, or that the task is learnable.

Image observations should be np.uint8 in the range 0 to 255 and ideally channel-first. SB3 does not support Discrete or MultiDiscrete spaces with start not equal to 0.

  • Full template: scripts/custom_env_template.py
  • Environment guide: references/custom_environments.md
  • gym.make adds the 100-step time limit in the template; direct instantiation does not
from stable_baselines3.common.env_checker import check_env check_env(env, warn=True)

Use vectorized environments and callbacks

Vectorized environments run several copies of an environment together. DummyVecEnv runs them sequentially in one process and suits light environments; SubprocVecEnv runs them in parallel processes and suits heavy ones. Its creation belongs under a main guard in a Python file. Callbacks add monitoring, periodic evaluation, checkpoint saving and early stopping without changing the algorithm.

The skill's examples use make_vec_env with a try/finally block so the environment is closed even if training fails.

  • make_vec_env for quick parallel setup
  • EvalCallback saves the best model during training
  • CheckpointCallback saves models at intervals
  • StopTrainingOnRewardThreshold stops when a target reward is reached
  • CallbackList chains several callbacks together
from stable_baselines3.common.env_util import make_vec_env env = make_vec_env("CartPole-v1", n_envs=4, seed=0) try: model = PPO("MlpPolicy", env, verbose=1, device="cpu") model.learn(total_timesteps=25000) finally: env.close()

Safety notes: scripts and network access

This skill runs scripts and uses the network. Installing packages downloads them from the internet. The included scripts are train_rl_agent.py, evaluate_agent.py and custom_env_template.py. Review them before running, especially if you did not write them.

The evaluation and normalization helpers load serialized Python objects. Loading a malicious model file can execute arbitrary code, so only evaluate model files you trust. Local RL training needs no credentials or service endpoints, according to the source.

  • Read scripts/evaluate_agent.py before running it on files from others
  • Only load model and normalization files you created or trust
  • Installation requires network unless packages are cached

What people use it for

  • Training PPO on a standard Gymnasium control task such as CartPole-v1 as a first reproducible experiment
  • Comparing algorithms like SAC and TD3 for continuous control with a fixed seed
  • Building a custom grid-world environment for teaching or prototyping and checking its API
  • Speeding up training with parallel environments via make_vec_env
  • Saving the best checkpoint during training with EvalCallback and CheckpointCallback

Frequently asked questions

Is the stable-baselines3 skill free to use?

The skill is published under the MIT licence, as stated on the item page. Stable Baselines3 itself is installed from pip, and the source does not list any pricing for the skill.

What do I need to run the stable-baselines3 skill?

You need Python 3.10 or newer, PyTorch 2.8 or newer, stable-baselines3 2.9.0, and Gymnasium for environments. MuJoCo and TensorBoard or ale-py for Atari are optional. The skill also needs network access for installation unless packages are cached.

How do I install the skill in Claude Code?

Download the ZIP from the item page and unzip it into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder. In the Claude apps, upload the ZIP under Settings → Capabilities → Skills.

Which algorithm should I pick, PPO, SAC or DQN?

The skill suggests PPO or A2C for general-purpose work that supports many action types, SAC or TD3 for continuous control, and DQN for discrete actions. These are starting points, and the source notes that tuning is task-dependent rather than guaranteed.

Can the skill handle multi-agent or production systems?

No. The skill covers single-agent reinforcement learning only, and the item's description says it does not cover multi-agent or large-scale production systems. It is best suited to experiments and learning.

Is it safe to load a model file someone sent me?

Be careful. The evaluation and normalization helpers load serialized Python objects, which can execute arbitrary code if the file is malicious. Only load model files you created or trust.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

This skill suits machine learning engineers, researchers and students who want reproducible Stable Baselines3 experiments in Python and tested code patterns to start from. It is not suited to non-programmers wanting a finished RL product, to multi-agent systems, or to loading model files from unknown sources.

Download the skill