Tianshou
A Python library for building and training reinforcement learning agents using PyTorch.
- Stars
- 11k
- Forks
- 1.4k
- Licence
- MIT
- Language
- Python
- Last update
- 7 months ago
- Type
- Library
What it is
Tianshou is a reinforcement learning library built on PyTorch and Gymnasium. It provides many ready-made learning algorithms, such as DQN, PPO, SAC and TD3, so you can train agents that learn by interacting with an environment. It offers both high-level interfaces for applying existing algorithms and lower-level interfaces for researchers writing new ones.
- Train a PPO or SAC agent on a Gymnasium environment such as a MuJoCo task
- Run offline RL algorithms like CQL or BCQ on a fixed dataset
- Experiment with imitation learning methods such as GAIL
- Log training runs to TensorBoard or Weights & Biases
Great for
- Researchers who want to implement or modify reinforcement learning algorithms in PyTorch
- Practitioners training a supported algorithm on their own Gymnasium-style environment
- Students studying how common RL algorithms are implemented
- Developers who want vectorized or parallel environment sampling
Not ideal for
- People new to reinforcement learning who want a guided, beginner-focused course (the README is a library reference, not a tutorial)
- Anyone needing a non-Python stack, since the project is Python-based and requires Python 3.11 or newer
- Users who need a fully stable, long-term API without changes, since version 2 broke compatibility with earlier versions
How to get it
- 1
Install with a package manager
You'll need: Python 3.11 or newer · PyTorch
Install from PyPI
pip install tianshouClone the repository
git clone https://github.com/thu-ml/tianshou.git - 2
Run it
- Install Tianshou from PyPI or clone the repository
- Set up a Python 3.11+ environment with PyTorch installed
- Choose a supported algorithm and a Gymnasium environment
- Follow the examples and tutorials at tianshou.org to start a training run
- 3
Get the most out of it
- Use the high-level API for quick application experiments and the procedural API for custom algorithms
- Enable vectorized environments to speed up sampling
- Try EnvPool integration for faster environment stepping
- Log results with TensorBoard or W&B using built-in support
- Check the MuJoCo examples to compare against the project's benchmark results
Downloads
Strengths
- Covers a broad set of algorithms across online, offline and imitation learning
- Offers both high-level and procedural APIs for different levels of control
- Supports vectorized environments, recurrent networks, multi-GPU training and common logging tools
Trade-offs
- Version 2 is not backwards compatible with earlier versions, so older code and tutorials may need migration
- Multi-agent and model-based RL are described only as experimental support
- Installation is recommended via cloning and using poetry, which adds setup steps
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| tianshou (this) | 11k | Library | advanced | 7 months ago | Researchers who want to implement or modify reinforcement learning algorithms in PyTorch |
| stable-baselines3 | 14k | Library | intermediate | 11 days ago | Researchers who want a reliable baseline to compare a new method against |
| pytorch-grad-cam | 13k | Library | intermediate | 6 days ago | Researchers comparing explainability methods for image models |
| External-Attention-pytorch | 12.2k | Library | beginner | 8 months ago | Students reading deep learning papers who want to see the core code of a module |
| annotated_deep_learning_paper_implementations | 67.6k | Learning resource | intermediate | 5 days ago | Students learning how transformers, GANs, or optimizers work internally |
| Made-With-ML | 49.9k | Learning resource | intermediate | 8 months ago | Software engineers who want to learn how ML models get deployed and maintained |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.