Skip to content
Get packs

reinforcement-learning-an-introduction

Python code that reproduces the figures and examples from the Sutton & Barto reinforcement learning textbook.

GitHub Download source
Stars
14.8k
Forks
4.9k
Licence
MIT
Language
Python
Last update
2.2 years ago
Type
Learning resource

No commits in over a year — check for forks or alternatives before adopting.

What it is

This repository contains Python code that replicates the experiments and figures from the book Reinforcement Learning: An Introduction (2nd Edition) by Sutton and Barto. It is organized by chapter, covering topics such as bandit problems, dynamic programming, Monte Carlo methods, temporal-difference learning, and function approximation. People use it to see how the book's algorithms behave when run on the textbook's examples.

  • Reproduce the 10-armed testbed bandit figures from Chapter 2
  • Run the Mountain Car semi-gradient Sarsa experiment from Chapter 10
  • Compare Q-learning and Double Q-learning on the chapter 6 figure setup
  • Study the blackjack Monte Carlo examples from Chapter 5

Great for

  • Students working through the Sutton & Barto book who want runnable versions of its examples
  • Readers who learn better by seeing plotted results of algorithms like Sarsa, Q-learning, or Dyna-Q
  • Developers who want simple Python reference implementations of classic RL methods

Not ideal for

  • People looking for a general-purpose RL library for production use, since the README presents this as a book replication
  • Anyone expecting exercise answers, because the author states they do not have exercise answers for the book
Step-by-step guide: how to use reinforcement-learning-an-introductionInstall, first run, examples, tips and FAQs
reinforcement-learning-an-introduction alternativesreinforcement-learning-an-introduction vs Stable Baselines3Gymnasium vs reinforcement-learning-an-introductionreinforcement-learning-an-introduction vs Spinning Up

How to get it

  1. 1

    Clone and build from source

    You'll need: Python

    Clone the repository

    git clone https://github.com/ShangtongZhang/reinforcement-learning-an-introduction.git
  2. 2

    Run it

    • Clone the repository and open the folder in a terminal.
    • Read the README chapter list and pick a chapter that matches the book.
    • Open the Python script for that chapter's figure and run it.
    • Compare the plotted output with the figure in the book.
  3. 3

    Get the most out of it

    • Use the chapter-by-chapter contents list in the README to pair each script with the matching section of the book.
    • Read the author's note that issues are the place to ask about code or bugs.
    • Start with Chapter 2 bandit figures, which are the simplest entry point in the listed contents.

Downloads

Source code (.zip)master branch · straight from GitHub

Strengths

  • Organized chapter by chapter so it maps directly onto the book
  • Covers a broad range of chapters, from tabular methods through function approximation
  • Released under the MIT license, so it is free to read and reuse

Trade-offs

  • The README is short and the contents list is truncated, so coverage of each chapter is not fully documented here
  • The author does not provide exercise solutions, so readers must work those out themselves
  • Last push was 2024-08-09 and there is no release, so maintenance status is unclear

How it compares

ProjectStarsTypeLevelUpdatedBest for
reinforcement-learning-an-introduction (this)14.8kLearning resourceintermediate2.2 years agoStudents working through the Sutton & Barto book who want runnable versions of its examples
spinningup12kLearning resourceintermediate2.3 years agoStudents who want a structured entry point into deep RL
stable-baselines314kLibraryintermediate11 days agoResearchers who want a reliable baseline to compare a new method against
Gymnasium12.8kLibraryintermediate8 days agoResearchers comparing reinforcement learning algorithms on the same environments
awesome-artificial-intelligence16.8kCurated listbeginner3 months agoSoftware developers who want to learn the foundations behind language models and agents
AI-For-Beginners70.5kLearning resourcebeginner14 days agoBeginners who want a structured, self-paced introduction to AI

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.