Guide · Open source
How to use GPU-Puzzles: learn CUDA by solving puzzles
Updated · written with AI from the project's own documentation and checked against our catalogue data
GPU Puzzles is an open-source Jupyter Notebook by Sasha Rush that teaches beginner GPU programming through exercises instead of long explanations. You write short GPU functions (kernels) and run them against tests. After working through it, you should have hands-on intuition for how GPU code is organized into threads and blocks.
GPU Puzzlessrush/GPU-Puzzles · ★ 12.6k · MIT Install options & downloads Quick start
- 1Open the notebook. Click the Open In Colab link in the README and make your own copy of GPU_puzzlers.ipynb.
- 2Turn on GPU mode. Go to Runtime, then Change runtime type, and set Hardware accelerator to GPU.
- 3Run the setup cells. Run the install and import cells at the top of the notebook so the helper library and image files are available.
- 4Read the first puzzle. Read the Map puzzle description and the tip that the function runs once per thread.
- 5Fill in the kernel. Replace the FILL ME IN line with simple code using threadIdx and the allowed operations.
- 6Check your answer. Run problem.check() and compare your output with the spec. Fix and repeat until the tests pass.
What you need before you start
The README recommends running the notebook in Google Colab, which needs a Google account. You also need GPU mode turned on, so you need access to a GPU. Python knowledge is helpful but the puzzles only use simple operations.
The project is not a fit if you need to write production GPU code, since the puzzles cover basics only.
- A Google account for Colab (recommended by the README)
- Access to a GPU, set through the Colab runtime settings
- Python knowledge is helpful
Open the notebook in Colab
Use the Open in Colab link from the README. The README advises making your own copy of the notebook before you start coding.
Then turn on the GPU. In Colab, go to Runtime, then Change runtime type, and set Hardware accelerator to GPU.
- Make your own copy of the notebook
- Set Hardware accelerator to GPU under Runtime / Change runtime type
https://colab.research.google.com/github/srush/GPU-Puzzles/blob/main/GPU_puzzlers.ipynbRun it locally by cloning the repository
The README also lists a clone command if you prefer to work from a local copy of the repository. The README's setup cells install a helper package and download the files the notebook needs.
git clone https://github.com/srush/GPU-PuzzlesSolve a puzzle and check your answer
Each puzzle has a spec, which is the correct result written in plain Python, and a kernel template with a FILL ME IN line. Replace that line with your code, then call problem.check() to see whether your output matches the spec. Calling problem.show() displays the puzzle's score for reads and writes.
The first puzzle, Map, asks you to add 10 to each position of a vector. Think of the function as running once per thread, where cuda.threadIdx.x changes each time.
- Write code only with simple operations: +, *, array indexing, for loops and if statements
- You may use local variables
- Avoid list comprehensions and NumPy properties like shape or size; the size is passed as an argument
- If you get an error, you probably used something the puzzles do not allow
problem.check()Move from threads to blocks and 2D problems
The later puzzles add guards for extra threads, 2D arrays, broadcasting and blocks. In the Blocks puzzle you have fewer threads per block than the size of the input, and cuda.blockIdx tells you which block you are in.
The Blocks puzzles are where the threads and blocks model becomes concrete, so take your time there.
- Guards: more threads than positions
- Map 2D: square 2D input
- Broadcast: adding two vectors into a 2D output
- Blocks and Blocks 2D: fewer threads per block than the input size
What people use it for
- Building intuition for how threads and blocks split up work on a GPU
- Giving a machine learning practitioner their first hands-on GPU code exercises
- Practising Numba CUDA-style kernels in short, testable steps
- Working through the puzzles as a student who learns by coding rather than reading
Frequently asked questions
Is GPU Puzzles free?
The repository is licensed under the MIT licence, and the README recommends running the notebook in Google Colab. The source does not list any prices, so check Colab's own terms for any account or usage limits.
Do I need a GPU to use GPU Puzzles?
The puzzles are designed for GPU execution. The README recommends Colab with GPU mode turned on, which is set under Runtime, then Change runtime type, with Hardware accelerator set to GPU.
What do I need to know before starting?
A Google account is needed for Colab, as the README recommends it. Python knowledge is helpful, but the puzzles only use simple operations.
What language does GPU Puzzles use?
The notebook is written in Jupyter Notebook and uses Numba, which lets you write Python-looking code that maps closely to low-level CUDA.
How is GPU Puzzles different from CUDA by Example or Triton tutorials?
GPU Puzzles is an interactive notebook of short exercises that you check against tests. The source does not compare its depth against CUDA by Example or the Triton tutorials, so review those resources to decide which fits your goal.
Can I use GPU Puzzles to write production GPU code?
No. The puzzles cover basics only, and the README says the code is basically low-level CUDA. For production work, the README points you to the NUMBA CUDA Guide.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
GPU Puzzles suits developers, machine learning practitioners and students who want hands-on intuition for threads and blocks without reading long text. It is not suited to production GPU work, to anyone without GPU access, or to readers who want a conceptual course. If you need production code, use the NUMBA CUDA Guide or a more complete CUDA resource instead.
Install options & downloads