Guide · Open source

How to use guidance: constrain LLM output in Python

Updated 2026-10-29 · written with AI from the project's own documentation and checked against our catalogue data

Guidance is a Python library for controlling how a language model writes its output. Instead of writing a prompt and hoping the format comes out right, you constrain generation with patterns such as regular expressions or grammars, and mix in ordinary code like conditionals and loops. This guide shows you how to install it, run a first example, constrain output, and test grammars offline.

guidanceguidance-ai/guidance · ★ 21.8k · MIT Install options & downloads

Quick start

1Check prerequisites. Confirm you have Python and a supported backend such as Transformers, llama.cpp or OpenAI for your model.
2Install guidance. Run pip install guidance from PyPI.
3Load a model. Create a model object, for example Transformers with a model name, as shown in the README.
4Build the conversation. Use with system(), with user() and with assistant() blocks to add messages.
5Constrain the output. Use gen(regex=...) for patterns or select([...]) for a fixed list of choices.
6Test offline. Check strings with grammar.match and run the grammar on the Mock model.

Install guidance and check your backend

Guidance is available through PyPI and supports several backends, including Transformers, llama.cpp and OpenAI. You need Python and a supported model backend. If you already have the backend required for your model, install the library with pip.

The README says you need the backend for your model in place first, so set that up before running the library against a model.

Prerequisites: Python and a supported model backend such as Transformers, llama.cpp or OpenAI

pip install guidance

Run your first model call with gen()

Guidance lets you work with a model using common Python idioms. You load a model object, add system and user messages inside `with` blocks, and then append a `gen()` call for the assistant's reply. The README notes that model objects are immutable, so each `lm = phi_lm` line gives you a fresh copy.

Pass `name=` to `gen()` to capture the generated text so you can read it later with `lm['name']`.

Load a model, for example Transformers with microsoft/Phi-4-mini-instruct as shown in the README
Use `with system():`, `with user():` and `with assistant():` blocks to build the conversation
Use `gen(max_tokens=20)` to generate, or `gen(name=...)` to capture the result

from guidance import system, user, assistant, gen
from guidance.models import Transformers

phi_lm = Transformers("microsoft/Phi-4-mini-instruct")
lm = phi_lm

with system():
    lm += "You are a helpful assistant"

with user():
    lm += "Hello. What is your name?"

with assistant():
    lm += gen(max_tokens=20)

print(lm)

Constrain output with regex or a fixed list

The main reason to use guidance is to guarantee output shape. A `gen()` call can be limited to match a regular expression, which is useful for numbers or simple codes.

When the answer must be one item from a known list, use `select()`. The model can only pick one of the options you give it, such as letters for a multiple-choice question.

regex=: restricts a gen() call to a pattern, for example r"\d+" for digits
select(): chooses one option from a list you provide
The README says output can conform to any context free grammar, as long as the backend fully supports guidance

from guidance import select

lm += select(["A", "B", "C", "D"], name="model_selection")

Test grammars offline with the Mock model

When you iterate on constraints, you can check candidate strings against a grammar without calling a model. The `match` method returns a result for strings that fit and None for strings that do not. You can also run the same grammar through the `Mock` model to see a full run.

This is the path the README describes for developers who want to test grammars offline.

grammar.match(string) returns None when the string does not fit the grammar
Mock(b"...") lets you run the grammar with a local mock model

from guidance import gen
from guidance.models import Mock

grammar = "expr=" + gen(regex=r"\d+([+*]\d+)*", name="expr")

assert grammar.match("expr=12+7*3") is not None
assert grammar.match("expr=12+*3") is None

lm = Mock(b"<s>expr=12+7*3")
lm += grammar
print(lm["expr"])

Write your own guidance functions

Mark a Python function with the `@guidance` decorator to make it work with models, as in the multiple-choice example. The language_model argument is filled in automatically when you call the function with `+=`.

Functions marked `stateless=True` are meant for composing grammars, such as building HTML from smaller pieces. The README's full example builds a simple HTML page this way and notes it is not a complete HTML implementation.

@guidance turns a function into a Guidance function
stateless=True marks functions intended for grammar composition
Guidance functions can be used like gen() or select()

import guidance
from guidance.models import Model

@guidance
def zero_shot_multiple_choice(language_model: Model, question: str, choices: list[str]):
    with user():
        language_model += question + "\n"
    with assistant():
        language_model += select([chr(i + ord("a")) for i in range(len(choices))], name="string_choice")
    return language_model

What people use it for

Getting a fixed-choice answer, such as picking a letter from a multiple-choice list, with select()
Forcing a number or pattern from a model with gen(regex=...)
Running a multi-step workflow that mixes model calls with ordinary Python conditionals and loops
Building structured text such as simple HTML by composing stateless Guidance functions
Testing a grammar locally with the Mock model before running a real model

Frequently asked questions

Is guidance free to use?

The source is licensed under MIT, which the listing identifies as its licence. Check the repository's licence file for the terms that apply to your use.

What do I need before installing guidance?

You need Python and a supported model backend such as Transformers, llama.cpp or OpenAI. The README says to have the backend for your model ready before you run the install command.

How do I install guidance?

Run pip install guidance. The library is published on PyPI.

Can I use guidance without writing Python?

No. Guidance is a code library for Python, so it is not suited to people who only use hosted chat interfaces.

How does guidance compare with Outlines or LMQL?

Outlines, LMQL and Microsoft JSON schema tools are listed as comparable projects. This guide does not cover their differences, so compare them directly on the features you need.

Is guidance stable enough for production?

The README does not describe stability guarantees, and the latest release is 0.3.2. Teams that need a stable API should evaluate this carefully before depending on it.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Guidance suits Python developers who need predictable output formats, want to run models locally, or build multi-step workflows that mix generation with code. If you only use hosted chat interfaces or need a guaranteed stable API, pick a different tool or evaluate this one carefully first.

Install options & downloads