Skip to content
Get packs

Guide · Open source

How to use promptflow: install, test flows and evaluate (2027 retirement)

Updated · written with AI from the project's own documentation and checked against our catalogue data

Promptflow is Microsoft's open-source MIT-licensed toolkit for building applications that use large language models. It lets you connect LLMs, prompts and Python code into a flow, test it, run it against datasets and deploy it. This guide shows the setup and basic workflow from the project README, and it also explains an important warning: Microsoft is retiring Prompt flow and no longer recommends it for new projects.

Prompt flowmicrosoft/promptflow · ★ 11.3k · MIT

Quick start

  1. 1Check the retirement notice. Read the README warning about the April 20, 2027 retirement and decide whether you are maintaining an existing app or starting a new one.
  2. 2Install the packages. Create a Python 3.9 to 3.11 environment and run pip install promptflow promptflow-tools.
  3. 3Initialise a chat flow. Run pf flow init with --type chat to create the my_chatbot folder and its files.
  4. 4Create a connection. Run pf connection create with your API key, using the openai.yaml or azure_openai.yaml file.
  5. 5Test interactively. Run pf flow test with --interactive and chat with your flow. Press Ctrl + C to stop.

Check whether promptflow fits your project before you start

The README states that Prompt flow in Microsoft Foundry and Azure Machine Learning will be retired on April 20, 2027, and is no longer recommended for new development. It suggests migrating existing applications to Microsoft Agent Framework before that date.

Prompt flow container images no longer receive security or package updates, so workloads that depend on them should also be moved.

  • Good fit: teams with existing Prompt flow applications that must keep running until migration, and developers who want to trace and evaluate LLM app behavior on datasets.
  • Poor fit: anyone starting a new project, teams relying on Prompt flow runtime container images, and non-developers, since setup uses the command line and a Python environment.

Install promptflow with pip

Promptflow is installed as Python packages. The README recommends a Python environment with Python 3.9 to 3.11. Install the two packages with pip.

The README also offers a pre-built development environment through GitHub Codespaces if you want to avoid local setup.

  • Requirement: Python 3.9 to 3.11 (recommended by the README).
pip install promptflow promptflow-tools

Create your first chat flow

The quick start initiates a flow from a chat template. The command creates a folder named my_chatbot and generates the required files inside it.

  • The flow is defined in flow.dag.yaml, which sets inputs, outputs, nodes, connection and the LLM model.
  • The chat node uses a connection named open_ai_connection and the deployment_name field to choose the model.
pf flow init --flow ./my_chatbot --type chat

Create a connection for your API key

Flows reach LLMs through a named connection. For OpenAI, use the openai.yaml file in the my_chatbot folder. You can pass your key with --set so you do not need to edit the YAML file.

For Azure OpenAI, use azure_openai.yaml instead and set the API key and base URL the same way.

  • Replace <your_api_key> with your own key. Do not commit real keys to source control.
  • The connection name in the example is open_ai_connection, which the chat node references.
pf connection create --file ./my_chatbot/openai.yaml --set api_key=<your_api_key> --name open_ai_connection

Test your flow interactively

Run the flow in interactive mode to chat with it in your terminal. Press Ctrl + C to end the session.

The README also describes a 15-minute tutorial for prompt tuning, batch testing and evaluation, which is the main quality workflow the project emphasises.

  • Use the VS Code extension for a visual flow designer, available from the Visual Studio Marketplace.
pf flow test --flow ./my_chatbot --interactive

Where to go next

The README points to a getting-started guide in the docs folder and an end-to-end Chat with PDF tutorial that covers flow development and evaluation with metrics. The project also documents tracing of LLM interactions and integrating tests into CI/CD.

If you plan to keep using Prompt flow for existing apps, start with the migration guide linked in the README's warning so you know the path to Microsoft Agent Framework.

What people use it for

  • Keeping an existing Prompt flow chatbot running until it is migrated to Microsoft Agent Framework
  • Tracing LLM calls in a flow to see how the model behaves step by step
  • Running a flow against a larger dataset to evaluate its quality
  • Adding flow tests to a CI/CD pipeline
  • Editing flows visually with the VS Code extension

Frequently asked questions

Is promptflow free to use?

Promptflow is released under the MIT licence, as shown in the repository. The README does not list prices for the Azure or Microsoft Foundry services, so check those services separately if you use them.

What do I need to install promptflow?

You need a Python environment; the README recommends Python 3.9 to 3.11. Install it with pip install promptflow promptflow-tools from the command line.

Is promptflow still supported?

Microsoft's README says Prompt flow will be retired on April 20, 2027, and is no longer recommended for new development. Existing users should plan migration to Microsoft Agent Framework before that date.

What are the alternatives to promptflow?

The project lists LangChain, LlamaIndex, DSPy and Haystack as comparable projects. The README suggests Microsoft Agent Framework as the replacement for Prompt flow workloads.

Can I use promptflow without the command line?

The README's setup uses the pf command line and a Python environment, though it also mentions a VS Code extension for visual flow editing. The extension is also scheduled for retirement with the rest of Prompt flow.

Where do I put my OpenAI API key?

The quick start creates an openai.yaml file in the flow folder and uses pf connection create with --set api_key to store the key. Avoid committing the file with a real key to source control.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Promptflow is a reasonable tool for maintaining existing Prompt flow applications, and for developers who want to trace and evaluate LLM apps on datasets. For new projects, the README recommends Microsoft Agent Framework instead, and frameworks such as LangChain, LlamaIndex, DSPy or Haystack are the other options to compare.

Install options & downloads