Portkey AI Gateway
Open-source gateway that routes requests to many AI models through one API, with guardrails.Portkey-AI/gateway
- Stars
- 13.3k
- Forks
- 1.4k
- Licence
- MIT
- Language
- TypeScript
- Last update
- 6 months ago
- Type
- Dev tool
What it is
Portkey's AI Gateway sits between your app and large language model providers. Your app sends requests to one API, and the gateway forwards them to the model provider you choose. The README says it adds routing, automatic retries, fallbacks, load balancing, and guardrails that check inputs or outputs, and it can be run locally or deployed.
- Send the same chat request to different model providers and switch between them without changing app code
- Configure retries and fallbacks so a request is retried when a provider errors
- Block replies that contain certain words using an output guardrail config
- View local request logs in the Gateway Console at localhost:8787/public/
Great for
- Developers who call several LLM providers and want one OpenAI-style interface
- Teams that want automatic retries or fallbacks when a model provider fails
- Apps that need content checks on model replies using configurable guardrails
- Developers who want a local gateway with a console showing request logs
Not ideal for
- People who only call a single model from one provider and do not need routing, since the gateway adds a layer to manage
- Non-developers without experience running Node.js or calling APIs, since setup assumes familiarity with code
- Organizations needing enterprise features like org management, which the README places in a separate paid Enterprise version
How to get it
- 1
Install with a package manager
You'll need: Node.js · npm · Python (only if using the Python client example)
Run the gateway locally
npx @portkey-ai/gatewayInstall the Python client
pip install -qU portkey-ai - 2
Run it
- Run npx @portkey-ai/gateway to start the gateway locally
- Open the Gateway Console at http://localhost:8787/public/ to view logs
- Install the portkey-ai Python client and point it at your provider API key
- Send a chat completion request and check the logs in the console
- 3
Get the most out of it
- Use a config object to add retries, for example retry attempts
- Add input or output guardrails to block or check model responses
- Use conditional routing and load balancing to spread traffic across providers
- Review the docs at the linked Portkey documentation for deployment options such as Docker or Cloudflare Workers
Downloads
Strengths
- The README describes a small footprint and low latency, and it is MIT licensed
- Works with an OpenAI-compatible client, so existing code can be pointed at it with limited changes
- Supports several deployment options including Docker, Node.js, and Cloudflare Workers
Trade-offs
- Many headline numbers (model counts, latency, token volume) come from the README's own claims and were not independently verified here
- Some advanced features, such as enterprise governance and private deployments, are described as part of a separate Enterprise version
- The README mentions a Gateway 2.0 pre-release that is merging enterprise features into open source, so the project may change
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| gateway (this) | 13.3k | Dev tool | intermediate | 6 months ago | Developers who call several LLM providers and want one OpenAI-style interface |
| generative-ai-for-beginners | 123k | Learning resource | beginner | 5 days ago | Developers who know some Python or TypeScript and want a guided introduction to generative AI |
| Prompt-Engineering-Guide | 79.8k | Learning resource | beginner | 8 months ago | People who are new to prompting and want structured lessons with examples |
| botpress | 15k | Dev tool | intermediate | 6 days ago | Developers who want to write integrations and bots as TypeScript code using the Botpress CLI and SDK |
| claude-context | 12.7k | Dev tool | beginner | 4 months ago | Developers working in large codebases where loading everything into an agent's context is impractical |
| langchain | 149k | Framework | intermediate | 4 days ago | Python developers building LLM-powered applications |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.