Skip to content
Get packs

Guide · Open source

How to use freellmapi: install, configure and call one local endpoint

Updated · written with AI from the project's own documentation and checked against our catalogue data

FreeLLMAPI is an open-source program you run on your own machine or server. You add API keys from several providers that offer free tiers, and it exposes one OpenAI-compatible /v1 endpoint that routes each request to an available model. This guide walks through what it does, how to get it running with Docker, and how to point tools at it, so you can experiment without paying for usage.

FreeLLMAPItashfeenahmed/freellmapi · ★ 46k · MIT

Quick start

  1. 1Clone the repository. Run the git clone command to download FreeLLMAPI from GitHub.
  2. 2Install Docker. Make sure Docker is available on the machine that will host the app.
  3. 3Start the self-host setup. Follow the Docker Compose instructions in the install guide linked from the README.
  4. 4Add provider API keys. Enter keys from the free-tier providers you want to use; they are stored encrypted.
  5. 5Add custom endpoints if needed. Use the Keys page to add any OpenAI-compatible server you run yourself.
  6. 6Point your client at /v1. Set your OpenAI-compatible tool's base URL to the local /v1 endpoint and use your unified key.

What FreeLLMAPI does

FreeLLMAPI puts many free LLM providers behind a single API. Instead of managing separate SDKs, rate limits and failure modes for each provider, you point an OpenAI client library at your local server and it routes requests across the providers you have added keys for.

A router picks an available model for each request, switches to another provider when one is rate-limited, and tracks usage so you stay within free-tier caps. Keys are stored encrypted. The project's README describes it as intended for personal experimentation.

  • One OpenAI-compatible /v1 endpoint for many providers
  • Automatic failover when a free provider is rate-limited
  • Per-key usage tracking against free-tier caps
  • Custom OpenAI-compatible chat, embedding, image and audio endpoints can be added

Prerequisites: what you need before installing

The self-host option shown in the README uses Docker. You also need API keys from the free-tier providers you want to use. The README lists supported providers such as Google, Groq, Cerebras, Mistral, OpenRouter, Cloudflare, Cohere, NVIDIA and HuggingFace, plus a custom provider slot for your own endpoints.

  • Docker installed on the machine that will host FreeLLMAPI
  • Git, to clone the repository
  • At least one free-tier API key from a supported provider

Install FreeLLMAPI with Docker

Start by cloning the repository from GitHub. The README's install guide covers the Docker Compose self-host path in more detail, so follow the install guide linked from the repository for the remaining steps.

The README also links a desktop download for macOS and Windows, and a Docker image is published on the GitHub container registry.

git clone https://github.com/tashfeenahmed/freellmapi

Add your provider keys

Once the app is running, add API keys for the providers you want to use. Keys are stored encrypted. The more providers you add, the more free capacity the router has to draw on when one provider is rate-limited.

For a custom endpoint such as llama.cpp, LM Studio, vLLM or a local Ollama, the README says you can add it from the Keys page.

  • Add keys from providers that offer free tiers
  • Add a custom OpenAI-compatible endpoint from the Keys page if you run your own model server

Point your tools at the /v1 endpoint

Any OpenAI-compatible client can use FreeLLMAPI by setting its base URL to your local server's /v1 path. Most coding agents can also be configured with a single setup command that reads the live catalog and merges into the existing config.

The README gives this example for Claude Code. Replace the placeholder with the unified key your install provides.

  • Set the base URL to your server's /v1 endpoint in any OpenAI client library
  • Use the unified key shown by your install as the API key
  • Check the supported agents guide for per-tool setup commands
npx freellmapi setup-claude --url http://localhost:3001 --api-key <unified-key>

Know the limits before you rely on it

FreeLLMAPI is built for personal experimentation. Free tiers can change without notice, so quotas and available models may shift. It is not a production service with guaranteed uptime or SLAs.

If you need a paid, supported gateway, or a local model without any remote provider, the README's comparison and the projects it names are worth reviewing instead.

  • Not suited to production services that need reliable uptime
  • Free-tier quotas and models can change without notice
  • Requires you to manage API keys and a local server

What people use it for

  • Testing several free LLM providers through one API without changing your code
  • Pointing OpenAI-compatible tools or coding agents at a single local endpoint
  • Students or hobbyists trying models without paying for usage
  • Keeping a chat or coding tool working when one free provider hits a rate limit

Frequently asked questions

Is FreeLLMAPI free to use?

The project is released under the MIT licence, and it is built to aggregate free tiers from providers. The README describes it as intended for personal experimentation only. Check each provider's own terms, since free tiers can change without notice.

What do I need to run FreeLLMAPI?

The self-host option in the README uses Docker. You also need API keys from the free-tier providers you want to route to. Git is needed to clone the repository.

Can I use FreeLLMAPI in production?

The README frames it as personal experimentation only, and free tiers do not come with guaranteed quotas or SLAs. For production services that need reliable uptime, choose a service built for that purpose.

How is FreeLLMAPI different from LiteLLM, OpenRouter or Ollama?

FreeLLMAPI focuses on aggregating free tiers from many providers behind one OpenAI-compatible endpoint, with routing and failover. LiteLLM, OpenRouter and Ollama are comparable projects; the README includes a comparison section, so check it for detailed differences.

What happens when a free provider is rate-limited?

The router switches the request to another provider that is available. It also tracks per-key usage so you stay under each free-tier cap.

Are my API keys safe?

The README says keys are stored encrypted. You still control the server you run it on, so protect that machine as you would any system holding credentials.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

FreeLLMAPI suits developers, students and hobbyists who want to experiment with several free LLM providers through one OpenAI-compatible endpoint, especially for coding agents and automatic failover. It is not a fit for production services, workloads needing guaranteed quotas, or anyone who does not want to manage API keys and a local server.

Install options & downloads