Skip to content
Get packs

LlamaGPT

A self-hosted chatbot that runs Llama 2 models locally, so your chats stay on your machine.

GitHub Download source
Stars
10.9k
Forks
702
Licence
MIT
Language
TypeScript
Last update
2.5 years ago
Type
App

No commits in over a year — check for forks or alternatives before adopting.

What it is

LlamaGPT is a ChatGPT-style chat interface that runs open Llama 2 and Code Llama models on your own hardware instead of a cloud service. The project says no data leaves your device. It is packaged with Docker, so you start it with a script and then use it in a browser at localhost:3000.

  • Chat privately with a local Llama 2 model in your browser
  • Ask coding questions with the Code Llama models
  • Point an OpenAI-compatible client at the local API on port 3001
  • Run it on a home server so other devices on your network can use it

Great for

  • People who want a private chatbot and don't want to send prompts to a cloud provider
  • Users with a home server running umbrelOS who want a one-click install
  • Developers who want a local, OpenAI-style API for testing
  • Mac users with Docker who want to try local models

Not ideal for

  • People with limited RAM or no capable CPU or GPU, since the larger models need 6GB to over 40GB of memory per the README table and run slowly on small devices
  • Anyone who needs the latest hosted models or top-quality answers, because these are smaller open models and the README does not compare their quality to GPT-4
  • Users who need to load their own custom models today, since the README lists that feature as not yet done
Step-by-step guide: how to use LlamaGPTInstall, first run, examples, tips and FAQs
LlamaGPT alternativesLlamaGPT vs OpenHandsAnythingLLM vs LlamaGPTLlamaGPT vs LocalAIDB-GPT vs LlamaGPTDocsGPT vs LlamaGPTLlamaGPT vs OllamaGPT4All vs LlamaGPTllama.cpp vs LlamaGPT

How to get it

  1. 1

    Clone and build from source

    You'll need: Docker · Xcode (for M1/M2 Mac install)

    Clone the repository

    git clone https://github.com/getumbrel/llama-gpt.git

    Enter the project folder

    cd llama-gpt

    Run on any x86 or arm64 system with Docker

    ./run.sh --model 7b

    Run on M1/M2 Mac

    ./run-mac.sh --model 7b
  2. 2

    Run it

    • Install Docker, and Xcode if you are on a Mac.
    • Clone the repo and cd into it.
    • Run ./run.sh --model 7b, or ./run-mac.sh --model 7b on a Mac.
    • Wait for the model to download on first run; the README says this can take a while.
    • Open http://localhost:3000 in your browser.
  3. 3

    Get the most out of it

    • Add --with-cuda to ./run.sh if you have an Nvidia GPU
    • Switch to 13b, 70b, or code-7b, code-13b, code-34b for other models, keeping the memory limits in mind
    • Use the OpenAI-compatible API at http://localhost:3001 with its docs at /docs
    • Install it on an umbrelOS home server for a one-click setup
    • Check the README benchmarks to match a model size to your hardware

Downloads

Source code (.zip)master branch · straight from GitHubOfficial website & docsapps.umbrel.com/app/llama-gpt

Strengths

  • Runs locally with no data sent out, as the README states
  • Supports several model sizes, from 7B to 70B, plus Code Llama variants
  • Offers an OpenAI-compatible API and one-command Docker setup

Trade-offs

  • Speed varies a lot by hardware; the README benchmarks range from 0.9 to 54 tokens per second
  • Large models need a lot of memory and disk space
  • Custom models and switching models in the UI are still on the roadmap

How it compares

ProjectStarsTypeLevelUpdatedBest for
llama-gpt (this)10.9kAppintermediate2.5 years agoPeople who want a private chatbot and don't want to send prompts to a cloud provider
ollama186kAppbeginner4 days agoDevelopers who want a local model API for their own apps
gpt4all77.3kAppbeginner1.4 years agoPeople who want to chat with a language model without sending their text to a cloud service
llama.cpp133kLibraryintermediate4 days agoDevelopers who want to run open-weight models locally without a cloud API
botpress15kDev toolintermediate6 days agoDevelopers who want to write integrations and bots as TypeScript code using the Botpress CLI and SDK
OpenHands92.5kAppintermediate4 days agoDevelopers who want coding agents running on a server while their laptop is off

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.