Skip to content
Get packs

localGPT

Chat with your own documents using a local AI model; files stay on your machine.

GitHub Download source
Stars
22.2k
Forks
2.5k
Licence
MIT
Language
Python
Last update
2 months ago
Type
App

What it is

LocalGPT is a document question-answering app. You load PDFs, Word files, HTML, Markdown or text files, then ask questions in plain English and get answers grounded in those files. It runs models through Ollama on your own computer, so the README says no data leaves your device.

  • Ask questions across a folder of PDFs or reports and see which chunks the answer came from
  • Summarise a document collection using per-document overviews
  • Build your own app on top of the RAG API on port 8001
  • Compare answers with and without retrieval, since the router picks RAG or direct LLM per query

Great for

  • People who want to query private documents without sending them to a cloud service
  • Developers who want a local RAG backend with a REST API to build on
  • Users comfortable running Docker or Python and installing Ollama
  • Anyone who wants to inspect and tweak retrieval settings through a config file

Not ideal for

  • People with limited hardware: the README recommends 8GB+ RAM and 16GB+ for comfort
  • Non-technical users who want a one-click install, since setup involves Ollama, model pulls and a terminal
  • Anyone needing hosted, multi-user deployment; the README describes a local-device tool
Step-by-step guide: how to use localGPTInstall, first run, examples, tips and FAQs
AnythingLLM vs localGPT

How to get it

  1. 1

    Run with Docker

    You'll need: Python 3.10+ · Node.js 20+ and npm · Ollama · Docker (optional)

    Clone the repository

    git clone https://github.com/PromtEngineer/localGPT.git

    Start Ollama and pull the models (Docker path)

    ollama pull qwen3.5:9b

    Start the Docker deployment

    ./start-docker.sh
  2. 2

    Run it

    • Install Ollama and pull the qwen3.5:9b and qwen3.5:4b models
    • Run ollama serve in one terminal
    • Run ./start-docker.sh in another terminal
    • Open http://localhost:3000 in your browser
    • Create an index from your documents, then start a chat
  3. 3

    Get the most out of it

    • Enable Late Chunking with retrieval.latechunk.enabled if your conversations are multi-turn with drifting phrasing, per the README
    • Turn on answer verification with verification.enabled to append a confidence percentage, though the README says it annotates rather than changes answers
    • Use the streaming phases over Server-Sent Events to watch each pipeline stage
    • Set options per request or via environment variables, since rag_system/main.py holds every default

Downloads

Source code (.zip)main branch · straight from GitHub

Strengths

  • Privacy-focused design: the README states documents and queries stay on the local device
  • Shows source chunks with answers, which helps check results
  • Open MIT license and an active project with a large star count

Trade-offs

  • Requires local hardware and model downloads; the README lists 8GB+ RAM minimum
  • Several advanced features are off by default and need config changes to enable
  • The README is long and layered, so first-time setup takes some reading

How it compares

ProjectStarsTypeLevelUpdatedBest for
localGPT (this)22.2kAppintermediate2 months agoPeople who want to query private documents without sending them to a cloud service
anything-llm67.8kAppbeginner7 days agoPeople who want a private ChatGPT-style assistant over their own PDFs and text files

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.