Skip to content
Get packs

DwarfStar

Local inference engine for running a few large open-weight models on consumer hardware.antirez/ds4

GitHub Download source
Stars
24.2k
Forks
2.4k
Licence
MIT
Language
C
Last update
4 days ago
Type
AI model / agent

What it is

DwarfStar (repo ds4) is a small, native C program that runs specific large language models on your own machine instead of a cloud service. It is built around DeepSeek V4 Flash and a few related models, and it targets Apple Metal, NVIDIA CUDA, and AMD ROCm hardware. It includes a command-line chat, a local HTTP server, and a native coding agent.

  • Run DeepSeek V4 Flash locally with a one-off prompt using ./ds4 -p
  • Serve a local OpenAI-style HTTP API on 127.0.0.1:8000 with ds4-server
  • Use the native ds4-agent coding agent with saved sessions in ~/.ds4/kvcache
  • Spread a model across multiple CUDA cards to serve several sessions at once

Great for

  • Developers with a high-memory Mac (64 GB or more) who want to run supported models locally
  • People with NVIDIA CUDA cards, including Ada Lovelace/L40S setups, who want a local multi-user server
  • Users with a DGX Spark or Strix Halo machine following the project's platform guides
  • Tinkerers who are comfortable asking a coding agent to adapt the code to their own hardware

Not ideal for

  • Anyone with a smaller machine and no interest in SSD streaming, since the README says the 96 GB and 128 GB sizes are what most models need
  • People who want a general GGUF runner for any model, because the README says it is deliberately narrow and uses only its own GGUF files
  • People who object to AI-assisted code, since the README says the project was developed with strong AI coding agent assistance
  • Teams needing production stability, since the README calls the software beta quality and fast changing
Step-by-step guide: how to use DwarfStarInstall, first run, examples, tips and FAQs
DwarfStar vs llama.cppDwarfStar vs Ollama

How to get it

  1. 1

    Clone and build from source

    You'll need: A platform-specific build environment as described in the linked platform guides (docs/METAL.md, docs/DGX_SPARK.md, docs/STRIX_HALO.md, docs/CUDA_MULTI_GPU.md) · Enough RAM or VRAM for the chosen model, per the README memory guidance

    Clone the repository

    git clone https://github.com/antirez/ds4.git

    Enter the project directory

    cd ds4

    Build on Apple Silicon (Metal)

    make

    Download DeepSeek V4 Flash Q2 model

    ./download_model.sh ds4f-q2
  2. 2

    Run it

    • Clone the repository and build for your platform using the matching make target
    • Download a model with ./download_model.sh ds4f-q2 (for a 96 or 128 GB machine)
    • Run a one-off prompt with ./ds4 -p "Explain Redis streams in one paragraph."
    • Or start the server with ./ds4-server --ctx 32768 and connect a client
  3. 3

    Get the most out of it

    • Read docs/MODELS.md to choose a model that fits your memory
    • Use docs/SSD_STREAMING.md if your Mac has less RAM than the model needs
    • Follow docs/CLIENTS.md to connect Pi, OpenCode, Codex CLI, or Claude Code to ds4-server
    • Use /hints on in ds4-agent for brief explanations of programming choices
    • Use /save, /list, and /switch in ds4-agent to resume past sessions without rebuilding the prompt

Downloads

Source code (.zip)main branch · straight from GitHub

Strengths

  • Narrow focus on a few models lets the code be tested end to end, according to the README
  • Supports several hardware backends (Metal, CUDA, ROCm) with platform-specific guides
  • MIT licensed, with credit to llama.cpp and GGML clearly stated in the README

Trade-offs

  • Status is beta quality and the README warns of possible regressions between releases
  • Supported models change over time; the README says a model may be removed when a better one arrives
  • Large memory requirements: the README gives 96 GB or more for many DeepSeek and GLM setups on Metal

How it compares

ProjectStarsTypeLevelUpdatedBest for
ds4 (this)24.2kAI model / agentadvanced4 days agoDevelopers with a high-memory Mac (64 GB or more) who want to run supported models locally
llama.cpp133kLibraryintermediate4 days agoDevelopers who want to run open-weight models locally without a cloud API
ollama186kAppbeginner4 days agoDevelopers who want a local model API for their own apps

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.