Skip to content
Get packs

Guide · Open source

How to use BitNet: clone, build and run 1-bit LLMs on CPU

Updated · written with AI from the project's own documentation and checked against our catalogue data

BitNet is Microsoft's official inference framework for 1-bit large language models, published as bitnet.cpp under the MIT licence. It is built for developers and researchers who want to run BitNet b1.58 models on an ordinary CPU, with less memory and energy than full-precision models. This guide explains what it is, how to get the code, and what to check before you start.

bitnet.cppmicrosoft/BitNet · ★ 41.4k · MIT

Quick start

  1. 1Clone the repository. Run git clone https://github.com/microsoft/BitNet to download the source code to your machine.
  2. 2Open the project folder. Change into the cloned BitNet directory and read the README for build instructions.
  3. 3Build from source. Follow the build-from-source section for your operating system and CPU type.
  4. 4Pick a supported model. Choose a BitNet model such as BitNet-b1.58-2B-4T and check the kernel table for your CPU.
  5. 5Run and test. Run the model locally and measure speed and memory use against your baseline.

What is BitNet and who is it for?

bitnet.cpp provides optimized kernels for 1-bit LLMs such as the BitNet b1.58 family, which stores weights with roughly 1.58 bits. The project describes fast and lossless inference of these models on CPU, with GPU support also mentioned in the README and NPU support listed as coming next.

It suits developers who want to run BitNet b1.58 models locally on a CPU, researchers exploring low-bit or ternary inference, and teams comparing speed and energy use against full-precision inference. It is not a ready-made chat app: setup requires building from source and using the command line.

  • Good fit: developers, researchers and engineers testing BitNet models
  • Poor fit: people who need standard non-BitNet models without conversion, or who want a point-and-click chat app
  • NPU support is not available yet according to the README

Get the BitNet source code

The README's install method is clone-and-build. Start by cloning the official repository from GitHub. The README does not list specific prerequisites, so check the repository's build instructions before you begin.

git clone https://github.com/microsoft/BitNet

Build bitnet.cpp from source

The README points to a build-from-source section in the repository for compiling on your own CPU. Follow the steps there for your operating system and processor. Building from source is required; the project does not offer a packaged binary in this guide's source.

  • Open the cloned BitNet folder
  • Follow the 'Build from source' instructions in the README
  • Use the GPU guide in the gpu folder only if you want GPU inference

Choose a supported model and kernel

The README lists official models with a kernel table. BitNet-b1.58-2B-4T is the first official BitNet b1.58 model, with 2.4B parameters, trained on 4 trillion tokens. The BitNet-embedding models at 0.6B and 270M parameters are 1-bit embedding models that use I2_S conversion on x86 CPUs.

The kernel table shows which kernels (I2_S, TL1, TL2) each model supports on x86 and ARM CPUs. Check that row before choosing a model, because not every kernel works on every CPU type.

  • BitNet-b1.58-2B-4T: 2.4B parameters, chat-ready conversational mode
  • BitNet-embedding-0.6B: 0.6B parameters, I2_S on x86
  • BitNet-embedding-270M: 270M parameters, designed for resource-constrained environments

Compare performance and energy claims

The README reports speedups of 1.37x to 5.07x on ARM CPUs and 2.37x to 6.17x on x86 CPUs compared with full-precision inference, with energy reductions of 55.4% to 70.0% on ARM and 71.9% to 82.2% on x86. It also says a 100B BitNet b1.58 model can run on a single CPU at around 5 to 7 tokens per second.

These figures come from the project's technical report and depend on hardware and model size. Test on your own machine before relying on them.

Troubleshooting and next steps

If a build fails or a model does not load, first confirm that the model is a BitNet model and that the kernel matches your CPU architecture. The README links to a technical report, a Hugging Face collection and an online demo, which you can use to check expected behaviour before debugging locally.

For embedding models, the project has a separate I2_S conversion guide in the docs folder.

  • Check the model is a 1-bit BitNet model
  • Confirm the kernel is listed as supported for your CPU
  • Read the I2_S guide in docs for embedding conversion

What people use it for

  • Running a BitNet b1.58 model locally on a CPU without a GPU
  • Comparing inference speed and energy use against full-precision models
  • Testing 1-bit embedding models with I2_S conversion on x86 CPUs
  • Researching low-bit and ternary model inference

Frequently asked questions

Is BitNet free to use?

Yes. The repository is licensed under the MIT licence, which permits use, modification and redistribution. Check the licence file in the repository for the full terms.

What do I need to run BitNet?

The README does not list specific prerequisites. You clone the repository and build from source, so you need a working build environment and command line access. Follow the build instructions in the repo for your system.

Can BitNet run standard models like Llama?

Not directly. The project targets 1-bit models such as BitNet b1.58, so standard non-BitNet models need conversion, and the README does not describe conversion for those.

Does BitNet support GPUs?

The README mentions an official GPU inference kernel with a separate guide in the gpu folder. NPU support is listed as coming next.

How does BitNet compare with llama.cpp or ollama?

BitNet focuses on 1-bit models and their kernels, while llama.cpp and ollama are general tools for many model formats. Choose based on whether your model is a 1-bit BitNet model and whether you want a command-line build or a simpler local runner.

What is I2_S conversion?

I2_S is a kernel format used for converting BitNet embedding models for optimized inference on x86 CPUs. The project provides a separate guide in the docs folder that explains the process.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

BitNet is a good choice for developers and researchers who want to run BitNet b1.58 or 1-bit embedding models on a CPU and measure speed and energy use. If you want a ready-made chat app or need to run standard non-BitNet models, pick a general tool such as ollama or llama.cpp instead.

Install options & downloads