The best ONNX Runtime alternatives

8 tools that do a similar job to ONNX Runtime, 8 of them open source. Each entry says what it does and who it suits, so you can pick the one that fits instead of the most famous.

You're replacing

ONNX Runtime

A cross-platform library for running and training machine learning models quickly.

1. 1

   Petals

   Open source · MIT · ★ 10.7k

   Run big language models by sharing the work across volunteers' GPUs, BitTorrent-style.

   Best for: Developers who want to experiment with large open models like Llama 3.1 or BLOOM without owning the hardware to run them alone; Researchers prompt-tuning large models on their own tasks

   Full review ONNX Runtime vs Petals
2. 2

   Weights & Biases

   Open source · MIT · ★ 11.3k

   Python library to track, visualize, and manage machine learning experiments and models.

   Best for: Machine learning engineers who want to compare training runs over time; Researchers tuning hyperparameters across many experiments

   Full review ONNX Runtime vs Weights & Biases
3. 3

   Silero VAD

   Open source · MIT · ★ 10.6k

   Pre-trained voice activity detector that finds where speech occurs in audio, for Python, ONNX, and other runti

   Best for: Developers adding speech detection to Python apps; Teams building telephony, call-center, or voice bot pipelines

   Full review ONNX Runtime vs Silero VAD
4. 4

   spaCy

   Open source · MIT · ★ 34k

   Python library for processing and analyzing text, with pretrained models for 70+ languages.

   Best for: Python developers building text-processing features into an application; Data scientists who need fast, production-oriented NLP pipelines

   Full review ONNX Runtime vs spaCy
5. 5

   PyG

   Open source · MIT · ★ 24.1k

   A PyTorch library for building and training graph neural networks on structured data.

   Best for: Researchers who want ready-made implementations of published graph neural network methods; Python developers already comfortable with PyTorch who want to work on graph data

   Full review
6. 6

   Nano-vLLM

   Open source · MIT · ★ 16.3k

   A small, readable Python re-implementation of vLLM for running language models quickly.

   Best for: Developers who want to understand how LLM inference engines work internally; Students studying LLM serving and optimization techniques

   Full review
7. 7

   LoRA

   Open source · MIT · ★ 13.9k

   Python package loralib for adapting large PyTorch language models with small trainable low-rank matrices.

   Best for: Machine learning engineers adapting RoBERTa, DeBERTa or GPT-2 style models in PyTorch; Researchers reproducing the LoRA paper's GLUE or GPT-2 experiments

   Full review
8. 8

   pytorch-grad-cam

   Open source · MIT · ★ 13k

   Python library that shows which parts of an image a computer vision model looked at to make its prediction.

   Best for: Researchers comparing explainability methods for image models; Engineers debugging CNN or Vision Transformer classifiers in PyTorch

   Full review

How we pick: tools in the same category that solve the same problem, projects that name each other as alternatives, and open-source projects that describe themselves as an alternative to ONNX Runtime. We don't take payment for placement.