Skip to content
Get packs

vit-pytorch

A PyTorch library of Vision Transformer models for image classification and related vision tasks.

GitHub Download 1.17.8
Stars
25.6k
Forks
3.5k
Licence
MIT
Language
Python
Last update
5 days ago
Type
Library

What it is

This is a Python library that provides implementations of the Vision Transformer, a neural network design that classifies images using a transformer encoder instead of convolutions. The README also lists many variants from research papers, such as SimpleViT, NaViT, and MaxViT. Developers use it to build, train, or experiment with image models in PyTorch.

  • Create a ViT model for classifying images into a set number of classes
  • Train a SimpleViT model using the simplified architecture described in the README
  • Distill knowledge from a ResNet50 teacher into a smaller vision transformer
  • Access attention maps from a model to see what parts of an image it focuses on

Great for

  • Developers who want to build an image classifier in PyTorch
  • Students learning how Vision Transformers work by reading readable code
  • Researchers comparing ViT variants from different papers
  • People who want to try distillation from a CNN teacher to a smaller ViT

Not ideal for

  • People who need pretrained models ready to use: the README points to another repository (pytorch-image-models) for pretrained weights
  • Beginners with no Python or PyTorch experience, since the usage examples assume both
  • Users who want a finished end-to-end app rather than building blocks
Step-by-step guide: how to use vit-pytorchInstall, first run, examples, tips and FAQs
FightingCV vs vit-pytorch

How to get it

  1. 1

    Install with a package manager

    You'll need: Python · PyTorch

    Install from PyPI

    pip install vit-pytorch
  2. 2

    Run it

    • Install the package with pip install vit-pytorch
    • Import ViT from vit_pytorch in a Python script
    • Create a model by setting image_size, patch_size, num_classes, dim, depth, heads, and mlp_dim
    • Pass a batch of images shaped (batch, channels, height, width) to the model to get predictions
  3. 3

    Get the most out of it

    • Check that image_size is divisible by patch_size and that the number of patches is greater than 16, as the README requires
    • Try SimpleViT for a simpler architecture that the README says trains faster and better
    • Use NaViT with group_images to handle images of different resolutions in one batch
    • Use the DistillWrapper with a teacher model to train a smaller student, then call to_vit() to convert back

Downloads

Source code (.zip)main branch · straight from GitHubRelease 1.17.8Release notes and source

Strengths

  • Simple, short code that the README says is meant to be easy to read and modify
  • Covers many Vision Transformer variants and training ideas in one place
  • MIT license permits free use in personal and commercial projects

Trade-offs

  • The README says there is 'not much to code here', so it provides building blocks rather than trained models or a full training pipeline
  • Some variants have specific requirements, such as NaViT's nested tensor version needing PyTorch 2.5

How it compares

ProjectStarsTypeLevelUpdatedBest for
vit-pytorch (this)25.6kLibraryintermediate5 days agoDevelopers who want to build an image classifier in PyTorch
carla14.6kAppadvanced4 days agoAutonomous driving researchers testing perception or control software
AI-For-Beginners70.5kLearning resourcebeginner13 days agoBeginners who want a structured, self-paced introduction to AI
cvat17kAppintermediate4 days agoComputer vision teams that need to label images or video for model training
supervision51.5kLibraryintermediate4 days agoPython developers building apps on top of object detection or segmentation models
spaCy34kLibraryintermediate10 days agoPython developers building text-processing features into an application

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.