Skip to content
Get packs

BLIP-2 Vision-Language Guide

Captions images and answers questions about them using Salesforce's BLIP-2 model

Source Download skill
Repository
Orchestra-Research/AI-Research-SKILLs
Stars
14,195
Licence
MIT
Files
3 files

What it does

Once installed, the agent knows how to use Salesforce's BLIP-2, an AI model that can look at a picture and describe it, answer questions about it, or match images to text. The agent can write working Python code using the Hugging Face Transformers or LAVIS libraries to load the model, caption a photo, ask a visual question, or process a batch of images. The skill is a reference guide: it explains model variants, sizes, and generation settings so the agent picks a suitable version.

Great for

  • Developers building image captioning or visual question answering features
  • Data teams who need to tag or describe large image collections with a local model
  • Researchers comparing vision-language models for zero-shot image understanding

Skip it if

  • Production chatbots that need the best modern multimodal quality, since the skill itself points to newer or proprietary alternatives
  • Users without a GPU or large disk space, because the model variants are several gigabytes and the examples expect CUDA
  • Sensitive images sent to a third-party service, because the skill's code downloads models from Hugging Face and the agent may run it on private data

Try asking

  • “Write a Python script that captions every photo in a folder using BLIP-2”
  • “Show me how to ask BLIP-2 'what color is the car' about an image”
  • “Help me choose between the OPT-2.7B and FlanT5-XL BLIP-2 variants for my GPU”
  • “Batch process four images with questions using BLIP-2 with padding”
Step-by-step guide: how to use BLIP-2 Vision-Language GuideInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    blip-2.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip blip-2.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip blip-2.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Specify your GPU memory so the agent picks the smallest suitable variant, such as blip2-opt-2.7b for about 4GB
  • Ask the agent to run the code on a few sample images first to verify output before batch jobs
  • Review the troubleshooting reference and pin library versions if you hit dependency errors

What's inside

  • references/advanced-usage.md
  • references/troubleshooting.md
  • SKILL.md

Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.