Skip to content
Get packs

Segment Anything Model (SAM)

Lets the agent write code that cuts out any object in an image using clicks, boxes, or automatic mask generation.

Source Download skill
Repository
Orchestra-Research/AI-Research-SKILLs
Stars
14,195
Licence
MIT
Files
3 files

What it does

When this skill is installed, the agent knows how to use Meta AI's Segment Anything Model, an image model that can outline objects it has never been specifically trained to recognise. Given a point clicked on an object, a rough box, or nothing at all, the agent can write Python code that produces precise object masks, including automatic generation of masks for every object in an image. The skill mainly teaches setup, checkpoint choice, prompt types and code patterns rather than performing segmentation itself.

Great for

  • Developers building annotation tools or image-editing features who need object masks
  • ML engineers generating segmentation training data for other vision models
  • Researchers working with medical, satellite or other non-standard images

Skip it if

  • Real-time object detection with named classes, where YOLO or Detectron2 is a better fit (the skill itself recommends these)
  • Video segmentation, where the skill points to SAM 2 instead
  • Anyone expecting a ready-made app: this only guides an agent writing code, and the model needs a GPU and multi-gigabyte downloads to run well

Try asking

  • “Write a Python script that segments the main object in photo.jpg from a click at pixel (500, 375)”
  • “Generate masks for every object in this folder of product images and save them as PNGs”
  • “Build a bounding-box based cutout tool using SAM with the HuggingFace transformers library”
Step-by-step guide: how to use Segment Anything Model (SAM)Install, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    segment-anything.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip segment-anything.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip segment-anything.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Specify the model size you want (ViT-B for speed, ViT-H for accuracy) and whether you have a GPU
  • Review downloaded checkpoints and any scripts the agent generates before running them, since they run on your machine
  • Provide example coordinates or a bounding box so the agent can produce a working prompt on the first try

What's inside

  • references/advanced-usage.md
  • references/troubleshooting.md
  • SKILL.md

Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.