Skip to content
Get packs

Guide · Agent skill

How to use the Fine-Tuning OpenVLA-OFT agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

This page explains how to install and use the fine-tuning-openvla-oft agent skill, a playbook that an AI agent can follow for OpenVLA-OFT, a research method for training robot control models built on the OpenVLA vision-language model. It is written for readers who have never used agent skills. By the end you will know what the skill lets the agent do, how to install it, how to phrase requests, and what hardware and safety considerations apply.

Fine-Tuning OpenVLA-OFTOrchestra-Research/AI-Research-SKILLs · MIT

Quick start

  1. 1Download and unzip the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder. Or upload it in the Claude apps under Settings, Capabilities, Skills.
  2. 2Prepare the environment. Create a conda environment with Python 3.10, install PyTorch 2.2.0 and robosuite 1.4.0, and clone the openvla-oft repository.
  3. 3Install LIBERO. Clone the LIBERO repository, install it in editable mode, and install the libero_requirements.txt packages.
  4. 4Pick a checkpoint and task suite. Match a moojink LIBERO checkpoint to its task suite name, such as libero_spatial.
  5. 5Run the evaluation. Run run_libero_eval.py with the checkpoint, task suite, center crop, 50 trials per task and seed 7.
  6. 6Check results and debug. Parse the per-task success rates from the log. If results are off, ask the agent to check normalization, LoRA merge and configuration consistency.

What does the skill make the agent do?

OpenVLA-OFT replaces the tokenized action outputs of standard OpenVLA with dedicated continuous action heads (L1 regression or diffusion), keeps the VLA backbone frozen, and adapts it with LoRA. The skill gives the agent the documented workflows for this: setting up the software environment, evaluating pretrained LIBERO simulation checkpoints, fine-tuning on LIBERO, running ALOHA server-client inference, and debugging problems such as normalization, LoRA merge, and cross-GPU issues.

The skill is for training and evaluation work. It is not a general chatbot or LLM fine-tuning guide.

  • Environment setup with Python 3.10, conda, PyTorch 2.2.0 and FlashAttention2 2.5.5
  • Evaluating official LIBERO checkpoints such as moojink/openvla-7b-oft-finetuned-libero-spatial
  • Fine-tuning on LIBERO RLDS datasets with torchrun
  • ALOHA preprocessing and server-client deployment
  • Troubleshooting configuration mismatches

How do you install the skill?

Download the ZIP from this page and unzip it into one of two locations. For a personal install, use ~/.claude/skills/. For a single project, use the .claude/skills/ folder inside that project. Alternatively, in the Claude apps, open Settings, go to Capabilities, and upload the skill under Skills.

After installing, the skill folder should contain SKILL.md and the references folder with aloha-workflow.md, config-troubleshooting.md, libero-workflow.md and paper-and-checkpoints.md.

  • Personal: ~/.claude/skills/
  • Project: .claude/skills/ inside your project
  • Claude apps: Settings → Capabilities → Skills, then upload

How do you trigger the skill with a natural request?

You do not need a special command. Describe what you want in plain language and mention OpenVLA-OFT, LIBERO, ALOHA, LoRA or the action head. The skill's description lists the situations it is meant for: reproducing paper results, training custom action heads (L1 or diffusion), deploying server-client inference for ALOHA, and debugging normalization, LoRA merge or cross-GPU issues.

Be specific about your goal and your hardware. The agent will then choose the matching workflow and reference file.

  • "Walk me through evaluating the OpenVLA-OFT LIBERO-Spatial checkpoint on my A100."
  • "Help me set up the openvla-oft conda environment with PyTorch 2.2.0."
  • "My merged LoRA model gives different actions than the training run. Help me debug the merge step."
  • "Set up server-client inference for ALOHA with the /act endpoint."

What do you need before you start?

The skill assumes a substantial GPU setup. Evaluation needs roughly 16 GB+ VRAM on a single A100 or A40 class GPU, while LIBERO fine-tuning is documented for 8x A100 at about 27 GB per GPU. ALOHA OFT+ fine-tuning is listed at about 35 GB per GPU on 8x A100. A typical laptop will not run these workflows.

You also need Python 3.10 with conda, NVIDIA GPU drivers, the cloned GitHub repositories moojink/openvla-oft and Lifelong-Robot-Learning/LIBERO, and Hugging Face checkpoints from the moojink namespace. The agent is expected to download packages and checkpoints, so the machine needs network access.

  • Python 3.10 and conda
  • PyTorch 2.2.0, torchvision, torchaudio, transformers
  • NVIDIA GPU (A100/A40 class recommended)
  • FlashAttention2 2.5.5
  • robosuite 1.4.0, peft 0.11.1, tensorflow 2.15.0, wandb, fastapi, uvicorn
conda create -n openvla-oft python=3.10 -y conda activate openvla-oft git clone https://github.com/moojink/openvla-oft.git cd openvla-oft pip3 install torch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pip3 install robosuite==1.4.0

What does a LIBERO evaluation look like?

Once the environment is set up, the skill guides you through evaluating a pretrained checkpoint. Pick a checkpoint that matches your task suite, run the evaluator, then parse the per-task success rates from the log. The official paper results used seed 7 and 50 trials per task, so these are the flags to keep if you want comparable numbers.

  • Choose a checkpoint and matching task suite: libero_spatial, libero_object, libero_goal or libero_10
  • Run the evaluator with --center_crop True and --num_trials_per_task 50
  • Parse the log for lines reporting successes per task
python experiments/robot/libero/run_libero_eval.py \ --pretrained_checkpoint moojink/openvla-7b-oft-finetuned-libero-spatial \ --task_suite_name libero_spatial \ --center_crop True \ --num_trials_per_task 50 \ --seed 7

Safety notes: scripts, network and real robots

This skill runs scripts and uses the network. It clones repositories, installs packages, and downloads checkpoints. Review the commands the agent proposes before approving them, and run them in an environment you control.

The skill covers training and evaluation. Real-robot deployment carries physical risk. Do not run a policy on physical hardware without your own safety review, emergency-stop procedures and supervision. The skill does not make a robot safe.

  • Read each shell command before it runs
  • Check that the repository and checkpoint names match the official ones
  • Keep a human supervising any real ALOHA run

Tips for getting good results

Keep configuration identical across training, evaluation and deployment. The skill's troubleshooting reference flags mismatches in the action head mode, FiLM usage, number of image streams, proprio input and LoRA rank as the most common causes of failure. Pin the exact versions, because the paper results were validated on torch 2.2.0, peft 0.11.1 and tensorflow 2.15.0; upgrading torch may require re-checking the LoRA merge and action outputs.

  • Use the same use_film, num_images_in_input and use_proprio settings everywhere
  • Do not upgrade torch casually; re-validate action outputs after any change
  • Ask the agent to open the matching reference file for your task (LIBERO, ALOHA or troubleshooting)

What people use it for

  • Reproducing OpenVLA-OFT LIBERO evaluation results for a research write-up
  • Training a custom L1 or diffusion action head on top of OpenVLA with LoRA
  • Setting up server-client inference so an ALOHA client can request actions from a GPU server
  • Debugging a LoRA merge that changes model outputs after training

Frequently asked questions

How do I install the fine-tuning OpenVLA-OFT agent skill?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use or into .claude/skills/ inside a project. In the Claude apps you can instead upload it under Settings, Capabilities, Skills.

What hardware do I need to use this skill?

Evaluation needs roughly 16 GB+ VRAM on one A100 or A40 class GPU. Fine-tuning is documented for 8x A100 GPUs, so a typical laptop will not run these workflows.

Is OpenVLA-OFT free to use?

The skill is listed under the MIT licence. The required packages and checkpoints come from their own repositories, so check each project's licence before using them.

What is the difference between OFT and OFT+?

OFT uses 2 images (front and wrist) with FiLM off and is used for LIBERO simulation. OFT+ uses 3 images (high plus left and right wrist) with FiLM on and is used for ALOHA real-world setups.

Can this skill fine-tune a general chatbot LLM?

No. The skill explicitly covers robot action generation with OpenVLA-OFT, not general LLM fine-tuning. For pi0 or pi0.5 models the source points to a different skill, fine-tuning-serving-openpi.

Why does my evaluation or merged model give different results?

Common causes listed in the troubleshooting reference are mismatched settings between training, evaluation and deployment, such as use_film, num_images_in_input or use_proprio, and changed library versions. Keep configurations identical and pin the versions the paper used.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

This skill suits researchers reproducing OpenVLA-OFT results on LIBERO or ALOHA, and robotics engineers training custom action heads with LoRA on a multi-GPU machine. If you have only a laptop, want a general LLM fine-tuning guide, or plan to run a real robot without your own safety review, pick a different resource.

Download the skill