Skip to content
Get packs

RWKV Architecture Guide

Explains and helps use RWKV, a fast, memory-light RNN-style language model architecture

Source Download skill
Repository
Orchestra-Research/AI-Research-SKILLs
Stars
14,195
Licence
MIT
Files
4 files

What it does

This skill gives the agent background and code patterns for RWKV, a neural network design that mixes Transformer-style training with RNN-style step-by-step inference. Once installed, the agent can explain how RWKV differs from Transformers, write code to load RWKV models, stream text generation token by token, carry state across long inputs, and set up fine-tuning or hardware sizing. It is a reference guide that shapes the agent's answers and code; it does not itself run a model.

Great for

  • Developers experimenting with alternative language model architectures
  • Engineers planning memory-constrained or long-stream inference on NVIDIA GPUs or CPUs
  • Students or researchers comparing RWKV with Transformers, Mamba or RetNet

Skip it if

  • Users who just want to chat with a ready-made assistant, since this skill only documents RWKV and does not provide a model (the code requires downloading model weights separately)
  • Teams needing verified performance claims, since the file's comparisons are rough and some numbers (such as the 1,000,000x memory figure and the 400GB KV cache example) are illustrative estimates, not benchmarks

Try asking

  • “Show me how to load an RWKV model in Python and generate text token by token”
  • “How much VRAM do I need to run a 7B RWKV model in FP16?”
  • “Write a fine-tuning script for RWKV with PyTorch Lightning and DeepSpeed”
  • “Explain why my RWKV outputs differ when I don't pass the state between forward calls”
Step-by-step guide: how to use RWKV Architecture GuideInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 4 original files, the MIT licence and an attribution note.

    rwkv.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip rwkv.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip rwkv.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Ask for code that matches your hardware, for example strategy 'cuda fp16' on GPU or 'cpu fp32' on CPU, so you avoid out-of-memory errors
  • Check the referenced files (rwkv7.md, state-management.md) for newer RWKV-7 details, and verify version-specific code against the official docs since the skill's examples use older RWKV-4 models
  • Before running any generated code, review it and confirm the model download source; the skill's sample code expects local model paths and does not verify downloads

What's inside

  • references/architecture-details.md
  • references/rwkv7.md
  • references/state-management.md
  • SKILL.md

Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.