Guide · Agent skill
How to use the Ray Data agent skill: setup and examples
Updated · written with AI from the project's own documentation and checked against our catalogue data
The ray-data skill is a reference guide that teaches an AI agent how to write code with Ray Data, an open-source library for processing large datasets on one machine or many. This guide explains what the skill changes, how to install it, how to ask for help in plain language, and what to check before running the code it produces. By the end you will know whether it fits your project and how to use it safely.
Ray Data ProcessingOrchestra-Research/AI-Research-SKILLs · MIT Download the skill Quick start
- 1Download and unzip the skill. Download the ray-data ZIP from this page and unzip it into ~/.claude/skills/ or your project's .claude/skills/ folder. In the Claude apps, you can instead upload it under Settings → Capabilities → Skills.
- 2Install the Ray Data library. Run pip install -U 'ray[data]' in the Python environment your project uses. Also make sure pyarrow and pandas are installed.
- 3Set up storage access. If your job reads from S3 or Google Cloud Storage, make sure your credentials for that storage are configured before you run the code.
- 4Describe your data job. Ask the agent in plain words what data you have, where it is, and what you want back, such as batch inference or preprocessing.
- 5Review the generated code. Check bucket paths, output locations and credential handling. Confirm the code matches what you asked for.
- 6Test on a small sample. Run the code on a small slice of your data first, then scale up once the output looks correct.
What the ray-data skill makes the agent do
Once the skill is installed, the agent knows the Ray Data patterns in the skill's SKILL.md and reference files. It can write code that reads Parquet, CSV, JSON and image files from cloud storage, applies transformations such as map, filter and groupby, runs batch inference with GPUs, and feeds data into PyTorch or TensorFlow training.
The skill is a set of patterns and code snippets, not a program that processes your data by itself. The agent adapts these patterns to your project, and you still need to install, run and debug the code it writes.
- Reads: Parquet, CSV, JSON, images, plus Python lists, ranges and pandas DataFrames
- Transforms: map_batches (vectorized), map (row by row), filter, flat_map, groupby
- Runs: batch inference and GPU-accelerated preprocessing
- Connects to: Ray Train, PyTorch and TensorFlow
Install the ray-data skill from the download ZIP
Download the ray-data ZIP from this page and unzip it. For a personal setup, place the skill folder in ~/.claude/skills/. To keep it only for one project, place it in that project's .claude/skills/ folder instead.
In the Claude apps you can also upload the skill under Settings → Capabilities → Skills, without touching folders.
- Personal use: unzip into ~/.claude/skills/
- Single project: unzip into the project's .claude/skills/ folder
- Claude apps: Settings → Capabilities → Skills, then upload
Install the Ray Data library your code needs
The skill only writes instructions and code. The code needs the ray[data] library, plus pyarrow and pandas, installed in your Python environment. The skill's own quick start uses this command.
- Requires Python
- Requires pyarrow and pandas
- Cloud examples need access to storage such as AWS S3 or Google Cloud Storage
- GPU and framework examples need GPUs and PyTorch or TensorFlow
pip install -U 'ray[data]'How to trigger the skill with natural requests
You do not need special commands. Describe the data job in plain words and mention the outcome you want. The agent uses the skill when your request matches its use cases: batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.
Give the agent the details it needs: where the data lives, its format, and what the output should look like. The skill's examples use paths such as s3:// and gs:// buckets, so say which bucket or folder you mean.
- Example request: Write a Ray Data script that reads Parquet files from my S3 bucket, lowercases the text column and writes the result back as Parquet.
- Example request: Set up batch inference with Ray Data on my GPU machine, using the model in this folder over the images in my bucket.
- Example request: Load my training Parquet data with Ray Data and feed it into a PyTorch training loop with Ray Train.
- Example request: Filter out rows where score is 0.5 or lower, then group by category and count.
Review the code before you run it
The examples read and write to s3:// and gs:// locations, so the generated code may use real buckets and credentials. Check every bucket path, output location and credential setting before you run a job, especially for production data.
The skill does not run scripts on its own, but it does use the network, because Ray Data reads from and writes to cloud storage. For small data, the skill itself recommends pandas for datasets under 1GB, since Ray adds complexity you may not need.
- Confirm input and output paths point where you expect
- Keep credentials out of code you share
- Test on a small sample before a full run
- Use pandas instead for small single-machine data
What people use it for
- Batch inference over large image or text sets using GPUs
- Preprocessing Parquet data before ML training
- Feeding training data into PyTorch or TensorFlow through Ray Train
- Distributed ETL that cleans and transforms files from cloud storage
Frequently asked questions
How do I install the ray-data agent skill?
Download the ray-data ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder for one project. In the Claude apps, you can instead upload it under Settings → Capabilities → Skills.
What do I need before the skill works?
You need Python, the ray[data] library installed with pip install -U 'ray[data]', plus pyarrow and pandas. Cloud examples need storage credentials such as AWS S3 or Google Cloud Storage, and GPU or framework examples need GPUs and PyTorch or TensorFlow.
Is the ray-data skill free?
The skill is listed under the MIT licence. Ray Data is an open-source library, so its use is governed by its own open-source licence, which you should check in its project repository.
Does the skill run code on my computer?
No. The skill is listed as not running scripts. It does use the network, because the code it writes can read from and write to cloud storage, so review paths and credentials first.
When should I use pandas or Dask instead of Ray Data?
For small data under about 1GB that fits on one machine, pandas is simpler and the skill itself recommends it. Dask suits tabular data and SQL-like operations, and Spark suits enterprise ETL and SQL queries, according to the skill's guidance.
Why does the agent's code fail when I run it?
The skill gives patterns and snippets, so the generated code still needs to be run and debugged in your environment. Check that ray[data], pyarrow and pandas are installed, that your storage credentials work, and that your bucket paths exist.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use the ray-data skill if you are a data engineer or ML engineer building distributed preprocessing, batch inference or ETL pipelines, and you are comfortable reviewing and running Python code. Pick pandas for small datasets, Dask or Spark for their specific strengths, or a no-code tool if you do not work in Python.
Download the skill