Ray Data Processing
Lets the agent write code for scalable, distributed data processing and ML data pipelines with Ray Data.
- Repository
- Orchestra-Research/AI-Research-SKILLs
- Stars
- 14,195
- Licence
- MIT
- Files
- 3 files
What it does
Once installed, the agent knows how to use Ray Data, an open-source library for processing large datasets across one or many machines. It can write code that reads Parquet, CSV, JSON and image files from cloud storage, applies transformations such as map, filter and groupby, runs batch inference with GPUs, and feeds data into PyTorch or TensorFlow training. The skill is a reference guide of patterns and code snippets, and the agent adapts them to your project.
Great for
- Data engineers building preprocessing or distributed ETL pipelines for ML training
- ML engineers running batch inference over large datasets, including on GPUs
- Developers who already use Ray Train or PyTorch and need efficient data loading
Skip it if
- Small datasets that fit comfortably in pandas (the skill itself recommends pandas for under 1GB), since Ray adds unnecessary complexity
- Anyone unfamiliar with Python who needs a no-code data tool; the output is code that must be installed, run and debugged
- Production jobs touching cloud storage without reviewing the generated credentials and bucket paths, because the examples read and write to s3:// and gs:// locations
Try asking
- “Write a Ray Data pipeline that reads Parquet files from S3, cleans text, and writes the result back to S3”
- “Set up batch inference with a PyTorch model on 1 GPU per worker using Ray Data”
- “Convert my Parquet dataset into a PyTorch dataloader for training with Ray Train”
- “Show me how to group a dataset by category and compute aggregates with Ray Data”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 3 original files, the MIT licence and an attribution note.
ray-data.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip ray-data.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip ray-data.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Install and test on a small local sample before pointing the pipeline at production cloud buckets.
- Tune batch_size and repartition settings to your cluster size; the skill's guidance on these is general, not measured for your workload.
- Review any code the agent generates for cloud paths, credentials and GPU requests before running it on shared infrastructure.
What's inside
- references/integration.md
- references/transformations.md
- SKILL.md
Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.