How to use UFO: install, run and automate Windows apps

What UFO is and which part to use

UFO³ has two parts. UFO² runs on a single Windows machine and uses a language model to read the screen and act on apps through clicks and API calls. Galaxy, the newer part, splits a request into a graph of tasks and assigns them to different devices such as Windows, Linux, or Android.

The README suggests UFO² for single Windows automation, quick task execution, and learning agent basics. It suggests Galaxy for cross-device collaboration and complex multi-step automation, and marks it as new and recommended, though still in active development.

• UFO² (single Windows machine): labelled stable and long-term supported in the README
• Galaxy (multi-device): labelled active development in the README

Check the prerequisites before you install

The README lists Python 3.10 or 3.11 as prerequisites. For UFO² desktop automation, you need Windows, because that is where the agent controls applications.

Expect setup and configuration steps. The README describes them, so this is not a one-click consumer app.

• Python 3.10 or 3.11
• Windows, for UFO² desktop automation
• Comfort with configuration and command-line setup

Install UFO by cloning and building the repository

The install method in the README is clone-and-build. Start by cloning the repository from GitHub. After that, follow the build and configuration steps in the project documentation for the part you chose, since the full steps live in the linked README files and online docs rather than in this summary.

• Read the UFO² README (ufo/README.md) for desktop setup
• Read the Galaxy README (galaxy/README.md) for multi-device setup
• The online documentation is at https://microsoft.github.io/UFO/

Run your first automation with UFO²

For a first run, keep the scope small. UFO² uses a language model to read the screen and act on one Windows app at a time. Pick a simple, repetitive task inside a single application so you can watch what the agent does and stop it if it goes wrong.

The README highlights hybrid GUI and API actions. The agent can use clicks or API calls depending on what the app supports.

• Start with one application and one short task
• Watch the first few actions before letting it run unattended
• Use the UFO² README for the exact run commands and configuration

Try Galaxy for cross-device workflows

Galaxy's Constellation planner turns a request into a DAG of tasks with dependencies, then assigns each task to a suitable device based on capability. Independent tasks can run in parallel, and the README describes dynamic DAG editing as the plan adapts to results.

Galaxy supports Windows, Linux, and Android device types in the README. Because it is in active development, plan for changes and test before relying on it.

• Constellation: task decomposition into executable DAGs
• Asynchronous execution with parallel task coordination
• Unified AIP protocol, described in the README as WebSocket-based
• Device types listed: Windows, Linux, Android

Know the limits before you rely on it

The README labels Galaxy as in active development and UFO² as stable. Neither is described as a fully tested production system, so treat the project as a research codebase. Its README also lists Windows, Linux, and Android as device types, so it is not a macOS automation tool.

Benchmarks and numbers in the README are the project's own claims. Check the linked paper or docs before quoting them.

• Galaxy: active development
• UFO²: described as stable
• Not macOS: supported device types are Windows, Linux, Android