Computer Use
Computer use lets your agent control a real desktop. It clicks, types, scrolls, and reads the screen the same way a person would. Use it for anything that needs a browser, GUI app, or terminal.
When it activates
Every run executes in a sandboxed Linux environment, so a full desktop is already available alongside your normal tools. The agent uses it when a task needs a browser, GUI, or terminal. It gets access to:
- computer: mouse clicks, keyboard input, screenshots
- bash: shell commands, file system, processes
- str_replace_based_edit_tool: read and edit files
Your existing integrations (Slack, GitHub, etc.) are available in the same run. Computer use tools and MCP integrations work side by side.
What the agent can do
Your agent gets a full Linux desktop with a browser, terminal, and file system. It can:
- Browse the web and fill out forms
- Run scripts and commands
- Open, edit, and save files
- Use any installed GUI application
- Copy, paste, drag, scroll, use keyboard shortcuts
After every action, the agent sees a screenshot and decides what to do next.
Stream events
Computer use runs produce the same SSE event stream as regular runs, plus a few extras:
Screenshots appear in the chat after each action. The web UI renders them inline; you can also access them from the stream:
Watch live
When computer use engages, a watch live link appears on the run's environment pill. It opens a read-only view of the desktop so you can watch the agent work in real time. The link is view-only (you can't take over the mouse or keyboard) and expires automatically.
Limitations
- No session resume. Desktop state resets between runs. Files you write are saved and accessible in the run's file list, but the desktop itself starts fresh each time.