Computer Use

Computer use gives the agent a real Linux desktop: click, type, scroll, screenshot. Use it when the task needs a browser, GUI app, or terminal.

When it activates

Every sandboxed run already has a desktop alongside normal tools. The agent uses it when the task needs one:

computer: mouse, keyboard, screenshots
bash: shell, filesystem, processes
str_replace_based_edit_tool: read and edit files

Integrations (Slack, GitHub, etc.) work in the same run.

Python

from m8tes import M8tes

client = M8tes()

with client.runs.create(
    agent_id=YOUR_AGENT_ID,
    message="Open a browser, go to example.com, and tell me the page title.",
) as stream:
    for chunk in stream.iter_text():
        print(chunk, end="", flush=True)

What the agent can do

Browse the web and fill out forms
Run scripts and commands
Open, edit, and save files
Use installed GUI apps
Copy, paste, drag, scroll, shortcuts

After each action the agent sees a screenshot and decides the next step.

Stream events

Same SSE stream as other runs, plus:

Event	When	Key fields
`sandbox-connecting`	Environment starting	`message`
`sandbox-connected`	Desktop ready	`message`, `duration_ms`
`tool_use`	Agent takes an action	`name` (computer/bash/editor), `input`
`tool_result`	Action result	`content` (screenshot for computer actions)
`text-delta`	Agent text	`event.delta`

Screenshots show in chat after each action. From the stream:

Python

with client.runs.create(agent_id=YOUR_AGENT_ID, message="Take a screenshot of the dashboard") as stream:
    for event in stream:
        if event.type == "tool_result":
            for block in event.content or []:
                if block.get("type") == "image":
                    data = block["source"]["data"]  # base64 PNG

Watch live

When computer use engages, a watch live link appears on the run's environment pill. Read-only view of the desktop; expires automatically. You cannot take over mouse or keyboard.

Limitations

No session resume. Desktop state resets between runs. Written files stay in the run's file list; the desktop starts fresh.

Next: Tools · Streaming · Files