diff --git a/docs/docs.json b/docs/docs.json index c3f116fcf45..18cd983517a 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -1180,6 +1180,7 @@ "plugins/community", "plugins/bundles", "plugins/codex-harness", + "plugins/codex-computer-use", "plugins/google-meet", "plugins/webhooks", "plugins/voice-call", diff --git a/docs/plugins/codex-computer-use.md b/docs/plugins/codex-computer-use.md new file mode 100644 index 00000000000..b119eebbb3f --- /dev/null +++ b/docs/plugins/codex-computer-use.md @@ -0,0 +1,186 @@ +--- +summary: "Set up Codex Computer Use for Codex-mode OpenClaw agents" +title: "Codex Computer Use" +read_when: + - You want Codex-mode OpenClaw agents to use Codex Computer Use + - You are configuring computerUse for the bundled Codex plugin + - You are troubleshooting /codex computer-use status or install +--- + +Computer Use is a Codex-native MCP plugin for local desktop control. OpenClaw +does not vendor the desktop app, execute desktop actions itself, or bypass +Codex permissions. The bundled `codex` plugin only prepares Codex app-server: +it enables Codex plugin support, finds or installs the configured Codex +Computer Use plugin, checks that the `computer-use` MCP server is available, and +then lets Codex own the native MCP tool calls during Codex-mode turns. + +Use this page when OpenClaw is already using the native Codex harness. For the +runtime setup itself, see [Codex harness](/plugins/codex-harness). + +## Quick setup + +Set `plugins.entries.codex.config.computerUse` when Codex-mode turns must have +Computer Use available before a thread starts: + +```json5 +{ + plugins: { + entries: { + codex: { + enabled: true, + config: { + computerUse: { + autoInstall: true, + }, + }, + }, + }, + }, + agents: { + defaults: { + model: "openai/gpt-5.5", + embeddedHarness: { + runtime: "codex", + }, + }, + }, +} +``` + +With this config, OpenClaw checks Codex app-server before each Codex-mode turn. +If Computer Use is missing but Codex app-server has already discovered an +installable marketplace, OpenClaw asks Codex app-server to install or re-enable +the plugin and reload MCP servers. If setup still cannot make the MCP server +available, the turn fails before the thread starts. + +## Commands + +Use the `/codex computer-use` commands from any chat surface where the `codex` +plugin command surface is available: + +```text +/codex computer-use status +/codex computer-use install +/codex computer-use install --source +/codex computer-use install --marketplace-path +/codex computer-use install --marketplace +``` + +`status` is read-only. It does not add marketplace sources, install plugins, or +enable Codex plugin support. + +`install` enables Codex app-server plugin support, optionally adds a configured +marketplace source, installs or re-enables the configured plugin through Codex +app-server, reloads MCP servers, and verifies that the MCP server exposes tools. + +## Marketplace choices + +OpenClaw uses the same app-server API that Codex itself exposes. The +marketplace fields choose where Codex should find `computer-use`. + +| Field | Use when | Install support | +| -------------------- | --------------------------------------------------------------- | -------------------------------------------------------- | +| No marketplace field | You want Codex app-server to use marketplaces it already knows. | Yes, when app-server returns a local marketplace. | +| `marketplaceSource` | You have a Codex marketplace source app-server can add. | Yes, for explicit `/codex computer-use install`. | +| `marketplacePath` | You already know the local marketplace file path on the host. | Yes, for explicit install and turn-start auto-install. | +| `marketplaceName` | You want to select one already registered marketplace by name. | Yes only when the selected marketplace has a local path. | + +Fresh Codex homes may need a short moment to seed their official marketplaces. +During install, OpenClaw polls `plugin/list` for up to +`marketplaceDiscoveryTimeoutMs` milliseconds. The default is 60 seconds. + +If multiple known marketplaces contain Computer Use, OpenClaw prefers +`openai-bundled`, then `openai-curated`, then `local`. Unknown ambiguous matches +fail closed and ask you to set `marketplaceName` or `marketplacePath`. + +## Remote catalog limit + +Codex app-server can list and read remote-only catalog entries, but it does not +currently support remote `plugin/install`. That means `marketplaceName` can +select a remote-only marketplace for status checks, but installs and re-enables +still need a local marketplace via `marketplaceSource` or `marketplacePath`. + +If status says the plugin is available in a remote Codex marketplace but remote +install is unsupported, run install with a local source or path: + +```text +/codex computer-use install --source +/codex computer-use install --marketplace-path +``` + +## Configuration reference + +| Field | Default | Meaning | +| ------------------------------- | -------------- | ------------------------------------------------------------------------------ | +| `enabled` | inferred | Require Computer Use. Defaults to true when another Computer Use field is set. | +| `autoInstall` | false | Install or re-enable from already discovered marketplaces at turn start. | +| `marketplaceDiscoveryTimeoutMs` | 60000 | How long install waits for Codex app-server marketplace discovery. | +| `marketplaceSource` | unset | Source string passed to Codex app-server `marketplace/add`. | +| `marketplacePath` | unset | Local Codex marketplace file path containing the plugin. | +| `marketplaceName` | unset | Registered Codex marketplace name to select. | +| `pluginName` | `computer-use` | Codex marketplace plugin name. | +| `mcpServerName` | `computer-use` | MCP server name exposed by the installed plugin. | + +Turn-start auto-install intentionally refuses configured `marketplaceSource` +values. Adding a new source is an explicit setup operation, so use +`/codex computer-use install --source ` once, then let +`autoInstall` handle future re-enables from discovered local marketplaces. + +## What OpenClaw checks + +OpenClaw reports a stable setup reason internally and formats the user-facing +status for chat: + +| Reason | Meaning | Next step | +| ---------------------------- | ------------------------------------------------------ | --------------------------------------------- | +| `disabled` | `computerUse.enabled` resolved to false. | Set `enabled` or another Computer Use field. | +| `marketplace_missing` | No matching marketplace was available. | Configure source, path, or marketplace name. | +| `plugin_not_installed` | Marketplace exists, but the plugin is not installed. | Run install or enable `autoInstall`. | +| `plugin_disabled` | Plugin is installed but disabled in Codex config. | Run install to re-enable it. | +| `remote_install_unsupported` | Selected marketplace is remote-only. | Use `marketplaceSource` or `marketplacePath`. | +| `mcp_missing` | Plugin is enabled, but the MCP server is unavailable. | Check Codex Computer Use and OS permissions. | +| `ready` | Plugin and MCP tools are available. | Start the Codex-mode turn. | +| `check_failed` | A Codex app-server request failed during status check. | Check app-server connectivity and logs. | +| `auto_install_blocked` | Turn-start setup would need to add a new source. | Run explicit install first. | + +The chat output includes the plugin state, MCP server state, marketplace, tools +when available, and the specific message for the failing setup step. + +## macOS permissions + +Computer Use is macOS-specific. The Codex-owned MCP server may need local OS +permissions before it can inspect or control apps. If OpenClaw says Computer Use +is installed but the MCP server is unavailable, verify the Codex-side Computer +Use setup first: + +- Codex app-server is running on the same host where desktop control should + happen. +- The Computer Use plugin is enabled in Codex config. +- The `computer-use` MCP server appears in Codex app-server MCP status. +- macOS has granted the required permissions for the desktop-control app. +- The current host session can access the desktop being controlled. + +OpenClaw intentionally fails closed when `computerUse.enabled` is true. A +Codex-mode turn should not silently proceed without the native desktop tools +that the config required. + +## Troubleshooting + +**Status says not installed.** Run `/codex computer-use install`. If the +marketplace is not discovered, pass `--source` or `--marketplace-path`. + +**Status says installed but disabled.** Run `/codex computer-use install` again. +Codex app-server install writes the plugin config back to enabled. + +**Status says remote install is unsupported.** Use a local marketplace source or +path. Remote-only catalog entries can be inspected but not installed through the +current app-server API. + +**Status says the MCP server is unavailable.** Re-run install once so MCP +servers reload. If it remains unavailable, fix the Codex Computer Use app, +Codex app-server MCP status, or macOS permissions. + +**Turn-start auto-install refuses a source.** This is intentional. Add the +source with explicit `/codex computer-use install --source ` +first, then future turn-start auto-install can use the discovered local +marketplace. diff --git a/docs/plugins/codex-harness.md b/docs/plugins/codex-harness.md index 209b5713d48..022eccc1697 100644 --- a/docs/plugins/codex-harness.md +++ b/docs/plugins/codex-harness.md @@ -544,14 +544,15 @@ same reviewed file as the rest of the Codex harness setup. ## Computer use -Computer Use is a Codex-native MCP plugin. OpenClaw does not vendor the desktop -control app or execute desktop actions itself; it enables Codex app-server -plugins, installs the configured Codex marketplace plugin when requested, checks -that the `computer-use` MCP server is available, and then lets Codex handle the -native MCP tool calls during Codex-mode turns. +Computer Use is covered in its own setup guide: +[Codex Computer Use](/plugins/codex-computer-use). -Set `plugins.entries.codex.config.computerUse` when you want Codex-mode turns to -require Computer Use: +The short version: OpenClaw does not vendor the desktop-control app or execute +desktop actions itself. It prepares Codex app-server, verifies that the +`computer-use` MCP server is available, and then lets Codex handle the native +MCP tool calls during Codex-mode turns. + +Minimal config: ```json5 { @@ -578,28 +579,7 @@ require Computer Use: } ``` -With no marketplace fields, OpenClaw asks Codex app-server to use its discovered -marketplaces. On a fresh Codex home, app-server seeds the official curated -marketplace and OpenClaw follows the same loading shape as Codex: it polls -`plugin/list` during install before treating Computer Use as unavailable. The -default discovery wait is 60 seconds and can be tuned with -`marketplaceDiscoveryTimeoutMs`. If multiple known Codex marketplaces contain -Computer Use, OpenClaw uses the Codex marketplace preference order before -failing closed for unknown ambiguous matches. - -Use `marketplaceSource` for a non-default Codex marketplace source that -app-server can add, or `marketplacePath` for a local marketplace file that -already exists on the machine. If the marketplace is already registered with -Codex app-server, use `marketplaceName` instead. `marketplaceName` can also -select a remote-only Codex catalog entry for status checks, but Codex app-server -does not yet support remote `plugin/install`; installs and re-enables still need -`marketplaceSource` or `marketplacePath`. The defaults are -`pluginName: "computer-use"` and `mcpServerName: "computer-use"`. -For safety, turn-start auto-install only uses marketplaces app-server has -already discovered. Use `/codex computer-use install` for explicit installs from -a configured `marketplaceSource` or `marketplacePath`. - -The same setup can be checked or installed from the command surface: +The setup can be checked or installed from the command surface: - `/codex computer-use status` - `/codex computer-use install` @@ -609,7 +589,9 @@ The same setup can be checked or installed from the command surface: Computer Use is macOS-specific and may require local OS permissions before the Codex MCP server can control apps. If `computerUse.enabled` is true and the MCP server is unavailable, Codex-mode turns fail before the thread starts instead of -silently running without the native Computer Use tools. +silently running without the native Computer Use tools. See +[Codex Computer Use](/plugins/codex-computer-use) for marketplace choices, +remote catalog limits, status reasons, and troubleshooting. ## Common recipes