diff --git a/CHANGELOG.md b/CHANGELOG.md index 6a9dadc9..7b06b962 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ All notable changes to SkillOpt are documented here. This project adheres to ## [Unreleased] ### Added +- **Cursor Agent research target harness** (`cursor_exec`) for running + supported benchmark rollouts through an installed, authenticated + `cursor-agent`, with sandboxed workspaces, structured trace capture, and + target-only optimizer separation. - **Handoff backend** (`--backend handoff`) for SkillOpt-Sleep — runs the sleep cycle with no model subprocess or API key: the engine writes each pending model call to `PROMPTS.md`/`pending.json` (exit code 3) and the diff --git a/README.md b/README.md index f4ccbac5..18263103 100644 --- a/README.md +++ b/README.md @@ -65,13 +65,13 @@ https://github.com/user-attachments/assets/eb12d3bc-371c-467f-904d-91b61f339ed7 A backend = a chat / exec target (e.g. `openai_chat`, `claude_chat`, `qwen_chat`, `minimax_chat`, `openai_compatible`, `codex_exec`, -`claude_code_exec`). If a provider implements the OpenAI Chat Completions +`claude_code_exec`, `cursor_exec`). If a provider implements the OpenAI Chat Completions protocol, try the built-in `openai_compatible` backend before adding code. See [`docs/guide/new-backend.md`](docs/guide/new-backend.md) for the full -contract; in short you add a `skillopt/model/_backend.py` module, -register it in `skillopt/model/common.py` + `backend_config.py`, and wire -it through the router in `skillopt/model/__init__.py`. `qwen_backend.py` -and `minimax_backend.py` are good templates. +contract. Chat backends add a `skillopt/model/_backend.py` module; +target-only exec backends use the shared harness in `codex_harness.py`. +Both register through `common.py`, `backend_config.py`, and +`skillopt/model/__init__.py`. ### Adding a new benchmark diff --git a/configs/_base_/default.yaml b/configs/_base_/default.yaml index fcfdd711..c8c068d1 100644 --- a/configs/_base_/default.yaml +++ b/configs/_base_/default.yaml @@ -24,6 +24,8 @@ model: claude_code_exec_use_sdk: auto claude_code_exec_effort: medium claude_code_exec_max_thinking_tokens: 16384 + cursor_exec_path: "" # blank uses CURSOR_EXEC_PATH or cursor-agent + cursor_exec_sandbox: "" # blank uses CURSOR_EXEC_SANDBOX or enabled codex_trace_to_optimizer: true azure_openai_endpoint: "" # e.g. "https://your-resource.openai.azure.com/" azure_openai_api_version: "2024-12-01-preview" diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index 8600c441..2a72a64a 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -45,6 +45,7 @@ model: | `minimax_chat` | ✓ | ✓ | MiniMax API | | `codex_exec` | — | ✓ | Codex CLI execution harness | | `claude_code_exec` | — | ✓ | Claude Code CLI execution harness | +| `cursor_exec` | — | ✓ | Cursor Agent CLI execution harness | The current MiniMax adapter has one shared deployment. Set `model.minimax_model` when MiniMax is the target; a mixed-backend run cannot @@ -183,6 +184,9 @@ Model credentials are loaded from environment variables: | `OPENAI_COMPATIBLE_MODEL` | `openai_compatible` | Shared provider model ID for direct library use; train/eval YAML role models take precedence | | `CLAUDE_CLI_BIN` | `claude_chat` | Optional path to the `claude` executable; defaults to `claude` | | `ANTHROPIC_API_KEY` | `claude_chat` | Optional authentication method understood by the Claude CLI, not a direct SkillOpt API client | +| `CURSOR_EXEC_PATH` | `cursor_exec` | Optional path to `cursor-agent`; defaults to `cursor-agent` | +| `CURSOR_EXEC_SANDBOX` | `cursor_exec` | Cursor sandbox mode: `enabled` (default) or `disabled` | +| `CURSOR_API_KEY` | `cursor_exec` | Optional authentication method understood directly by Cursor Agent | | `QWEN_CHAT_BASE_URL` | `qwen_chat` | Local Qwen/vLLM endpoint | | `QWEN_CHAT_MODEL` | `qwen_chat` | Served model name for direct library use; train/eval YAML role models take precedence | | `MINIMAX_BASE_URL` | `minimax_chat` | MiniMax-compatible base URL | @@ -197,6 +201,14 @@ and authenticate that CLI before use. Setting `ANTHROPIC_API_KEY` is one way the CLI may authenticate, but SkillOpt does not call the Anthropic API directly through this backend. +`cursor_exec` is a target-only benchmark harness. Install and authenticate +Cursor Agent first, then select it with `model.target_backend=cursor_exec`. +Read-only rollouts use Ask mode; artifact-producing rollouts add Cursor's +headless `--force` flag inside the benchmark workspace. SkillOpt enables the +Cursor sandbox by default and rejects file-edit rollouts if it is disabled; +read-only Ask-mode rollouts may explicitly disable it. SkillOpt does not approve +MCP servers automatically. + ### Three OpenAI-compatible paths - Research, generic provider: select `openai_compatible` and use diff --git a/docs/guide/new-backend.md b/docs/guide/new-backend.md index 67855274..98d598f8 100644 --- a/docs/guide/new-backend.md +++ b/docs/guide/new-backend.md @@ -146,9 +146,9 @@ Provider-specific configuration helpers and `count_tokens()` are optional, but their state must be safe to update while calls may run concurrently. Keep credentials out of logs and persisted artifacts. -Exec-style targets do not implement this chat contract. They are target-only -and are integrated through `codex_harness.py` plus environment-specific rollout -code. +Exec-style targets such as `claude_code_exec` and `cursor_exec` do not +implement this chat contract. They are target-only and are integrated through +`codex_harness.py` plus environment-specific rollout code. ## Step 2: register and route the backend diff --git a/docs/guideline.html b/docs/guideline.html index 3bd625db..2973dec7 100644 --- a/docs/guideline.html +++ b/docs/guideline.html @@ -396,6 +396,7 @@

Research model backends

minimax_chatYesYesMiniMax chat endpoint. codex_execYesSupported adapters onlyExecutes Codex for optimizer calls and as a target agent where supported. claude_code_execNoSupported adapters onlyExecutes Claude Code as a target agent. + cursor_execNoSupported adapters onlyExecutes Cursor Agent as a sandboxed target agent where supported. diff --git a/docs/reference/api.md b/docs/reference/api.md index 287a7175..34176d7c 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -192,6 +192,7 @@ not via a base class subclass. Supported values (as of this writing): | `openai_compatible` | ✓ | ✓ | | `codex_exec` | ✓ | ✓ | | `claude_code_exec` | — | ✓ | +| `cursor_exec` | — | ✓ | See `skillopt/model/backend_config.py` for the live whitelist and [`docs/reference/config.md`](./config.md) for the per-backend diff --git a/docs/reference/cli.md b/docs/reference/cli.md index e4d277aa..57269f48 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -2,9 +2,9 @@ > **Version note.** This reference tracks `main`. PyPI 0.2.0 does not yet > include the generic research `openai_compatible` backend, Sleep handoff, -> Sleep support for non-Azure OpenAI-compatible endpoints, or the Sleep -> `--preferences` flag; use a source install from `main` for those features -> until the next release. +> Sleep support for non-Azure OpenAI-compatible endpoints, the Sleep +> `--preferences` flag, or the research `cursor_exec` target harness; use a +> source install from `main` for those features until the next release. ## Training @@ -91,6 +91,27 @@ python scripts/train.py \ model.target=deepseek-chat ``` +To benchmark an installed, authenticated Cursor Agent through an environment +that supports exec targets: + +```bash +python scripts/eval_only.py \ + --config configs/searchqa/default.yaml \ + --skill skills/my_skill.md \ + --cfg-options \ + model.optimizer_backend=openai_chat \ + model.target_backend=cursor_exec \ + model.target=composer-2.5 +``` + +`cursor_exec` runs the target only; the optimizer remains separately +configured. Read-only rollouts use Cursor Ask mode. Rollouts that request file +edits use `--force` inside the benchmark workspace, with Cursor sandboxing +enabled. The harness refuses file-edit rollouts when the Cursor sandbox is +disabled. Read-only Ask-mode rollouts may explicitly disable it. Override the +executable or sandbox through `model.cursor_exec_path` and +`model.cursor_exec_sandbox`. + ## SkillOpt-Sleep ```bash diff --git a/docs/reference/config.md b/docs/reference/config.md index b65c204d..b8243c77 100644 --- a/docs/reference/config.md +++ b/docs/reference/config.md @@ -18,6 +18,7 @@ selecting the generic OpenAI-compatible backend. | `minimax_chat` | ✓ | ✓ | | `codex_exec` | ✓ | ✓ | | `claude_code_exec` | — | ✓ | +| `cursor_exec` | — | ✓ | MiniMax currently has one shared deployment. `model.minimax_model` is applied when MiniMax is the target; mixed-backend runs cannot independently choose a @@ -64,6 +65,8 @@ defaults to `claude` and can be overridden with `CLAUDE_CLI_BIN`. | `model.minimax_*` | MiniMax `base_url`, `api_key`, shared `minimax_model`, `temperature`, `max_tokens`, and `enable_thinking`; `minimax_model` applies when MiniMax is the target | | `model.codex_exec_*` | Codex path, sandbox, profile, SDK mode, reasoning, network/search, and approval policy | | `model.claude_code_exec_*` | Claude path, profile, SDK mode, effort, and thinking-token cap | +| `model.cursor_exec_path` | Cursor Agent executable path; default `cursor-agent` | +| `model.cursor_exec_sandbox` | Cursor sandbox mode: `enabled` (default) or `disabled`; file-edit rollouts require `enabled` | ## Training (`train`) diff --git a/scripts/eval_only.py b/scripts/eval_only.py index f3c5f86f..a40a4bd5 100644 --- a/scripts/eval_only.py +++ b/scripts/eval_only.py @@ -28,6 +28,7 @@ configure_azure_openai, configure_claude_code_exec, configure_codex_exec, + configure_cursor_exec, configure_qwen_chat, configure_minimax_chat, set_reasoning_effort, @@ -139,7 +140,7 @@ def parse_args() -> argparse.Namespace: # Legacy flat overrides p.add_argument("--env", type=str) p.add_argument("--backend", type=str, - choices=["azure_openai", "codex", "codex_exec", "claude", "claude_chat", "claude_code_exec", "minimax", "minimax_chat"]) + choices=["azure_openai", "codex", "codex_exec", "claude", "claude_chat", "claude_code_exec", "cursor", "cursor_exec", "minimax", "minimax_chat"]) p.add_argument("--optimizer_model", type=str) p.add_argument("--target_model", type=str) p.add_argument("--optimizer_backend", type=str) @@ -181,6 +182,8 @@ def parse_args() -> argparse.Namespace: p.add_argument("--claude_code_exec_use_sdk", type=str) p.add_argument("--claude_code_exec_effort", type=str) p.add_argument("--claude_code_exec_max_thinking_tokens", type=int) + p.add_argument("--cursor_exec_path", type=str) + p.add_argument("--cursor_exec_sandbox", type=str) p.add_argument("--minimax_base_url", type=str) p.add_argument("--minimax_api_key", type=str) p.add_argument("--minimax_model", type=str) @@ -262,6 +265,8 @@ def main() -> None: "claude_code_exec_use_sdk": "model.claude_code_exec_use_sdk", "claude_code_exec_effort": "model.claude_code_exec_effort", "claude_code_exec_max_thinking_tokens": "model.claude_code_exec_max_thinking_tokens", + "cursor_exec_path": "model.cursor_exec_path", + "cursor_exec_sandbox": "model.cursor_exec_sandbox", "minimax_base_url": "model.minimax_base_url", "minimax_api_key": "model.minimax_api_key", "minimax_model": "model.minimax_model", @@ -327,6 +332,11 @@ def _has_model_override(dotted_key: str, legacy_key: str) -> bool: elif backend == "claude_code_exec": cfg.setdefault("optimizer_backend", "openai_chat") cfg.setdefault("target_backend", "claude_code_exec") + elif backend == "cursor_exec": + if not _has_model_override("model.optimizer_backend", "optimizer_backend"): + cfg["optimizer_backend"] = "openai_chat" + if not _has_model_override("model.target_backend", "target_backend"): + cfg["target_backend"] = "cursor_exec" elif backend in {"minimax", "minimax_chat"}: cfg.setdefault("optimizer_backend", "openai_chat") cfg.setdefault("target_backend", "minimax_chat") @@ -355,6 +365,12 @@ def _has_model_override(dotted_key: str, legacy_key: str) -> bool: and not _has_model_override("model.target", "target_model") ): cfg["target_model"] = default_model_for_backend("claude_chat") + if cfg.get("target_backend") == "cursor_exec": + if ( + str(cfg.get("target_model", "") or "").strip() in _OPENAI_DEFAULT_MODEL_SENTINELS + and not _has_model_override("model.target", "target_model") + ): + cfg["target_model"] = default_model_for_backend("cursor_exec") if cfg.get("target_backend") == "minimax_chat": if ( str(cfg.get("target_model", "") or "").strip() in _OPENAI_DEFAULT_MODEL_SENTINELS @@ -429,6 +445,10 @@ def _has_model_override(dotted_key: str, legacy_key: str) -> bool: effort=cfg.get("claude_code_exec_effort", cfg.get("reasoning_effort", "medium")), max_thinking_tokens=cfg.get("claude_code_exec_max_thinking_tokens", 16384), ) + configure_cursor_exec( + path=cfg.get("cursor_exec_path") or None, + sandbox=cfg.get("cursor_exec_sandbox") or None, + ) configure_qwen_chat( base_url=cfg.get("qwen_chat_base_url") or None, api_key=cfg.get("qwen_chat_api_key") or None, diff --git a/scripts/train.py b/scripts/train.py index 918e684a..63cad9e1 100644 --- a/scripts/train.py +++ b/scripts/train.py @@ -137,7 +137,7 @@ def parse_args() -> argparse.Namespace: # Legacy flat CLI overrides (still work, prefer --cfg-options for new usage) p.add_argument("--env", type=str) p.add_argument("--backend", type=str, - choices=["azure_openai", "codex", "codex_exec", "claude", "claude_chat", "claude_code_exec", "qwen", "qwen_chat", "minimax", "minimax_chat"]) + choices=["azure_openai", "codex", "codex_exec", "claude", "claude_chat", "claude_code_exec", "cursor", "cursor_exec", "qwen", "qwen_chat", "minimax", "minimax_chat"]) p.add_argument("--optimizer_model", type=str) p.add_argument("--target_model", type=str) p.add_argument("--optimizer_backend", type=str) @@ -205,6 +205,8 @@ def parse_args() -> argparse.Namespace: p.add_argument("--claude_code_exec_use_sdk", type=str) p.add_argument("--claude_code_exec_effort", type=str) p.add_argument("--claude_code_exec_max_thinking_tokens", type=int) + p.add_argument("--cursor_exec_path", type=str) + p.add_argument("--cursor_exec_sandbox", type=str) p.add_argument("--codex_trace_to_optimizer", type=_BOOL) p.add_argument("--skill_init", type=str) p.add_argument("--num_epochs", type=int) @@ -343,6 +345,8 @@ def parse_args() -> argparse.Namespace: "claude_code_exec_use_sdk": "model.claude_code_exec_use_sdk", "claude_code_exec_effort": "model.claude_code_exec_effort", "claude_code_exec_max_thinking_tokens": "model.claude_code_exec_max_thinking_tokens", + "cursor_exec_path": "model.cursor_exec_path", + "cursor_exec_sandbox": "model.cursor_exec_sandbox", "codex_trace_to_optimizer": "model.codex_trace_to_optimizer", "num_epochs": "train.num_epochs", "train_size": "train.train_size", @@ -445,6 +449,11 @@ def _has_model_override(dotted_key: str, legacy_key: str) -> bool: elif backend == "claude_code_exec": flat.setdefault("optimizer_backend", "openai_chat") flat.setdefault("target_backend", "claude_code_exec") + elif backend == "cursor_exec": + if not _has_model_override("model.optimizer_backend", "optimizer_backend"): + flat["optimizer_backend"] = "openai_chat" + if not _has_model_override("model.target_backend", "target_backend"): + flat["target_backend"] = "cursor_exec" elif backend in {"qwen", "qwen_chat"}: flat.setdefault("optimizer_backend", "openai_chat") flat.setdefault("target_backend", "qwen_chat") @@ -482,6 +491,12 @@ def _has_model_override(dotted_key: str, legacy_key: str) -> bool: and not _has_model_override("model.target", "target_model") ): flat["target_model"] = default_model_for_backend("claude_chat") + if flat.get("target_backend") == "cursor_exec": + if ( + str(flat.get("target_model", "") or "").strip() in _OPENAI_DEFAULT_MODEL_SENTINELS + and not _has_model_override("model.target", "target_model") + ): + flat["target_model"] = default_model_for_backend("cursor_exec") if flat.get("target_backend") == "qwen_chat": if ( str(flat.get("target_model", "") or "").strip() in _OPENAI_DEFAULT_MODEL_SENTINELS diff --git a/skillopt/config.py b/skillopt/config.py index 2c1c4e93..c47e6d9b 100644 --- a/skillopt/config.py +++ b/skillopt/config.py @@ -51,6 +51,8 @@ "model.claude_code_exec_use_sdk": "claude_code_exec_use_sdk", "model.claude_code_exec_effort": "claude_code_exec_effort", "model.claude_code_exec_max_thinking_tokens": "claude_code_exec_max_thinking_tokens", + "model.cursor_exec_path": "cursor_exec_path", + "model.cursor_exec_sandbox": "cursor_exec_sandbox", "model.codex_trace_to_optimizer": "codex_trace_to_optimizer", "model.azure_endpoint": "azure_endpoint", "model.azure_api_version": "azure_api_version", diff --git a/skillopt/engine/trainer.py b/skillopt/engine/trainer.py index ff535b05..1eae4209 100644 --- a/skillopt/engine/trainer.py +++ b/skillopt/engine/trainer.py @@ -63,6 +63,7 @@ configure_azure_openai, configure_claude_code_exec, configure_codex_exec, + configure_cursor_exec, configure_minimax_chat, configure_qwen_chat, get_token_summary, @@ -681,6 +682,9 @@ def _build_eval_env(split: str, env_num: int, seed: int): elif backend == "claude_code_exec": optimizer_backend = optimizer_backend or "openai_chat" target_backend = target_backend or "claude_code_exec" + elif backend in {"cursor", "cursor_exec"}: + optimizer_backend = optimizer_backend or "openai_chat" + target_backend = target_backend or "cursor_exec" elif backend in {"qwen", "qwen_chat"}: optimizer_backend = optimizer_backend or "openai_chat" target_backend = target_backend or "qwen_chat" @@ -711,6 +715,10 @@ def _build_eval_env(split: str, env_num: int, seed: int): effort=cfg.get("claude_code_exec_effort", cfg.get("reasoning_effort", "medium")), max_thinking_tokens=cfg.get("claude_code_exec_max_thinking_tokens", 16384), ) + configure_cursor_exec( + path=cfg.get("cursor_exec_path") or None, + sandbox=cfg.get("cursor_exec_sandbox") or None, + ) configure_qwen_chat( base_url=cfg.get("qwen_chat_base_url") or None, api_key=cfg.get("qwen_chat_api_key") or None, diff --git a/skillopt/envs/_template/config_template.yaml b/skillopt/envs/_template/config_template.yaml index d98851dc..0778738b 100644 --- a/skillopt/envs/_template/config_template.yaml +++ b/skillopt/envs/_template/config_template.yaml @@ -51,5 +51,5 @@ evaluation: # Override only what differs from the inherited defaults. model: optimizer_backend: openai_chat # openai_chat | claude_chat | qwen_chat | minimax_chat | codex_exec - target_backend: openai_chat # chat backends plus codex_exec / claude_code_exec + target_backend: openai_chat # chat backends plus codex_exec / claude_code_exec / cursor_exec reasoning_effort: medium diff --git a/skillopt/model/__init__.py b/skillopt/model/__init__.py index f58be67b..6c84e6b9 100644 --- a/skillopt/model/__init__.py +++ b/skillopt/model/__init__.py @@ -13,8 +13,10 @@ from skillopt.model.backend_config import ( # noqa: F401 configure_claude_code_exec, configure_codex_exec, + configure_cursor_exec, get_claude_code_exec_config, get_codex_exec_config, + get_cursor_exec_config, get_optimizer_backend, get_target_backend, is_optimizer_chat_backend, @@ -53,6 +55,10 @@ def set_backend(name: str | None) -> str: set_optimizer_backend("openai_chat") set_target_backend(normalized) return normalized + if normalized in {"cursor", "cursor_agent", "cursor_exec"}: + set_optimizer_backend("openai_chat") + set_target_backend("cursor_exec") + return "cursor_exec" if normalized in {"qwen", "qwen_chat"}: set_optimizer_backend("openai_chat") set_target_backend("qwen_chat") @@ -84,6 +90,8 @@ def get_backend_name() -> str: return "qwen_chat" if optimizer == "openai_chat" and target == "minimax_chat": return "minimax_chat" + if optimizer == "openai_chat" and target == "cursor_exec": + return "cursor_exec" if optimizer == "openai_compatible" and target == "openai_compatible": return "openai_compatible" return f"{optimizer}+{target}" diff --git a/skillopt/model/backend_config.py b/skillopt/model/backend_config.py index d74bd135..d48b050b 100644 --- a/skillopt/model/backend_config.py +++ b/skillopt/model/backend_config.py @@ -28,6 +28,8 @@ def _parse_bool(value: str | None, default: bool) -> bool: CLAUDE_CODE_EXEC_PROFILE = os.environ.get("CLAUDE_CODE_EXEC_PROFILE", "") CLAUDE_CODE_EXEC_USE_SDK = os.environ.get("CLAUDE_CODE_EXEC_USE_SDK", "auto") CLAUDE_CODE_EXEC_EFFORT = os.environ.get("CLAUDE_CODE_EXEC_EFFORT", "medium") +CURSOR_EXEC_PATH = os.environ.get("CURSOR_EXEC_PATH", "cursor-agent") +CURSOR_EXEC_SANDBOX = os.environ.get("CURSOR_EXEC_SANDBOX", "enabled") def _parse_int(value: str | None, default: int) -> int: @@ -72,10 +74,11 @@ def get_optimizer_backend() -> str: def set_target_backend(backend: str) -> None: global TARGET_BACKEND TARGET_BACKEND = normalize_backend_name(backend or "openai_chat") - if TARGET_BACKEND not in {"openai_chat", "claude_chat", "qwen_chat", "minimax_chat", "openai_compatible", "codex_exec", "claude_code_exec"}: + if TARGET_BACKEND not in {"openai_chat", "claude_chat", "qwen_chat", "minimax_chat", "openai_compatible", "codex_exec", "claude_code_exec", "cursor_exec"}: raise ValueError( f"Unsupported target backend: {TARGET_BACKEND!r}. " - "Supported values are 'openai_chat', 'claude_chat', 'qwen_chat', 'minimax_chat', 'openai_compatible', 'codex_exec', and 'claude_code_exec'." + "Supported values are 'openai_chat', 'claude_chat', 'qwen_chat', 'minimax_chat', " + "'openai_compatible', 'codex_exec', 'claude_code_exec', and 'cursor_exec'." ) os.environ["TARGET_BACKEND"] = TARGET_BACKEND @@ -85,7 +88,7 @@ def get_target_backend() -> str: def is_target_exec_backend() -> bool: - return TARGET_BACKEND in {"codex_exec", "claude_code_exec"} + return TARGET_BACKEND in {"codex_exec", "claude_code_exec", "cursor_exec"} def is_optimizer_chat_backend() -> bool: @@ -198,3 +201,30 @@ def get_claude_code_exec_config() -> dict[str, str | int]: "max_thinking_tokens": CLAUDE_CODE_EXEC_MAX_THINKING_TOKENS, "empty_response_retries": EXEC_EMPTY_RESPONSE_RETRIES, } + + +def configure_cursor_exec( + *, + path: str | None = None, + sandbox: str | None = None, +) -> None: + global CURSOR_EXEC_PATH, CURSOR_EXEC_SANDBOX + if path is not None: + CURSOR_EXEC_PATH = str(path).strip() or "cursor-agent" + os.environ["CURSOR_EXEC_PATH"] = CURSOR_EXEC_PATH + if sandbox is not None: + normalized_sandbox = str(sandbox).strip().lower() or "enabled" + if normalized_sandbox not in {"enabled", "disabled"}: + raise ValueError("cursor_exec sandbox must be 'enabled' or 'disabled'") + CURSOR_EXEC_SANDBOX = normalized_sandbox + os.environ["CURSOR_EXEC_SANDBOX"] = CURSOR_EXEC_SANDBOX + + +def get_cursor_exec_config() -> dict[str, str | int]: + if CURSOR_EXEC_SANDBOX not in {"enabled", "disabled"}: + raise ValueError("cursor_exec sandbox must be 'enabled' or 'disabled'") + return { + "path": CURSOR_EXEC_PATH, + "sandbox": CURSOR_EXEC_SANDBOX, + "empty_response_retries": EXEC_EMPTY_RESPONSE_RETRIES, + } diff --git a/skillopt/model/codex_harness.py b/skillopt/model/codex_harness.py index a8513ea1..6bbb0939 100644 --- a/skillopt/model/codex_harness.py +++ b/skillopt/model/codex_harness.py @@ -14,10 +14,10 @@ from skillopt.model.backend_config import ( get_claude_code_exec_config, get_codex_exec_config, + get_cursor_exec_config, get_target_backend, ) - ANSWER_SCHEMA: dict[str, Any] = { "type": "object", "properties": { @@ -227,6 +227,51 @@ def _build_claude_trace_summary(raw: str, response: str) -> str: return "\n".join(parts) +def _build_cursor_trace_summary(raw: str, response: str) -> str: + model = "" + permission_mode = "" + session_id = "" + duration_ms = 0 + tool_calls = 0 + terminal_error = False + for line in (raw or "").splitlines(): + line = line.strip() + if not line.startswith("{"): + continue + try: + event = json.loads(line) + except json.JSONDecodeError: + continue + if not isinstance(event, dict): + continue + if event.get("type") == "system" and event.get("subtype") == "init": + model = str(event.get("model") or model) + permission_mode = str(event.get("permissionMode") or permission_mode) + session_id = str(event.get("session_id") or session_id) + elif event.get("type") == "tool_call" and event.get("subtype") == "started": + tool_calls += 1 + elif event.get("type") == "result": + session_id = str(event.get("session_id") or session_id) + try: + duration_ms = int(event.get("duration_ms") or 0) + except (TypeError, ValueError): + duration_ms = 0 + terminal_error = bool(event.get("is_error")) or event.get("subtype") == "error" + + parts = ["Cursor Agent Trace Summary"] + if model: + parts.append(f"- model: {model}") + if permission_mode: + parts.append(f"- permission mode: {permission_mode}") + if session_id: + parts.append(f"- session id: {session_id}") + parts.append(f"- tool calls: {tool_calls}") + parts.append(f"- duration ms: {duration_ms}") + parts.append(f"- terminal error: {'yes' if terminal_error else 'no'}") + parts.append(f"- final response chars: {len(response or '')}") + return "\n".join(parts) + + def _persist_artifacts( *, work_dir: str, @@ -271,6 +316,16 @@ def _persist_claude_artifacts(work_dir: str, raw: str, response: str) -> None: ) +def _persist_cursor_artifacts(work_dir: str, raw: str, response: str) -> None: + _persist_artifacts( + work_dir=work_dir, + raw=_sanitize_cursor_trace(raw, preserve_markers=True), + response=response, + prefix="cursor", + summary_builder=_build_cursor_trace_summary, + ) + + def parse_codex_raw(raw: str) -> dict: """Parse raw Codex CLI output into step sections. @@ -1016,6 +1071,239 @@ def run_codex_exec( return last_response, combined +_CURSOR_SECRET_ASSIGNMENT = re.compile( + r"(?i)\b(cursor_api_key|api[_ -]?key|authorization|bearer|" + r"access[_ -]?token|refresh[_ -]?token|token|password)\b" + r"(\s*[:=]\s*|\s+)(?:bearer\s+)?([^\s,;]+)" +) +_CURSOR_SECRET_TOKEN = re.compile(r"\b(?:sk|key)[_-][A-Za-z0-9_-]{8,}\b") +_CURSOR_OMITTED_TRACE_FIELDS = {"args", "content", "filetext", "prompt", "result"} +_CURSOR_SECRET_TRACE_FIELDS = { + "accesstoken", + "apikey", + "authorization", + "cursorapikey", + "password", + "refreshtoken", + "secret", + "token", +} + + +def _redact_cursor_error(value: str) -> str: + text = _CURSOR_SECRET_ASSIGNMENT.sub(r"\1\2[REDACTED]", value or "") + return _CURSOR_SECRET_TOKEN.sub("[REDACTED]", text) + + +def _sanitize_cursor_json(value: Any, *, field: str = "") -> Any: + normalized_field = re.sub(r"[^a-z0-9]", "", field.lower()) + if normalized_field in _CURSOR_OMITTED_TRACE_FIELDS: + return "[OMITTED]" + if ( + normalized_field in _CURSOR_SECRET_TRACE_FIELDS + or normalized_field.endswith("apikey") + ): + return "[REDACTED]" + if isinstance(value, dict): + return { + str(key): _sanitize_cursor_json(item, field=str(key)) + for key, item in value.items() + } + if isinstance(value, list): + return [_sanitize_cursor_json(item) for item in value] + if isinstance(value, str): + return _redact_cursor_error(value) + return value + + +def _cursor_process_text(value: str | bytes) -> str: + if isinstance(value, bytes): + return value.decode("utf-8", errors="replace") + return str(value or "") + + +def _sanitize_cursor_trace( + raw: str | bytes, + *, + preserve_markers: bool = False, +) -> str: + text = _cursor_process_text(raw) + sanitized: list[str] = [] + in_stderr = False + omitted_stdout = False + for line in text.splitlines(): + stripped = line.strip() + if preserve_markers and stripped.startswith("===== CURSOR CLI ATTEMPT "): + in_stderr = False + omitted_stdout = False + sanitized.append(line) + continue + if preserve_markers and stripped == "[stderr]": + in_stderr = True + sanitized.append(line) + continue + if in_stderr: + sanitized.append(_redact_cursor_error(line)) + continue + if stripped.startswith("{"): + try: + event = json.loads(stripped) + except json.JSONDecodeError: + pass + else: + sanitized.append( + json.dumps( + _sanitize_cursor_json(event), + ensure_ascii=False, + separators=(",", ":"), + ) + ) + continue + if not stripped: + sanitized.append("") + elif not omitted_stdout: + sanitized.append("[OMITTED NON-JSON OUTPUT]") + omitted_stdout = True + return "\n".join(sanitized) + + +def _parse_cursor_terminal(raw: str) -> tuple[str, str]: + terminal: dict[str, Any] | None = None + for line in (raw or "").splitlines(): + line = line.strip() + if not line.startswith("{"): + continue + try: + event = json.loads(line) + except json.JSONDecodeError: + continue + if isinstance(event, dict) and event.get("type") == "result": + terminal = event + + if terminal is None: + return "", "Cursor Agent did not emit a terminal result" + if terminal.get("is_error") is True or terminal.get("subtype") == "error": + detail = str(terminal.get("result") or terminal.get("error") or "unknown error") + return "", f"Cursor Agent returned an error result: {_redact_cursor_error(detail)}" + result = terminal.get("result") + if not isinstance(result, str) or not result.strip(): + return "", "Cursor Agent returned an empty terminal result" + return result.strip(), "" + + +def run_cursor_exec( + *, + work_dir: str, + prompt: str, + model: str, + timeout: int, + images: list[str] | None = None, + data_dirs: list[str] | None = None, + sandbox: str | None = None, + allow_file_edits: bool = False, +) -> tuple[str, str]: + """Run Cursor Agent headlessly as a benchmark target.""" + config = get_cursor_exec_config() + retries = int(config.get("empty_response_retries", 0) or 0) + add_dirs = _validated_add_dirs(work_dir, data_dirs, images)[1:] + all_raw: list[str] = [] + last_error = "Cursor Agent returned no response" + actual_sandbox = str(sandbox or config["sandbox"]) + if actual_sandbox not in {"enabled", "disabled"}: + raise ValueError("Cursor Agent sandbox must be 'enabled' or 'disabled'") + if allow_file_edits and actual_sandbox == "disabled": + raise ValueError( + "Cursor Agent file-edit rollouts require sandbox='enabled'; " + "refusing to combine --force with a disabled sandbox" + ) + + for attempt in range(retries + 1): + attempt_prompt = _exec_prompt( + _retry_prompt(prompt, attempt), + allow_file_edits=allow_file_edits, + ) + cmd = [ + str(config["path"]), + "-p", + "--output-format", + "stream-json", + "--trust", + "--workspace", + work_dir, + "--sandbox", + actual_sandbox, + ] + if allow_file_edits: + cmd.append("--force") + else: + cmd.extend(["--mode", "ask"]) + if model: + cmd.extend(["--model", model]) + for path in add_dirs: + cmd.extend(["--add-dir", path]) + + try: + proc = subprocess.run( + cmd, + cwd=work_dir, + capture_output=True, + text=True, + timeout=timeout, + input=attempt_prompt, + ) + except subprocess.TimeoutExpired as exc: + stdout = exc.stdout or "" + stderr = exc.stderr or "" + raw = stdout + safe_raw = _sanitize_cursor_trace(raw) + if stderr: + safe_stderr = _redact_cursor_error(_cursor_process_text(stderr)) + safe_raw = ( + f"{safe_raw}\n[stderr]\n{safe_stderr}" + if safe_raw + else f"[stderr]\n{safe_stderr}" + ) + all_raw.append(f"===== CURSOR CLI ATTEMPT {attempt + 1} =====\n{safe_raw}") + _persist_cursor_artifacts(work_dir, "\n\n".join(all_raw), "") + raise + except OSError as exc: + detail = _redact_cursor_error(str(exc)) + raise RuntimeError(f"Cursor Agent could not be executed: {detail}") from exc + + stdout = proc.stdout or "" + stderr = proc.stderr or "" + raw = stdout + safe_raw = _sanitize_cursor_trace(raw) + if stderr: + safe_stderr = _redact_cursor_error(_cursor_process_text(stderr)) + safe_raw = ( + f"{safe_raw}\n[stderr]\n{safe_stderr}" + if safe_raw + else f"[stderr]\n{safe_stderr}" + ) + all_raw.append(f"===== CURSOR CLI ATTEMPT {attempt + 1} =====\n{safe_raw}") + combined = "\n\n".join(all_raw) + + if proc.returncode != 0: + _persist_cursor_artifacts(work_dir, combined, "") + detail = _redact_cursor_error((stderr or stdout).strip())[:4000] + raise RuntimeError( + f"Cursor Agent failed with exit code {proc.returncode}: {detail}" + ) + + response, last_error = _parse_cursor_terminal(stdout) + if response: + _persist_cursor_artifacts(work_dir, combined, response) + return response, combined + if last_error.startswith("Cursor Agent returned an error result"): + _persist_cursor_artifacts(work_dir, combined, "") + raise RuntimeError(last_error) + + combined = "\n\n".join(all_raw) + _persist_cursor_artifacts(work_dir, combined, "") + raise RuntimeError(last_error) + + def run_target_exec( *, work_dir: str, @@ -1054,4 +1342,15 @@ def run_target_exec( permission_mode=permission_mode, allow_file_edits=allow_file_edits, ) + if backend == "cursor_exec": + return run_cursor_exec( + work_dir=work_dir, + prompt=prompt, + model=model, + timeout=timeout, + images=images, + data_dirs=data_dirs, + sandbox=sandbox, + allow_file_edits=allow_file_edits, + ) raise ValueError(f"Unsupported exec backend: {backend}") diff --git a/skillopt/model/common.py b/skillopt/model/common.py index ee916073..97e4be81 100644 --- a/skillopt/model/common.py +++ b/skillopt/model/common.py @@ -24,6 +24,7 @@ "claude": "claude-sonnet-4-6", "claude_chat": "claude-sonnet-4-6", "claude_code_exec": "claude-sonnet-4-6", + "cursor_exec": "composer-2.5", "qwen_chat": "Qwen/Qwen3.5-4B", "minimax_chat": "MiniMax-M2.7", "openai_compatible": "gpt-4o-mini", @@ -40,6 +41,9 @@ "claude": "claude_chat", "claude_chat": "claude_chat", "claude_code_exec": "claude_code_exec", + "cursor": "cursor_exec", + "cursor_agent": "cursor_exec", + "cursor_exec": "cursor_exec", "anthropic": "claude_chat", "qwen": "qwen_chat", "qwen_chat": "qwen_chat", diff --git a/tests/test_cursor_exec_backend.py b/tests/test_cursor_exec_backend.py new file mode 100644 index 00000000..3fea88fb --- /dev/null +++ b/tests/test_cursor_exec_backend.py @@ -0,0 +1,438 @@ +from __future__ import annotations + +import json +import os +import subprocess +import sys +from collections.abc import Iterator +from pathlib import Path +from types import SimpleNamespace +from typing import Any + +import pytest + +import skillopt.model as model +from skillopt.config import flatten_config +from skillopt.model import backend_config +from skillopt.model import codex_harness as harness +from skillopt.model.common import default_model_for_backend, normalize_backend_name + + +@pytest.fixture(autouse=True) +def restore_backend_state() -> Iterator[None]: + optimizer_backend = backend_config.get_optimizer_backend() + target_backend = backend_config.get_target_backend() + cursor_path = backend_config.CURSOR_EXEC_PATH + cursor_sandbox = backend_config.CURSOR_EXEC_SANDBOX + retries = backend_config.EXEC_EMPTY_RESPONSE_RETRIES + env = { + key: os.environ.get(key) + for key in ( + "OPTIMIZER_BACKEND", + "TARGET_BACKEND", + "CURSOR_EXEC_PATH", + "CURSOR_EXEC_SANDBOX", + ) + } + yield + backend_config.OPTIMIZER_BACKEND = optimizer_backend + backend_config.TARGET_BACKEND = target_backend + backend_config.CURSOR_EXEC_PATH = cursor_path + backend_config.CURSOR_EXEC_SANDBOX = cursor_sandbox + backend_config.EXEC_EMPTY_RESPONSE_RETRIES = retries + for key, value in env.items(): + if value is None: + os.environ.pop(key, None) + else: + os.environ[key] = value + + +def _result(text: str = "A") -> str: + return ( + '{"type":"system","subtype":"init","model":"composer-2.5",' + '"permissionMode":"default","session_id":"session-1"}\n' + '{"type":"tool_call","subtype":"started","call_id":"call-1",' + '"tool_call":{"readToolCall":{"args":{"path":"task.md"}}}}\n' + f'{{"type":"result","subtype":"success","is_error":false,' + f'"duration_ms":12,"result":"{text}","session_id":"session-1"}}\n' + ) + + +def _workspace(tmp_path: Path) -> Path: + work_dir = tmp_path / "predictions" / "task-1" / "cursor_exec" + work_dir.mkdir(parents=True) + return work_dir + + +def test_cursor_exec_is_target_only() -> None: + backend_config.set_target_backend("cursor") + + assert backend_config.get_target_backend() == "cursor_exec" + assert backend_config.is_target_exec_backend() + with pytest.raises(ValueError, match="Unsupported optimizer backend"): + backend_config.set_optimizer_backend("cursor_exec") + with pytest.raises(NotImplementedError, match="Exec backends"): + model.chat_target("system", "user") + + +def test_cursor_alias_and_default_model() -> None: + assert normalize_backend_name("cursor") == "cursor_exec" + assert normalize_backend_name("cursor_agent") == "cursor_exec" + assert default_model_for_backend("cursor_exec") == "composer-2.5" + + assert model.set_backend("cursor") == "cursor_exec" + assert backend_config.get_optimizer_backend() == "openai_chat" + assert backend_config.get_target_backend() == "cursor_exec" + assert model.get_backend_name() == "cursor_exec" + + +def test_cursor_config_flattens_and_validates() -> None: + flat = flatten_config( + { + "model": { + "cursor_exec_path": "/opt/cursor-agent", + "cursor_exec_sandbox": "disabled", + } + } + ) + assert flat["cursor_exec_path"] == "/opt/cursor-agent" + assert flat["cursor_exec_sandbox"] == "disabled" + + backend_config.configure_cursor_exec(path="cursor-test", sandbox="disabled") + assert backend_config.get_cursor_exec_config()["path"] == "cursor-test" + assert backend_config.get_cursor_exec_config()["sandbox"] == "disabled" + with pytest.raises(ValueError, match="sandbox must be"): + backend_config.configure_cursor_exec(sandbox="invalid") + + +def test_train_cursor_shorthand_configures_target_only( + monkeypatch: pytest.MonkeyPatch, +) -> None: + from scripts import train + + config_path = Path(__file__).parents[1] / "configs" / "_base_" / "default.yaml" + monkeypatch.setattr( + sys, + "argv", + ["train.py", "--config", str(config_path), "--backend", "cursor"], + ) + + cfg = train.load_config(train.parse_args()) + + assert cfg["optimizer_backend"] == "openai_chat" + assert cfg["target_backend"] == "cursor_exec" + assert cfg["target_model"] == "composer-2.5" + + +def test_read_only_cursor_exec_uses_stdin_and_preserves_trace( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + work_dir = _workspace(tmp_path) + data_dir = tmp_path / "corpus" + data_dir.mkdir() + calls: list[tuple[list[str], dict[str, Any]]] = [] + + def fake_run(cmd: list[str], **kwargs: Any) -> SimpleNamespace: + calls.append((cmd, kwargs)) + return SimpleNamespace(returncode=0, stdout="not json\n" + _result(), stderr="") + + backend_config.configure_cursor_exec(path="cursor-test", sandbox="enabled") + monkeypatch.setattr(harness.subprocess, "run", fake_run) + + response, raw = harness.run_cursor_exec( + work_dir=str(work_dir), + prompt="Answer the benchmark task.", + model="composer-2.5", + timeout=17, + data_dirs=[str(data_dir)], + ) + + assert response == "A" + assert "tool_call" in raw + assert "not json" not in raw + assert "task.md" not in raw + assert "A" not in raw + cmd, kwargs = calls[0] + assert cmd[:4] == ["cursor-test", "-p", "--output-format", "stream-json"] + assert ["--mode", "ask"] == cmd[cmd.index("--mode"):cmd.index("--mode") + 2] + assert "--force" not in cmd + assert cmd[cmd.index("--workspace") + 1] == str(work_dir) + assert cmd[cmd.index("--sandbox") + 1] == "enabled" + assert cmd[cmd.index("--model") + 1] == "composer-2.5" + assert cmd[cmd.index("--add-dir") + 1] == str(data_dir) + assert kwargs["cwd"] == str(work_dir) + assert kwargs["timeout"] == 17 + assert ".agents/skills/skillopt-target/SKILL.md" in kwargs["input"] + assert "Do not modify files" in kwargs["input"] + persisted_raw = (work_dir.parent / "cursor_raw.txt").read_text() + assert "not json" not in persisted_raw + assert "task.md" not in persisted_raw + assert "A" not in persisted_raw + summary = (work_dir.parent / "cursor_trace_summary.txt").read_text() + assert "tool calls: 1" in summary + assert "session-1" in summary + + +def test_cursor_exec_force_is_limited_to_file_edit_rollouts( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + work_dir = _workspace(tmp_path) + calls: list[tuple[list[str], str]] = [] + + def fake_run(cmd: list[str], **kwargs: Any) -> SimpleNamespace: + calls.append((cmd, kwargs["input"])) + return SimpleNamespace(returncode=0, stdout=_result(), stderr="") + + monkeypatch.setattr(harness.subprocess, "run", fake_run) + + harness.run_cursor_exec( + work_dir=str(work_dir), + prompt="Write solution.py.", + model="", + timeout=10, + allow_file_edits=True, + ) + + cmd, prompt = calls[0] + assert "--force" in cmd + assert "--mode" not in cmd + assert "You may modify files" in prompt + + +def test_cursor_exec_rejects_file_edits_with_disabled_sandbox( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + work_dir = _workspace(tmp_path) + calls = 0 + + def fake_run(_cmd: list[str], **_kwargs: Any) -> SimpleNamespace: + nonlocal calls + calls += 1 + return SimpleNamespace(returncode=0, stdout=_result(), stderr="") + + backend_config.configure_cursor_exec(sandbox="disabled") + monkeypatch.setattr(harness.subprocess, "run", fake_run) + + with pytest.raises(ValueError, match="refusing to combine --force"): + harness.run_cursor_exec( + work_dir=str(work_dir), + prompt="Write solution.py.", + model="composer-2.5", + timeout=10, + allow_file_edits=True, + ) + + assert calls == 0 + + +def test_cursor_exec_retries_zero_exit_malformed_output( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + work_dir = _workspace(tmp_path) + prompts: list[str] = [] + + def fake_run(_cmd: list[str], **kwargs: Any) -> SimpleNamespace: + prompts.append(kwargs["input"]) + stdout = "malformed output" if len(prompts) == 1 else _result() + return SimpleNamespace(returncode=0, stdout=stdout, stderr="") + + backend_config.EXEC_EMPTY_RESPONSE_RETRIES = 1 + monkeypatch.setattr(harness.subprocess, "run", fake_run) + + response, _raw = harness.run_cursor_exec( + work_dir=str(work_dir), + prompt="Answer.", + model="", + timeout=10, + ) + + assert response == "A" + assert len(prompts) == 2 + assert "Previous execution returned an empty final response" in prompts[1] + + +def test_cursor_trace_summary_tolerates_malformed_metadata() -> None: + raw = ( + '{"type":"result","subtype":"success","is_error":false,' + '"duration_ms":"unknown","result":"done"}\n' + ) + + summary = harness._build_cursor_trace_summary(raw, "done") + + assert "duration ms: 0" in summary + + +def test_cursor_trace_omits_message_and_tool_payloads() -> None: + raw = "\n".join( + [ + '{"type":"user","message":{"role":"user","content":' + '[{"type":"text","text":"private prompt"}]}}', + '{"type":"assistant","message":{"role":"assistant","content":' + '[{"type":"text","text":"private response"}]}}', + '{"type":"tool_call","subtype":"completed","tool_call":' + '{"readToolCall":{"args":{"path":"secret.txt"},"result":' + '{"success":{"content":"private file contents"}}}}}', + '{"type":"result","subtype":"success","is_error":false,' + '"duration_ms":1,"result":"private final answer"}', + ] + ) + + sanitized = harness._sanitize_cursor_trace(raw) + events = [json.loads(line) for line in sanitized.splitlines()] + + assert events[0]["message"]["content"] == "[OMITTED]" + assert events[1]["message"]["content"] == "[OMITTED]" + assert events[2]["tool_call"]["readToolCall"]["args"] == "[OMITTED]" + assert events[2]["tool_call"]["readToolCall"]["result"] == "[OMITTED]" + assert events[3]["result"] == "[OMITTED]" + assert "private" not in sanitized + assert "secret.txt" not in sanitized + + +def test_cursor_exec_does_not_retry_error_result_and_redacts_detail( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + work_dir = _workspace(tmp_path) + calls = 0 + + def fake_run(_cmd: list[str], **_kwargs: Any) -> SimpleNamespace: + nonlocal calls + calls += 1 + stdout = ( + '{"type":"result","subtype":"error","is_error":true,' + '"result":"API key: cursor-secret-value"}\n' + ) + return SimpleNamespace(returncode=0, stdout=stdout, stderr="") + + backend_config.EXEC_EMPTY_RESPONSE_RETRIES = 1 + monkeypatch.setattr(harness.subprocess, "run", fake_run) + + with pytest.raises(RuntimeError) as exc_info: + harness.run_cursor_exec( + work_dir=str(work_dir), + prompt="Answer.", + model="", + timeout=10, + ) + + assert calls == 1 + assert "[REDACTED]" in str(exc_info.value) + assert "cursor-secret-value" not in str(exc_info.value) + persisted_raw = (work_dir.parent / "cursor_raw.txt").read_text() + assert "cursor-secret-value" not in persisted_raw + assert '"result":"[OMITTED]"' in persisted_raw + + +def test_cursor_exec_nonzero_exit_is_not_retried( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + work_dir = _workspace(tmp_path) + calls = 0 + + def fake_run(_cmd: list[str], **_kwargs: Any) -> SimpleNamespace: + nonlocal calls + calls += 1 + return SimpleNamespace( + returncode=1, + stdout="", + stderr="CURSOR_API_KEY=cursor-secret-token authentication failed", + ) + + backend_config.EXEC_EMPTY_RESPONSE_RETRIES = 1 + monkeypatch.setattr(harness.subprocess, "run", fake_run) + + with pytest.raises(RuntimeError) as exc_info: + harness.run_cursor_exec( + work_dir=str(work_dir), + prompt="Answer.", + model="", + timeout=10, + ) + + assert calls == 1 + assert "[REDACTED]" in str(exc_info.value) + assert "cursor-secret-token" not in str(exc_info.value) + persisted_raw = (work_dir.parent / "cursor_raw.txt").read_text() + assert "cursor-secret-token" not in persisted_raw + assert "CURSOR_API_KEY=[REDACTED]" in persisted_raw + + +def test_cursor_exec_timeout_is_persisted( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + work_dir = _workspace(tmp_path) + + def fake_run(_cmd: list[str], **_kwargs: Any) -> SimpleNamespace: + raise subprocess.TimeoutExpired( + "cursor-test", + 3, + output=b"partial CURSOR_API_KEY=timeout-secret", + ) + + monkeypatch.setattr(harness.subprocess, "run", fake_run) + + with pytest.raises(subprocess.TimeoutExpired): + harness.run_cursor_exec( + work_dir=str(work_dir), + prompt="Answer.", + model="", + timeout=3, + ) + + persisted_raw = (work_dir.parent / "cursor_raw.txt").read_text() + assert "partial" not in persisted_raw + assert "[OMITTED NON-JSON OUTPUT]" in persisted_raw + assert "timeout-secret" not in persisted_raw + + +def test_cursor_exec_spawn_failure_is_actionable( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + work_dir = _workspace(tmp_path) + + def fake_run(_cmd: list[str], **_kwargs: Any) -> SimpleNamespace: + raise FileNotFoundError("cursor-agent-test was not found") + + monkeypatch.setattr(harness.subprocess, "run", fake_run) + + with pytest.raises(RuntimeError, match="could not be executed"): + harness.run_cursor_exec( + work_dir=str(work_dir), + prompt="Answer.", + model="", + timeout=3, + ) + + +def test_run_target_exec_dispatches_cursor( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + captured: dict[str, Any] = {} + + def fake_cursor(**kwargs: Any) -> tuple[str, str]: + captured.update(kwargs) + return "cursor response", "cursor trace" + + backend_config.set_target_backend("cursor_exec") + monkeypatch.setattr(harness, "run_cursor_exec", fake_cursor) + + response, raw = harness.run_target_exec( + work_dir=str(tmp_path), + prompt="task", + model="composer-2.5", + timeout=20, + allow_file_edits=True, + ) + + assert (response, raw) == ("cursor response", "cursor trace") + assert captured["allow_file_edits"] is True + assert captured["model"] == "composer-2.5"