Verify fixes · v1

“Verify this fix on the new preview” authorizes one new execution of an owned baseline's saved plan. Call verify_run with its runId, the replacement targetOrigin (scheme, host and optional port only), and a new UUID idempotencyKey.

Constants preserves the entire supported saved plan, hash, persona and device. Version 2 plans replay scenarios and checkpoints, including prerequisites and secret-binding behavior. Version 3 plans repeat the saved tasks and questions without generating a new plan; the agent chooses browser actions as it goes. Task and question IDs remain stable. Constants generates fresh synthetic runtime identities. The baseline entry path and query are retained on the replacement origin. Selective tasks or scenarios are not supported.

Agent-issued top-level navigations to the original application origin are rebased. Application network requests, external authentication URLs and literal test inputs are not rewritten. Navigation back to the original deployment blocks affected coverage. Requests for restrictions that conflict with the saved plan must be resolved with the user; do not silently alter the baseline or launch an exploration.

While compilation is preparing, the request is retryable and reserves no allowance. Unavailable or unsupported plans return a clear error. Verification never silently substitutes a fresh exploration.

Immediately show the new run's reportUrl to the user as a clickable progress link before polling or waiting. The signed-in owner can follow progress there, and the completed report opens at the same URL. The test continues if the chat disconnects, so tell the user they can leave the chat and follow the link.

Poll the new run with get_run initially every fifteen seconds, backing off to thirty, then retrieve both reports with get_report:

  1. Confirm both reports refer to the same saved-plan hash.
  2. For version 2, match scenario and checkpoint IDs. For version 3 (format: "tasks-v1"), match taskId and answer questionId.
  3. Read outcomes, observations and relevant selected evidence together. For tasks, compare actions, answers and feedback; task completion is not a pass/fail verdict.
  4. Preserve blocked, not-tested, not-reached, unverified or missing outcomes as incomplete verification.
  5. Do not treat a disappeared finding as proof of a fix.
  6. Explain uncertainty when the exploration did not establish a comparable baseline outcome.

Constants records the verification execution and baseline link. It does not calculate or store improvement/regression classifications, or treat the coding agent's explanation as an authoritative comparison result.