Two type families, deliberately separate — what we observed and how the run ended:
ObservationClass |
meaning | leads to |
|---|---|---|
EXPECTED |
matches a precondition/checkpoint | proceed |
BUSINESS_OUTCOME |
matches a declared outcome | terminate, exit 0 |
RECOVERABLE |
matches a declared recovery rule | bounded remedy, then retry |
UNEXPECTED_STATE |
matches nothing declared | fail closed |
RunStatus is terminal and separate: SUCCESS | BUSINESS_OUTCOME | NEEDS_HUMAN | FAILED.
"No such member" exits 0 — a successful execution returning a negative answer, not a failure; collapsing the two turns a routine result into a 2am page and trains everyone to ignore the alert that also fires for real defects.
Fail-closed is the default. A state matching nothing declared stops the run rather than
proceeding on the assumption a click probably worked; two surviving candidates is
TARGET_AMBIGUOUS, and the resolver refuses rather than breaking the tie on a similarity score
— a tunable threshold is a knob that eventually gets turned. Determinism itself is enforced, not
asserted: cua.replay may not import cua.agent or any HTTP client, the resolution ladder scores
purely with ties broken by document order (never dict iteration, clock or random), and the
vision rung is hard-disabled in replay. Where a hard failure routes — pause for a human, or
terminate — is escalation.triggers' decision, not the executor's.
Drift is measured as descent. A target recorded at semantic_exact that now resolves only at
structural_anchor has moved, even though the run still succeeded, so run_record.drift_score
rises before replays begin to fail rather than after — make eval reports determinism as a boolean
per capability, since "94% deterministic" is not a property anyone can act on.
The seam is UiSnapshot — a normalized {node_id, role, name, value, states, bounds, scope} tree.
Drivers populate it from whatever their surface offers: a CDP accessibility tree today, Windows UIA
tomorrow. cua.targeting and cua.perception may not import Playwright or cua.surfaces
(enforced), so neither the artifact nor the resolver can grow web-shaped assumptions.
surfaces/desktop_uia/ is an interface-only stub documenting the UIA→UiSnapshot mapping.
Tenants are an overlay, never a fork: TenantBinding supplies vars and per-step overrides,
resolved at load into an effective capability whose content hash covers the overlay.
The no-CSS claim is measured, in
evidence/evals/cross_tenant.md. Variant B relabels controls
and restyles markup, so every hints.css in the artifact names a selector that does not exist
there — and the runs succeed anyway: 100%, zero wrong actions. Markup churn is carried by
semantic_exact; rebranding by structural_anchor plus the overlay — both load-bearing, which is
why the ladder has both. structural_anchor does not itself survive rebranding, since the anchor
text is the label cell and gets relabeled too, so the honest claim is markup change, not
vocabulary change; vocabulary is what the overlay is for. One limit that leaves: drift is computed
against the effective capability, so an overlay reports zero drift for a surface that genuinely
differs.