Download PDF Download Markdown View on GitHub

Design write-up

The model is allowed to be uncertain exactly once. That uncertainty is compiled into a typed artifact that never needs it again, and every run after the first is deterministic.

Every claim below is backed by something runnable: an enforced import contract, a test, or a file in evidence/. What is not built says so in §7.


1. Architecture

An LLM drives a live legacy UI to a goal once. That run is compiled into a typed, versioned capability artifact. From then on the artifact is replayed with no model in the decision loop, and an AI agent invokes it like a typed function.

      Action  (from discovery, replay, or a human operator)
        │
  ┌─────▼──────┐
  │TargetResolv│  pure, read-only, over an already-captured snapshot
  └─────┬──────┘
  ┌─────▼──────┐
  │PolicyEngine│  the only thing that can mint an AuthorizedAction
  └─────┬──────┘
  ┌─────▼──────┐
  │SurfaceDrivr│  dispatch(AuthorizedAction) -- accepts nothing else
  └─────┬──────┘
  ┌─────▼──────┐
  │ EvidenceBus│  append-only, redacted, actor-tagged
  └────────────┘

Discovery, replay, and human intervention share one path, differing only in who originates the action, so a guardrail cannot be present on one and missing from another. Resolution runs before policy: risk classification's second signal is what the control says it does ("Transfer Funds"), which needs the resolved node, so authorizing first would mean authorizing half-blind. The invariant is not "policy runs first" but nothing reaches a surface without authorization — enforced three ways: an import contract, a type guard (dispatch() accepts only an AuthorizedAction), and an AST scan that has caught violations the import graph missed.

Stack. Pydantic v2 lets the artifact schema be the type system. Playwright over CDP, since the accessibility tree is the closest browser analogue to UIA/AX. The model sits behind an LlmPort protocol at temperature=0. YAML artifacts, so a bank reviewer reads one without running anything.

2. Artifact schema

cua.capability/v1 — Pydantic, YAML, immutable, content-addressed. Examples in evidence/capabilities/.

Targets are semantic, never selectors. A TargetDescriptor is role, accessible name, structural anchor, scope, ordinal. A CSS selector may appear only under hints — a cache accepted only if the node it finds also satisfies the semantic assertion. role: button, name: "Search" means the same in Windows UIA as in ARIA; input.btn1 does not.

Business outcomes, recovery, and failure are three distinct top-level keys — the brief names conflating the first with the third as the most common mistake here, so the schema makes it structurally impossible rather than trusting the executor.

The capability is immutable; everything a run produces lives elsewhere — RunRecord, CapabilityApproval, TenantBinding, keyed by id@version — since a definition that accumulates telemetry stops being reviewable, and "which version ran?" stops being answerable. capability.id is namespaced by vendor product, not tenant, and surface.driver_capabilities declares what a driver must be able to do rather than naming a library: both preconditions for reuse across institutions running the same product.

The compiler emits a draft and says so: one run saw only the happy path, so it declares no outcomes and no recovery, with review notes naming what a reviewer must add — inventing detectors a run never observed would produce error handling that is fiction.

← Start of the write-upDeterminism, errors & scale →