An LLM works out how to accomplish a goal in an enterprise legacy application — once. That discovery is compiled into a typed, versioned capability artifact. From then on, it replays deterministically with no model in the loop.
Banks run a long tail of back-office applications with no API. When an AI agent needs to get real work done, the only way in is to drive the UI the way a human operator would. Doing that with an LLM on every invocation is slow, expensive, non-deterministic, and unauditable — and the last three are disqualifying in a regulated environment.
Each is enforced by an invariant test that fails the build if it stops holding.
Same artifact + same inputs + same app state = identical decision trace. No model in the loop, zero temperature, no network calls to LLMs.
Controls are identified by role, accessible name, and structural anchors — never brittle CSS selectors. A six-rung ladder survives markup churn.
Every action — from LLM discovery, replay engine, or human operator — passes through one authorization chokepoint. Zero bypasses, zero exceptions.
Unknown UI states stop the run immediately. Ambiguous targets are refused, never guessed. In banking, an unverified click is an incident.
An operator takes the live session, acts under policy, and hands it back. The engine re-anchors and resumes at the right step, not step 1.
A rebranded tenant label requires a four-line overlay document, not a re-record. Drift is quantified per tenant before replays fail.
Figures from committed runs in the repository, not estimates. Every one is reproducible from the commands below.
make verify-live re-checks any discovery run mechanically.One command walks the whole scenario end to end, on your machine.
| # | Stage | Observable Behavior | Result |
|---|---|---|---|
| 1 | Mock back-office running | Hostile frameset application, zero test IDs | READY |
| 2 | LLM Discovery | Agent drives live UI to goal, exploring paths | TRACE |
| 3 | Capability compiled | Typed I/O, semantic targets, content hash | SEALED |
| 4 | Deterministic replay | New member inputs, 0 model calls in the loop | SUCCESS |
| 5 | Business outcome | member_not_found — valid answer, not a crash | EXIT 0 |
| 6 | Injected 502 error | Declared recovery rule remediates and retries | SUCCESS |
| 7 | Fault past budget | Recovery exhausted → escalates to human queue | NEEDS_HUMAN |
| 8 | Malformed input | Rejected by schema guard before dispatch | FAILED |
| 9 | Undeclared screen | Fails closed immediately and requests audit | NEEDS_HUMAN |
| 10 | Human takeover | Operator drives same session with Lease lock | EPOCH 3 |
| 11 | Re-anchor resume | Resumes at step 4 without re-running past steps | SUCCESS |
| 12 | Agent invocation | Exposed to external agent as a typed tool call | SUCCESS |
| 13 | Approval gate | Integrity and review shown as separate axes; unattended replay gated on the second | SUCCESS |
| 14 | Generated test | A runnable Playwright test emitted from the artifact, carrying no CSS hint | SUCCESS |
Works completely offline without an API key — discovery falls back to a verified recorded transcript. Supply
OMNIROUTE_API_KEY
for a live LLM execution.
Discovery, replay and a human operator all reach the browser the same way. Nothing dispatches without passing policy.
The design write-up answers the brief directly — architecture, determinism, the error taxonomy, escalation, and what was deliberately left out. The documentation goes a level down: component designs, ADRs, runbooks and the test strategy.