Find the failure. Build the environment. Grade the work. Sign it off.
Turn the places AI breaks into the places it learns.
We work across the whole arc of agent work: finding where a model breaks down in a real workflow, building the environment that reproduces it, grading the work against the records rather than the summary, and handing over the evidence before anything touches production.
The arc
Four stages, one arc. Start anywhere, or hand us the whole loop.
01 · Diagnose
Failure-mode studies and held-out task sets
We find where an agent breaks down in a specific workflow, then turn each failure into a task that can be scored.
- Tasks written by people who have done the work, not by annotators approximating it.
- Each task is graded by shared domain rules plus task-specific evidence, so a failing score points at a named rule.
- Delivered as a task set you can re-run against every checkpoint, model, and vendor.
02 · Simulate
Environments and scenario generators
The workflow rebuilt as a stateful system the agent can act inside, and a generator that seeds as many variants of it as a training run needs.
- API, UI and MCP surfaces over the same state (each environment lists its own), so one environment serves tool-calling and computer-use training.
- Resettable, with snapshots: a failed run can be replayed from the step before it went wrong.
- Describe a company in plain language and get a seeded business: open items, a half-finished period, a reconciliation that does not tie.
03 · Verify
Verifiers, invariants, and adversarial audits
Grading against the records themselves. We attack our own verifiers first and record which strategies got through.
- Invariants checked across the whole trajectory, not only the final state, because the right end state can be reached by the wrong actions.
- Expert rubrics only where the work genuinely calls for judgment, kept separate from the deterministic layers.
- Every verifier ships with the round it failed, the constraint we added, and the re-run results.
The layers we grade
L0 · Hard invariants
Identities that cannot be violated in any valid state. Purely programmatic.
L1 · Business correctness
Right period, right account, right counterparty, complete document chain.
L2 · Process compliance
Authority, self-approval, workflow bypass, forbidden side effects. Judged from the event log.
L3 · Judgment
Whether an estimate is reasonable and an assumption holds. Expert rubric, scored separately.
Evidence
Each action tied to the record that supports it. Missing evidence is a failure, not a deduction.
Determinism
The same trajectory scored twice returns the same result, or the verifier is the defect.
04 · Sign off
Evidence before production access
What an enterprise needs in order to sign off: the full record of what the agent did, and what it was not allowed to do.
- Every tool call, side effect, and state change captured, with forbidden effects counted rather than described.
- Private task sets covering the exceptions that matter to one firm, held out from anything published.
- A pre-deployment report a reviewer can read end to end, with full replay of any run in it.
Delivered as