Outcomes

Give an agent a definition of done and a rubric, and let the platform grade its work and drive bounded revision.

Most turns end when the agent stops talking. An Outcome ends a turn when the work meets a bar you set. You post a description, a rubric, and an iteration budget; the platform runs the agent, grades the result against your rubric in a separate context, and either accepts it or feeds the critique back for another attempt, up to your limit.

Defining one

An Outcome is a drive event, user.define_outcome, accepted from a pending or idle Session:

{
  "type": "user.define_outcome",
  "payload": {
    "description": "Write a release note for the merged change.",
    "rubric": "- explains user impact\n- cites verification evidence\n- under 200 words",
    "max_iterations": 3,
    "deliverables": ["/mnt/session/outputs/release-note.md"],
    "report": {
      "extraction": "model",
      "outputs": [
        {
          "key": "ready_to_publish",
          "title": "Ready to publish",
          "type": "boolean",
          "acceptance": { "kind": "equals", "value": true }
        }
      ]
    },
    "authored_by": "prin_0123456789abcdef0123456789abcdef",
    "caused_by": { "kind": "api" }
  }
}

Field	Rule
description	Required. 1–4,096 characters.
rubric	Required. 1–65,536 characters of inline markdown; the grader reads it as freeform text.
max_iterations	Optional integer, 1–20. Defaults to 3.
deliverables	Optional. 1–32 absolute paths, each captured at evaluation time.
report	Optional. 1–32 typed outputs extracted and validated with each evaluation.

The whole evaluate-and-revise loop is a single turn, and at most one Outcome is active per Session at a time. A new one is accepted only after the previous reaches a terminal result. After that, the Session continues conversationally with its full history intact.

Deliverables

If you name deliverables, each path is read from the sandbox when the Run settles and graded alongside the conversation. Paths must live inside a mounted tree or under /mnt/session/outputs. Each file is capped at 256 KiB and the set at 1 MiB total.

Capture is all-or-nothing: if any deliverable is missing, too large, outside the allowed roots, or unreadable, the entire evaluation fails with a typed verdict naming the failure: never a partial grade, never a fabricated one, and no model call is spent.

A zero-data-retention Workspace cannot durably capture files or typed Report evidence, so user.define_outcome carrying deliverables or report is rejected at define time there. ZDR evaluations grade live conversational output only.

Grading happens in a separate context

When the Run settles its work, the Session parks in waiting and appends outcome.evaluation_started. A grader then judges the result. The grader is never a harness, and it never sees the agent’s reasoning (only its output), so it cannot be talked into a pass by the same context that produced the work.

Two grader tiers ship:

• Marker: a deterministic local grader, no model call, verdict read from rubric markers. Its cost is empty.
• Model: one model call in its own context, at the planning tier of the Session’s admitted ModelRoutingProfile, judging the current revision’s output. Model-graded verdicts carry a route (which model answered) and an input_digest (a hash over rubric, transcript, the typed Report declaration when present, and deliverables).

Which tier runs is a platform deployment setting, not a per-request choice. Grader tokens are billed under the parked Run’s real identity. There is no synthetic evaluation account.

Hosted deployments must select the model grader explicitly. Missing, invalid, or marker configuration makes the Outcome capability unavailable: a direct definition is refused before a Run is admitted, and an Outcome-driven Automation records a failed firing before creating a Session. The marker grader is a local/test adapter and never silently stands in for independent model grading in a hosted deployment. Read the same readiness fact from GET /v1/support/capabilities.

An Outcome has control-flow authority because needs_revision can create another Run. A FlowReview is different: it runs once after settlement, reports on the process, and can never revise or change the Outcome.

Typed Reports

Add report when downstream systems need comparable values instead of only a verdict. Every output has a stable snake_case key, display title, primitive type (string, integer, number, or boolean), and optional acceptance criteria. Numeric outputs use an inclusive range, strings use an explicit one_of set, and booleans use exact equality. Checkfu does not execute customer expressions as policy.

extraction: "model" asks the independent Outcome grader to extract the object from the transcript and captured deliverables. extraction: "json" deterministically decodes the final visible agent message as one JSON object. In either mode Checkfu applies the same domain validator after extraction; missing, mistyped, or out-of-range fields are rejected explicitly.

Report values are execution content. They remain in the retention-protected Session log, while firing history receives only content-free output and acceptance counts. A zero-data-retention Workspace rejects a declared Report at define time because it cannot make the promised durable evidence/history contract. A Report never changes a terminal failed verdict into success, and an invalid Report cannot accompany satisfied work.

For an Automation, GET /v1/automation-firings/{id}/report returns the detailed retained result plus provenance for the exact evaluation event, input digest, and grader route. GET /v1/automations/{id}/reports provides cursor-paginated history, and /v1/automations/{id}/reports/export returns a self-describing JSON export. History is ordered by when each Report became available. Ascending traversal discovers a late-settling firing after cursors already issued for earlier Reports; descending traversal is a newest-first snapshot, so refresh its first page to discover new arrivals. If retention has expired or erased the underlying evaluation event, the detailed read returns no report even though the content-free firing summary can remain.

Verdicts and iteration

outcome.evaluation_completed carries a result, a freeform explanation (the grader’s rationale), and cost. The result is one of five values, not a boolean and not a score:

result	Meaning	Terminal
satisfied	The work met the rubric	Yes
needs_revision	Not yet: revise and try again	No
max_iterations_reached	Out of attempts	Yes
failed	The rubric cannot apply to this work	Yes
interrupted	user.interrupt landed after evaluation began	Yes

needs_revision is the only non-terminal result: the grader’s explanation becomes the authored input of one continuation Run, the iteration counter advances, and the agent tries again. The four terminal results settle the turn to idle. Checkfu settles max_iterations_reached immediately. There is no extra acknowledgment turn.

Outcome events are first-class domain events (outcome.evaluation_started, outcome.evaluation_completed), not telemetry spans. They are in the outbound webhook catalog, so you can react to a grading verdict without polling.

Workspace analytics

GET /v1/outcomes/analytics summarizes the latest Outcome in every addressable Session. The API owns the aggregation: it reads content-free projections from the Session event-log authorities and returns terminal-result distribution plus evaluation and revision counts. Optional principal_id and agent_id filters narrow the Session directory before aggregation.

The scope is deliberately latest_per_session, not every historical Outcome. Rubrics, grader explanations, deliverables, and typed Report values never enter this response. If more than 250 Sessions match, Checkfu returns complete: false with reason: session_limit_exceeded and no partial metrics, so a client cannot mistake a sampled success rate for Workspace truth.

Next steps

Events

The full drive-event and outcome-event shapes.

Multiagent threads

When one agent’s outcome is another agent’s job.