# actionbench methodology

how reliability is measured, what is counted, and what is published.

ActionBench reports reliability per site, action and recipe version, with the raw run data downloadable at `/v1/benchmark`.

## what a run is

A run counts when it finished: succeeded or failed. Denied and canceled runs (policy, dry runs) are not reliability data. A run succeeds only when every postcondition passed.

## where runs come from

Two sources, reported separately and together:

- **canary**: our own scheduled runs with synthetic inputs, against public reads and against sites whose verified owner allowed the `orsel-canary` agent.
- **client**: runs from opted-in clients. Outcomes only; never page content or credentials.

## what is reported

Success rate, postcondition pass rate, latency percentiles, results by tier, drift events and the median time from drift to a verified repair, over 24 hours, 7 days and 30 days.

## what is not hidden

- Reads and writes are never blended into one number.
- Failures and drift events are published, not filtered.
- A score with few runs says how few. An action with no runs shows no score, not a default.
- Injection-resistance results are reported with and without the trust plane enabled.

## limits

These numbers describe our fleet and opted-in clients, not every user's experience. Public agent benchmarks report much lower success on write tasks than on reads; ours will be read the same way.
