Read a run

A coding run does not finish on an opinion. Between the attempt that writes the code and the attempt that reviews it, the platform runs a set of checks of its own: your repository’s gates, and the command paired with each acceptance criterion. No model takes part in it. The reviewer is shown the results and cannot change them.

When the checks run

Once per attempt at the work, after the agent has committed and before the review reads it.

Only a task bound to a repository carries them, because everything they run comes from the task’s own spec: the setup and verify commands read from your repository, the check paired with each criterion, and the negative cases. See what a software draft has to carry. A task with no repository behind it runs no checks, and nothing about it changes.

Every check runs every time, including after one has already failed, with the mutation floor as the single exception noted below. The whole list is worth more than the first line of it, so a red result tells you everything that is wrong at once rather than one thing per iteration.

The checks

Check	What it does, and when it is red
Prepare	Records the setup commands your repository needs, run once on the working copy the agent got. Setup that fails ends the run before any of this, so by the time you read a result this row is history rather than news.
Repository gates	Runs your repository’s own test, lint, and build commands, one row each, exactly as the task recorded them. Red on any command that exits with an error.
Acceptance checks	Runs the command paired with each acceptance criterion, one row per criterion, shown as `AC1`, `AC2`, and so on. Red when the command fails, or when its output is not what the criterion said to expect.
Evidence	One row per criterion, read from the evidence file below. Red when a criterion has no block in the file, or the block has no output in it.
Tests were added	Red when the whole change touches no test file at all.
Hollow checks	Catches a check that is not checking anything: a test command that ran zero tests while reporting success, a gate script the change edited without a negative case to prove it still rejects bad input, and any negative case the gate accepted instead of rejecting. Red on any of those.
Mutation floor	Changes one small thing at a time in each file the task changed and runs the tests again, to see whether the tests notice. Red when too many of those changes go unnoticed. See the mutation floor below.

The evidence file

A criterion has to be shown working, not asserted. The agent writes .commandchain-agent/evidence.md in the working copy: one heading per criterion, and under each heading the command it ran and the output that command printed, pasted as it came out.

An empty output block counts as no evidence. A heading with a command under it and nothing under the output proves only that the agent knew a block belonged there, which is the exact failure this phase exists to catch. A missing heading and a missing block read the same way. All three turn the criterion’s row red.

You do not have to take the file’s word for anything either. The full output of every command these checks ran is attached to the run as a log, so a red row can be read rather than guessed at.

The mutation floor

A suite that passes proves nothing on its own. It may be exercising the behaviour it appears to cover, or it may be running past it without looking, and a green result reads the same either way.

So the platform changes one small thing at a time in each file the task changed, a comparison, an argument, the body of a function, and runs the tests again. A test that fails has caught the change, which is what it is there for. A suite that still passes was not checking that behaviour, and the share of changes that went unnoticed is the score for that file.

It runs on the files the task changed and on nothing else, once, after the other checks have come back green. A run whose checks are already red skips it, because a slow analysis buys nothing for an iteration that has failed. It runs only on the Full verification profile, which is where the slowest proof belongs. A run on the Standard or Lite profile shows the row as skipped.

Language	Tool
JavaScript and TypeScript	Stryker
Python	mutmut
Go	gremlins
Rust	cargo-mutants
Java and Kotlin	PIT

A repository in any other language shows the row as skipped and the run carries on.

The floor itself is yours. It applies to a repository only once that repository commits a mutation configuration with a score threshold in it, so raising the bar is a change you make in your own code rather than a setting here. Until a repository carries one, the row reads skipped, and skipped is not a pass. It means nobody checked.

A red check stops the run

A run cannot be accepted while any check is red, whatever the reviewer scored. A review of 100 with one red row finishes the run without accepting it, and the work stays where you can see it instead of moving on as done.

The reverse does not hold. Green checks do not accept a run by themselves: the review still has to pass, and what a run produced still waits for you. See runs and attempts for the outcomes a run can end at.

What the reviewer reports

The review that reads the work is not a single number either. For every acceptance criterion it runs the paired command itself, in its own copy of the branch, and pastes the command it ran and the output that came back. Each criterion comes back in one of three states.

Status	What it means
Met	The pasted output demonstrates the criterion
Unverified	The code looks right, and no output was produced that shows it working
Missing	The criterion is not implemented

Unverified scores zero, exactly as missing does. A plausible implementation with nothing to show for it does not pass, which is the whole reason the review pastes output instead of describing it. The score is the share of criteria marked met, and nothing else, so a run clears the bar only when every criterion has been demonstrated.

A serious bug does not subtract points on its own. It is written up under the review’s gaps and required fixes, and where it means the pasted output no longer demonstrates a criterion, that criterion is not met, which is what moves the score.

The review has to run something

The platform counts the commands a review actually executed. A review that files a full report having run nothing has described the code rather than checked it, so the report is thrown away and the review is retried, up to three attempts in all. If none of them runs a command, the run ends inconclusive. A score that cannot be traced back to a command is not worth accepting.

When the review and the checks disagree

Both the platform and the reviewer run the command paired with a criterion, so the two can come back with different answers. When they do, the run ends inconclusive: not accepted, and not rejected either. The same command produced two results, which makes the check the unreliable part, and the repair belongs in the check rather than in the code. Both outputs are kept side by side on the review so you can see where they parted.

That ending applies only when the review was the deciding one, which is to say when every check on the attempt was already green. On an attempt whose checks were red the review is advisory, as runs and attempts describes: a disagreement is recorded and nothing more, and the work carries on to the next attempt.

The red team

A review good enough to finish the work is not the last word on it. Before a run is accepted, a separate agent is asked to break what was built: the same model as the reviewer, its own copy of the code, and one objective, which is to make the implementation fail against the spec it was given.

It is told what the change touched and what the checks already cover, so it looks elsewhere. Wrong arguments at a call site, values a gate is meant to reject, boundaries, the second call after the first one worked, the path the existing tests only mock. It writes small tests aimed at those places, under tests/redteam/ in your repository’s own test framework, and runs them.

What the attack found	What happens to the run
Something serious	The run does not finish. The failing test and its output go back to the next round of work, as something to fix
Nothing serious	The run carries on to its ending as normal

The report it wrote is attached to the run beside the check logs, so a finding can be read rather than guessed at.

The attack is kept

Either way, the probes that passed stay. The platform copies them into your repository and commits them on the task’s branch, in one commit named test(redteam): probes for <task key>. Only test files under tests/redteam/ that also match your repository’s own test patterns are taken, a probe never overwrites a file you already have, and the commit is made by the platform rather than by the agent. Everything else the attacking agent wrote goes with its copy of the code.

An attack that failed to break the code today keeps running against it tomorrow, so the next change to that code has to get past it too.

A run on the Lite profile skips this step, which is one reason the verification profile is shown on the run.

The four ways a run ends

How a run ended tells you your next move.

Ending	What happened	What you do
Accepted	The review passed and no check was red	Nothing. The work moves on
Rejected	The work was produced and turned down	Read the review’s required fixes, edit the task, run it again
Inconclusive	Something around the work failed, not the work itself	Fix the cause and run it again
Partial	The run ran out of time, iterations or money with real work on a branch	Answer the decision task it raised, rather than re-running the task

A partial run is neither a failure nor an acceptance: the agent committed work and the review scored it, but the run hit a limit before that score reached the threshold. It records the best score reached and the commit that earned it, and pushes that commit to the task’s branch. The platform will not accept it for you. A decision task goes to the delivery owner instead, carrying three choices. See when a run reaches a limit.

Next steps

Runs and attempts: what a run and an attempt are, and where a run can stop.
Work in the Task Workspace: where to open the runs of a task.
Review the tasks a huddle proposes: where the paired checks and negative cases get written.