Checked Against the Record

In short. A model's answer is a claim about what happened. Talos, a local coding assistant, derives each turn's status from how it classified the request and from its own execution and verification records, labels file-change claims it recognizes and cannot match to those records, and prints what it did not assess. The model's weights are unchanged. The harness changes the conditions under which the model works and what the turn's report can be trusted to say.

Two files claimed, one edited

A test in the Talos codebase pins a simple case. The model's final answer says "I updated README.md and settings.json." The turn's execution records show one successful edit, to README.md. The test checks that the claim is flagged as unsupported. The label layout and the advisory status shown below follow from reading the code, and no test pins them.

Talos reads the answer for current claims of file changes by the assistant, resolves each named target against the workspace path, and compares it with the targets of recorded successful file operations. settings.json has no match, so the answer is prefixed with a label and a list of the unmatched targets.

[Truth check: the response below claims a Talos file change, but supporting
successful execution is not recorded for that claim in this turn. Other actors
and unrecorded filesystem changes are not assessed.]

Targets without matching execution: settings.json.

The model's text follows unchanged below the label. Unless a stronger fact decides the status first, such as a partial change, the turn is classified as advisory instead of complete.

Under the old check the same answer passed without a label. The old check was existential. Any successful edit anywhere in the turn suppressed it, so one real edit covered a claim about two files. The old label also ended with a sentence the runtime could not support, "No file on disk was actually modified." Talos records its own tool executions. It does not track an editor, a build script or another process writing to the same disk. The change made the check per target and narrowed the label to what the records show. The published 0.10.8 beta still carries the old check and the old wording, so in that release the answer above passes without a label.

What the harness changes, and what it does not

A harness does not change the model's weights. It does change the conditions the model works under, and with them what the complete system accomplishes in some cases. A file-change request that produces no tool call gets one bounded retry with a nudge toward the write tools. A workspace question answered without reading anything is retried with read tools, and the runtime can read a file and hand its content to the model.

Those interventions can also go wrong. In a recorded run on a development build, the retry that followed a model's first reply dropped the conversation history, the file content and the textual tool schemas, and the model's second attempt produced an invalid edit. The code that builds that retry request has not changed since. The same compaction drops a note the runtime prepares when a model denies it can access files, so that note never reaches the model.

So the subject of this piece is narrower than model quality. It is what the runtime can establish about a turn, and how it reports what it cannot.

Status comes from records

A completed Talos turn ends with one of eight classifications. They are verified, unverified, read-only answered, partial, blocked by approval, blocked by policy, advisory and failed. No success classification is taken from the model's prose. The prose can only lower a status, through the claim check above and a few other lexical checks, such as withheld command output or a recognized denial of file access.

• A request that requires a file change, where the model never calls a file tool, ends as blocked by policy. A test named for the rule, requiredMutationWithoutToolsStaysIncompleteRegardlessOfNarrative, feeds seven narratives, among them quoted, passive and proposed ones, and checks that none lets such a turn count as complete.
• Verified is reached only when a verifier passed or a required verification command succeeded. For an exact edit it means the approved replacement was observed in the file after the edit under the verifier's checks. It does not mean the fix is correct.
• A change no verifier could check is reported as complete but unverified, with a neutral marker instead of the success glyph, and a line that begins "[File write/readback passed. No task-specific verifier was applicable, so task completion was not verified."
• When static verification fails on a turn whose changes all applied, the turn becomes failed and the model's success summary is replaced by the runtime's result, with the sentence "The assistant success summary was replaced with this runtime verification result because verification failed."

When several facts apply, a fixed order decides, and it shows which facts outrank a false claim.

1. Malformed tool-call text in an answer that called no tool, or invalid change arguments when no change succeeded, make the turn failed.
2. A denied command blocks it by approval, and a failed command fails it.
3. A change refused by a read-only mode, or a required change with no tool call, blocks it by policy.
4. A denied change or a denied protected read blocks it by approval. A protected read that never got approval blocks it by policy.
5. A partial change makes it partial.
6. Failed verification makes it failed.
7. On a request that asks only for verification, a successful required command makes it verified.
8. Advisory facts, among them an unsupported change claim, missing evidence or an ungrounded answer, make it advisory.
9. A passing verifier makes it verified.
10. A request that asked for no change is read-only answered, and anything else is complete but unverified.

Rule 7 comes before rule 8, so a request that asked only for verification ends as verified when its required command succeeded, even if the model also claimed an edit it did not make. The runtime then replaces the answer with a one-line summary of the command's result, so neither the claim nor the label is shown, and the turn's trace keeps only a fixed warning that an unsupported change claim was made, without its text or targets.

Unless a failed change left effects on disk, the post-apply verifier runs only when the facts known before it leave the turn complete or partial. A turn already advisory because of an unsupported claim is not checked by it, so an edit that applied but broke the file can leave that turn advisory instead of failed.

After an interactive turn that does not end in an error, the terminal also prints the work outcome and the line Whole-request coverage: not assessed, unless the timing display is turned off in the settings. The runtime checks parts of a request, such as named targets, but never whether the whole request was satisfied, and the output says so.

What the claim check recognizes

The detection is not a model judging a model. It is a bounded lexical grammar over 27 verb forms, described in its own source as "Bounded lexical attribution of answer claims. No execution evidence is inferred from prose." That makes it predictable and testable. It also sets its limits.

• Negations, proposals, reported or rejected quotations, changes attributed to someone else and abstract objects are left alone. "I updated my assessment" is not a file claim. A bare quotation of a change claim gets the weaker label.
• A first-person claim whose time cannot be resolved, or a claim placed in this turn without a clear actor, gets a weaker label, which says the records do not establish it, instead of calling it false. A passive claim with no actor and no time, such as "the README has been updated", gets no label.
• Some verbs are outside the grammar. "Fixed", "added" and "implemented" are among them, so "I fixed the bug in calc.py" passes unlabeled.
• A claim that names no file passes when any file change in the turn succeeded, unless it says all, every or each, which gets the weaker label.
• The check does not run when a change in the turn was denied, when a failed change left applied or uncertain effects, or when the only changes were invalid calls. Those turns get their own deterministic summaries.
• Target names are compared exactly, including letter case. On a case-insensitive file system that can label a true claim about Readme.md as unsupported, which is the safer error.

Some runtime messages still use the broad wording the change removed from the label, for example [Action obligation failed: no file was changed in this turn.]. That message is accurate only for Talos's own tools.

The approved payload is the written payload

Approval only means something if the approved change is the change that lands. Before anything is shown to the user, Talos may transform a proposed write in several ways. Among them are removal of markdown commentary the model wrapped around the content, an exact-literal correction that restores a payload the user dictated word for word, path canonicalization, and preservation of the file's existing line endings, byte order mark and final newline.

All of these run before the approval preview is built. The preview, the approval prompt, the checkpoint and the write are then built from the same call content, and nothing changes it between approval and execution. The commentary sanitizer is documented as non-idempotent, so a second pass could strip legitimate content. It has one production caller, and the write tools are documented not to sanitize again.

After the write, the tool reads the file back and compares the text with what it wrote. A mismatch fails the call as an integrity failure, after the file was already replaced, so the checkpoint is the undo path. A test named writeContentSanitizedBeforeApprovalSoApprovedBytesAreWrittenBytes checks that the file on disk equals the sanitized payload, that the approval showed the byte count that landed, and that the trace hash matches the approval hash.

The precise guarantee is about what is built and written. The person approving sees a preview and a diff that may be shortened, not every byte.

What is not checked

Some of these limits are written into the project's documentation, and a test fails the build if that wording drifts. Others come from reading the code and from one recorded run.

• Read-only explanations may remain unverified. A wrong explanation of code that changed nothing is not caught.
• Claims about command output are withheld only for recognized shapes, such as a git status, a test run or a file listing, when no matching tool result exists.
• A no-tool turn that was not a change request is not checked for change claims.
• Claim detection is lexical, so paraphrases outside the grammar pass.
• Traces and logs are not tamper-evident. There is no hash chain, signature or append-only enforcement.
• In that recorded run the harness itself produced misleading text. A correct answer was replaced by "did not complete" text, which the code now prevents but no live run has rechecked. A question about a bug got a "[Path existence verified]" banner, and that trigger is still present.

The README draws the boundary in two sentences. "Talos is a governance shell around local model execution, not proof that the model is capable or honest without runtime evidence. A documented limitation is not an overclaim."

What transfers

• Derive a turn's status from execution and verification records, and let the model's prose only lower it.
• Match change claims per target. One real edit must not cover a claim about two files.
• Scope every runtime message to what the runtime can observe. A tool log cannot speak for the whole disk.
• Build the preview, the approval and the write from the same payload, after every transform.
• Print what was not assessed, every time, instead of leaving it implied.

The model's prose is kept, kept under a label, or replaced by runtime text, and pass markers the model wrote for itself are removed. What can be held to account is the turn's report about the model.