Handle large tool results

This is the documentation for unreleased changes on main. For the released library, read v0.1.

Two decisions, deliberately independent because they answer different questions.

Elision is about context economy and belongs to the agent's own budget. Offload is about storage engines. Keeping offload off the Spec is what makes a Version hash independent of which backend a deployment happens to be wired to: the same Spec must run identically on all four stores.

A 100KB result can sit in the log inline while still being elided for the model. A 5MB one cannot sit in the log on any adapter, because the offload threshold is chosen against the tightest backend and applied uniformly so behaviour does not vary by which store you picked.

An offloaded result always elides too, regardless of the Spec's own threshold, because the record's result is None once the payload moves and the conversation projection needs a handle to know that.

The guarantee that is kept, and the one that changes

The design says a large result is elided in the model's context but "the log always holds the whole thing". DynamoDB caps one item at 400KB. Above that, "the log holds the whole result" and "DynamoDB is first-class" cannot both describe the same write, because there is no such write.

What gives is where the bytes live, not the guarantee. Nothing a tool returned is ever discarded, and every byte stays durably retrievable for as long as the Run's log exists. It stops being one physical write of the ToolCallFinished record: above the threshold the record carries a reference, a size and a content type, and the payload lives in a BlobStore.

Wiring

from psych_runtime.store.blob_fs import FilesystemBlobStore
from psych_runtime.store.blob_s3 import S3BlobStore
from psych_runtime.store.blob_memory import InMemoryBlobStore

runtime = psych_runtime.Runtime(..., blob=FilesystemBlobStore(root="/var/lib/psych/blobs"))

Without blob=, large results still elide for the model exactly as before, but one large enough to need offload records an explicit failure rather than being written inline. That failure is the honest answer for a consumer who has not wired one up, not a reason to force a blob store into every Runtime that never produces results this large.

FilesystemBlobStore creates root if it does not exist. Retention and cleanup for it are yours, the same as any other directory or bucket.

How the model reads an elided result

It gets a handle and calls read_tool_output. That is the only handle Psych has, which is why invalid_deferred_handle guards it.

read_tool_output never constructs a BlobKey from anything the model sent. It resolves the handle against the Run's own recorded handles first, so a model cannot address a blob by making one up.

BlobKey is (tenant, run_id, call_id), never a bare string

The same discipline as pooling an MCP client by (scope, server, credential) rather than by URL. A flat string key is that mistake waiting to happen: a handle is an opaque string a call argument could carry, and if the address a caller could construct from one ever came from untrusted input, the tenant that minted it would not be part of the address at all.

Every adapter folds the tenant into where bytes are actually kept, a filesystem subdirectory or an S3 key prefix, so two tenants cannot collide even if a run_id/call_id pair somehow repeated.

Attachments: several outputs from one call

One tool result is one blob. A code execution is not: it produces stdout, stderr, a returned value and any files the program wrote, and each is read separately. Those arrive as ResultAttachments on the ToolCallFinished record, each with its own handle:

attachment.name  # "stdout", "stderr", "value", "file:report.csv"
attachment.handle  # out_<call_id>_<slot>, registered in the reducer
attachment.content_type, attachment.size_bytes, attachment.observed_bytes
attachment.truncated  # the backend stopped capturing at max_bytes
attachment.stored  # "inline" | "blob" | "preview_only"
attachment.sha256, attachment.readable

Their blob keys fold the slot into the call segment (BlobKey(tenant, run_id, f"{call_id}.{slot}")), so the tenant is still part of the address and two attachments of one call never collide. read_tool_output resolves an attachment handle exactly the way it resolves a result handle: against the Run's own recorded handles, with the slot prefix checked, so a fabricated or borrowed handle is AccessDenied rather than someone else's bytes.

Small attachments stay inline in the record. Larger ones go to the BlobStore. With no BlobStore wired they become preview_only: the preview is kept, readable is False, and the payload says how many bytes were let go — unless the agent asked for OutputPreservation.REQUIRED, which fails the call with blob_store_required rather than losing them quietly.

Ranged reading matters more here than anywhere else, because a program's stdout can be megabytes: read_tool_output takes a line offset and limit, or a regular expression, and returns a window. Nothing re-inserts a whole attachment into the model's context. See psych-sandbox.

Compact bytes, both ends

A structured result's preview is rendered with separators=(",", ":"). The threshold and the preview the model reads are measured from the same compact bytes, never from pretty-printed whitespace nobody downstream pays for. A str result never reaches that json.dumps call at all: it is text the tool returned, not a structure Psych serialises, so it is left byte for byte alone.

Adding an adapter

Same shape as a Store: subclass BlobStoreContractSuite from psych_runtime.store.blob_contract in tests/functional/test_blob_<name>.py with a blob_store fixture.

Psych's own S3 tests run against a real local S3-compatible server (moto's ThreadedMotoServer) rather than skipping.

Reading it back in a report

for call in report.tool_calls:
    call.result_bytes  # the full size, always
    call.result_handle  # set when offloaded
    call.preview  # what the model saw when elided

Gotchas

• read_tool_output is a reserved tool name.
• An attachment handle is not a blob address. It resolves through the Run's records. Nothing the model writes is ever turned into a BlobKey directly.
• Elision is not truncation of the record. The log's record is complete or carries a reference; only what reaches the model is shortened.
• Limits.large_result_bytes changes the Version hash. Offload's threshold does not exist on the Spec at all, which is the point.