Runtime

This is the documentation for unreleased changes on main. For the released library, read v0.1.

Runtime

class

class Runtime(store: 'Store', model: 'ModelClient', registry: 'ToolRegistry', resolver: 'ToolResolver | None' = None, executor: 'ToolExecutor | None' = None, prices: 'PriceResolver | None' = None, cost_policy: 'CostPolicy' = 'prefer_provider', memories: 'Sequence[str]' = <factory>, policy: 'Policy | None' = None, approval_selectors: 'Sequence[str]' = <factory>, telemetry: 'Telemetry | None' = None, sandbox: 'Sandbox | None' = None, memory: 'MemoryPort | None' = None, end_user_id: 'str | None' = None, mcp: 'McpTools | None' = None, a2a: 'A2ATools | None' = None, blob: 'BlobStore | None' = None, catalogue_budget_chars: 'int' = 20000, blob_offload_bytes: 'int' = 300000, http: 'HttpCaller | None' = None)

Everything a Worker needs to execute a Run.

Assembled once at boot by the consumer and handed to every Worker in the process. Holds no per-Run state, so one instance serves any number of concurrent attempts.

Worker

class

class Worker(store: 'Store', runner: 'AttemptRunner', *, worker_id: 'WorkerId | None' = None, lease_seconds: 'float' = 30.0, heartbeat_seconds: 'float' = 10.0, grace_seconds: 'float' = 60.0, supervisor_interval: 'float' = 5.0, poll_interval: 'float' = 0.5, concurrency: 'int' = 4, max_attempts: 'int' = 10) -> 'None'

Claims runnable Runs and executes them until told to stop.

Session

class

class Session(store: 'Store', registry: 'ToolRegistry', runtime: 'Runtime', worker: 'Worker', scope: 'Scope', _versions: 'dict[str, Version]' = <factory>)

A running Worker, a store, and the publish-dispatch-wait sequence.

Built by session(); constructing one directly means owning the Worker's lifecycle yourself, which is what session() exists to avoid.

The four fields below are the real objects, not wrappers. A consumer who needs a second Worker, a per-tenant Runtime or a store transaction reaches through them rather than around this class.

Dispatched

class

class Dispatched(run_id: 'RunId', created: 'bool')

What a dispatch produced.

Attributes: run_id: the Run, new or existing. created: False when an existing Run was returned for a repeated idempotency key. A caller that wants to know whether their retry was the one that did the work reads this.

ToolRegistry

class

class ToolRegistry() -> 'None'

Names to functions, for one process.

Not thread-safe for concurrent registration, and deliberately so: registration happens at boot on one thread, and a lock here would suggest otherwise. Lookups after boot are reads and are safe from anywhere.

Scope

model

class Scope(*, tenant: Annotated[str, MinLen(min_length=1), MaxLen(max_length=256)], principal: Annotated[str | None, MaxLen(max_length=256)] = None, labels: dict[str, str] = <factory>)

The tenancy and identity context threaded through every call.

Frozen, because a Scope that can be mutated after a Record is stamped with it is a Scope that can be mutated between the authorization check and the call it authorised.

Attributes: tenant: the isolation boundary. Two Runs with different tenants must never see each other's data, and this is the field that decides it. principal: who is acting, when the consumer knows. Psych records it and passes it to Policy; it never interprets it. labels: free-form consumer metadata, stamped on Records and available for their own filtering and metering.

McpPool

class

class McpPool(*, transport: 'McpTransport', secrets: 'SecretResolver', catalogue_ttl_seconds: 'float' = 300.0, oauth: 'OAuthClient | None' = None, oauth_identity: 'ClientIdentityConfig | None' = None, oauth_grant: 'GrantKind' = 'client_credentials') -> 'None'

Pools McpConnections by McpPoolKey.

One pool per process is the intended shape: every Run, for every tenant, shares it, and isolation comes entirely from the key, not from having a separate pool per tenant (which would just move the one-line mistake DESIGN.md §10.4 warns about to whoever wires up per-tenant pools).

oauth is optional. Absent, a server behind OAuth still fails with McpServerUnreachable on its first 401, the same as it always has, except the message now says the server wants OAuth rather than just naming the status code. Present, this pool's connections acquire, refresh, and step up tokens against it automatically; see this module's docstring for the full flow and how the pool key follows an acquired token.

oauth_identity and oauth_grant are the default client identity and grant kind, used only for a server whose Spec sets no McpServer.oauth. A server that does declare one is connected with that identity and grant instead, so one pool serves any number of servers with different OAuth client identities and different grant kinds in the same Run -- see this module's docstring, "One pool, many OAuth identities and grants".

McpTools

class

class McpTools(pool: 'McpPool', *, tenant_policy: 'Any' = None, describe_server: 'Callable[[Scope, McpServer], str | None] | None' = None) -> 'None'

The seam between a pool of MCP connections and a running agent.

Two halves, because the resolver and the executor need different things and both must narrow the same way (DESIGN.md §10.5):

• tools_for satisfies psych_runtime.tools.resolver.McpCatalog, so a turn's tool set includes what each granted server currently offers.
• call runs one of those tools.

They live on one object rather than being wired separately because they share the inputs that decide access: the pool, which carries tenancy in its key, and the tenant policy. Wiring them from two places is how the resolver's view and the executor's view drift apart, and a drift in this direction is a model calling a tool the Spec excluded.

Why call narrows again

The resolver already filtered the tool set the model was shown. That is not a control, because the model chooses the name it sends: a caller that looked a name up across every connected server and invoked whatever answered would let a model reach a tool the Spec's allow list excluded simply by naming it. So this applies the same narrow the resolver applies, against the same three planes, and a name that does not survive it is refused however the model got hold of it.

Tenancy rides on scope: a name is resolved only against connections keyed to the calling Scope, so tenant A's Run cannot reach tenant B's server even if both grant a tool of the same name.

A2APool

class

class A2APool(*, transport: 'A2ATransport', secrets: 'SecretResolver', card_ttl_seconds: 'float' = 300.0) -> 'None'

Pools A2AConnections by A2APoolKey.

One pool per process, shared by every Run of every tenant, with isolation coming entirely from the key -- the same shape as McpPool, and for the same reason: a pool per tenant just moves the one-line mistake to whoever wires the pools up.

A2ATools

class

class A2ATools(pool: 'A2APool', *, tenant_policy: 'Any' = None) -> 'None'

The seam between a pool of peer connections and a running agent.

Two halves that must narrow identically, on one object for the reason McpTools gives: wiring the resolver's view and the executor's view from two places is how a model ends up able to call a peer the Spec excluded.

HttpTransport

class

class HttpTransport(*, policy: 'EgressPolicy | None' = None, client: 'httpx.AsyncClient | None' = None, timeout: 'httpx.Timeout | float | None' = Timeout(connect=10.0, read=30.0, write=30.0, pool=10.0)) -> 'None'

The one object that owns outbound HTTP in Psych.

Wraps a single httpx.AsyncClient and checks every request against an EgressPolicy before it goes out. The model client, HTTP tool executor and MCP client all take one of these rather than building their own httpx client, which is what makes the seam a single seam instead of a convention repeated at each call site.

Two methods only: request for a call that returns a complete response, and stream for one where the body is consumed incrementally, which the OpenAI-compatible model adapter needs for server-sent events. Both check the policy before doing anything else.

OpenAICompatibleClient

class

class OpenAICompatibleClient(*, base_url: 'str', transport: 'HttpTransport', scope: 'Scope', api_key: 'str | None' = None, headers: 'Mapping[str, str] | None' = None) -> 'None'

ModelClient over the OpenAI-compatible chat completions wire format.

Attributes are private; construct with keyword arguments only.

Args: base_url: the API root, e.g. https://proxy.internal/v1. Requests go to \{base_url}/chat/completions and \{base_url}/models. transport: the egress seam. Never construct an httpx client here directly; that would be a second, uncontrolled path to the network. scope: the tenant this client instance acts for. Stamped on every egress check this client makes. api_key: sent as Authorization: Bearer <api_key> when set. Some proxy deployments authenticate at the network layer instead and need none. headers: extra headers merged into every request, for a proxy that wants a tenant header or a routing hint.

AllowAll

class

class AllowAll()

The default Policy: yes to everything.

Psych ships this because a consumer evaluating the library should not have to write an authorization system first, and because Psych genuinely does not have an opinion about who may do what. It is not a security control and its name says so.

Decision

class

class Decision(allowed: 'bool', reason: 'str' = '', requires_approval: 'bool' = False)

The consumer's answer.

Attributes: allowed: whether the call may proceed. reason: why not, when it may not. Reaches the model as the tool's result so it can explain to the user or try something else, which is why it should read as an explanation rather than an error code. requires_approval: the consumer wants a human decision. The Run suspends rather than failing, and resumes on the decision.

ToolClass

enum

class ToolClass(*values)

How dangerous a tool is, from its MCP annotations.

ResolvedCredential

model

class ResolvedCredential(*, identity: str, secret: pydantic.types.SecretStr)

One resolved credential: a secret value plus its non-secret identity.

identity is safe wherever secret is not: as a pool key, in a log line, in a repr taken while debugging. Pydantic's SecretStr already keeps secret out of this model's default repr and str (it prints as SecretStr('**********')), and the model is frozen so neither field can be swapped for an unmasked one after construction.

SandboxLimits

model

class SandboxLimits(*, cpu_seconds: Annotated[float, Gt(gt=0)], address_space_bytes: Annotated[int, Gt(gt=0)], file_size_bytes: Annotated[int, Gt(gt=0)], process_count: Annotated[int, Gt(gt=0)], wall_seconds: Annotated[float, Gt(gt=0)])

Resource caps for one execution.

Defaults and rationale live on the adapters that apply them (psych_runtime.sandbox.subprocess, psych_runtime.sandbox.container), because a number with no story behind it is a number nobody can safely change later. This model only fixes the shape both adapters share, so the contract suite (psych_runtime.sandbox.contract) can drive both with the same values and prove they mean the same thing on both backends.

Attributes: cpu_seconds: CPU time actually consumed, not wall-clock time. A program blocked waiting on a host binding's reply burns no CPU budget while it waits. address_space_bytes: the virtual address space the process may map. Exceeding it surfaces to the program as an ordinary MemoryError, not a killed process, because CPython's allocator checks malloc's return value rather than trusting it. file_size_bytes: the largest file the process may write, counting every file it opens, not a total across files. process_count: how many processes (Linux counts threads too) the executing user may hold at once, across everything else running as that user. See psych_runtime.sandbox.subprocess for why this is a per-user limit rather than a per-execution one and what that implies for concurrent runs. wall_seconds: real time from spawn to teardown. This is the only cap here that is not a resource limit on the process; it is enforced by the host killing the process (or container) directly.

SandboxResult

model

class SandboxResult(*, stdout: str = '', stderr: str = '', value: Any = None, failure: psych_runtime.sandbox.port.SandboxFailure | None = None, duration_seconds: Annotated[float, Ge(ge=0)], limit_hit: psych_runtime.sandbox.port.SandboxLimit | None = None, network_denied: bool = False, stdout_truncated: bool = False, stderr_truncated: bool = False)

What one execution produced, success or failure.

Attributes: stdout: everything the program printed to standard output. stderr: everything it printed to standard error. value: what the program's top-level code evaluated to, JSON-shaped. None both when the program returned nothing and when it genuinely returned None; the two are not distinguished because JSON does not distinguish them either. failure: set when the program did not complete cleanly. See SandboxFailure. duration_seconds: wall-clock time from spawn to teardown. limit_hit: which cap in SandboxLimits ended the execution, if one did. None on a clean completion or on a plain program error that no limit caused. network_denied: True only when this execution actively verified that it has no route to the network, False when it did not (network was granted, or denial was requested but the host could not establish it). See the adapter docstrings for exactly what "verified" means on that backend and what it does not guarantee; this field reports the adapter's own honest self-check, never an assumption. stdout_truncated: the program printed more than the adapter buffers. stderr_truncated: same, for standard error.

SandboxFailure

model

class SandboxFailure(*, kind: Annotated[str, MinLen(min_length=1), MaxLen(max_length=128)], message: Annotated[str, MaxLen(max_length=8192)], traceback: Annotated[str | None, MaxLen(max_length=65536)] = None)

A sandboxed program's failure, as data the model can read.

Deliberately shaped like psych_runtime.core.records.ToolFailure (kind, message, traceback) so a caller can build one from the other with a field-for-field copy. It is not the same type: psych_runtime.sandbox does not import psych_runtime.core.records for it, because a port should not depend on the record shape of whatever happens to consume it today, and tying the two together would make an unrelated change to the Record schema a breaking change here too.

SandboxLimit

enum

class SandboxLimit(*values)

Which resource cap ended an execution, when one did.

A single field naming the failing dimension rather than five booleans: exactly one limit ends a given execution (the process dies, or the program raises, on the first cap it crosses), so a scalar is the honest shape and a set would imply a possibility that cannot happen.

HostBinding

value

def HostBinding(*args, **kwargs)

No docstring. That is a bug in the library rather than in this page: the reader who found this symbol in their editor sees the same gap.

BlobKey

class

class BlobKey(tenant: 'str', run_id: 'RunId', call_id: 'ToolCallId')

A blob's address: tenant, Run and call, never fewer.

Deliberately not a bare string. See the module docstring's tenant-scoping section: the point of this type existing is that a caller cannot construct a valid address without a tenant, the same way Scope.pool_key makes tenancy a mandatory half of an MCP pool key rather than something a call site has to remember to include.

RunHeader

model

class RunHeader(*, run_id: psych_runtime.core.ids.RunId, scope: psych_runtime.core.scope.Scope, version_hash: psych_runtime.core.ids.VersionHash, state: psych_runtime.store.port.RunState, created_at: datetime.datetime, deadline_at: datetime.datetime, lease_holder: Optional[psych_runtime.core.ids.WorkerId] = None, lease_expires_at: datetime.datetime | None = None, idempotency_key: Annotated[str | None, MaxLen(max_length=256)] = None, attempt_count: Annotated[int, Ge(ge=0)] = 0, runnable_at: datetime.datetime | None = None, parent_run_id: Optional[psych_runtime.core.ids.RunId] = None, delegation_depth: Annotated[int, Ge(ge=0)] = 0, continues_run_id: Optional[psych_runtime.core.ids.RunId] = None)

The mutable row beside a Run's immutable log.

Everything that changes about a Run lives here; everything that happened lives in the log. Keeping the two apart is what lets the log stay append-only while a lease is renewed every few seconds.

Attributes: run_id: the Run. scope: tenancy. Every store query filters by it (DESIGN.md §14). version_hash: the Version this Run pinned at admission and reads for its whole life. state: claimability. lease_holder: the Worker holding the lease, if any. lease_expires_at: when that lease stops being honoured. A Run whose lease is past this is claimable by anyone, and that is the only mechanism by which a crashed Worker's Run recovers. deadline_at: when the supervisor should fire the abort signal. idempotency_key: what makes admission exactly-once per key. attempt_count: how many times this Run has been claimed. Bounds crash reclaim so a Run that kills every Worker it touches eventually stops. runnable_at: not claimable before this. Used by a Run waiting on a clock, which suspends as EXTERNAL. continues_run_id: the immediate predecessor Run in a conversation thread, mirrored from RunAdmitted.continues_run_id so a caller can check a continuation's Scope with one get_run rather than reading the whole log. Distinct from parent_run_id, which is delegation -- see that field's own docstring in psych_runtime.core.records.

RunState

enum

class RunState(*values)

Where a Run is, from the store's point of view.

Deliberately coarser than the terminal states in psych_runtime.core.records: the store needs to know what is claimable, and the log holds the detail.

ModelRequest

model

class ModelRequest(*, model: Annotated[str, MinLen(min_length=1)], messages: tuple[typing.Annotated[psych_runtime.core.messages.SystemMessage | psych_runtime.core.messages.UserMessage | psych_runtime.core.messages.AssistantMessage | psych_runtime.core.messages.ToolResultMessage, FieldInfo(annotation=NoneType, required=True, discriminator='role')], ...], tools: tuple[psych_runtime.core.messages.ToolDefinition, ...] = (), temperature: float | None = None, top_p: float | None = None, max_output_tokens: int | None = None, reasoning_effort: Optional[Literal['low', 'medium', 'high']] = None, stop: tuple[str, ...] = (), cache_breakpoints: tuple[int, ...] = (), idle_timeout_seconds: Annotated[float, Ge(ge=0)] = 300.0, extra: dict[str, typing.Any] = <factory>)

One call to a model.

Attributes: model: the model id. A Spec names its model; there is no router (DESIGN.md §19). messages: the conversation, system message first. tools: what the model may call this turn. Resolved per turn and fixed for its duration, so the provider's cached prompt prefix stays valid (§10.2). cache_breakpoints: indices into messages after which the provider may place a cache breakpoint. Providers that do not support explicit breakpoints ignore this. Expressed as indices rather than as flags on the messages so that the same conversation can be sent to two providers with different caching models. idle_timeout_seconds: fail the read when no chunk arrives for this long. 0 disables it. The timer runs only while a source read is outstanding, so consumer backpressure never trips it (§8.5).

ModelTimings

model

class ModelTimings(*, queue_wait_seconds: Annotated[float, Ge(ge=0)] = 0.0, time_to_first_token_seconds: Annotated[float | None, Ge(ge=0)] = None, stream_duration_seconds: Annotated[float, Ge(ge=0)] = 0.0)

Where the time went in one model call (DESIGN.md §13.3).

Separate from the record's own timestamp because the gap between a Run's wall-clock and the sum of its parts is the interesting number, and it cannot be computed if the parts were never measured.

ToolDefinition

model

class ToolDefinition(*, name: Annotated[str, MinLen(min_length=1)], description: str = '', input_schema: dict[str, typing.Any] = <factory>, annotations: frozenset[str] = frozenset())

A tool as described to the model.

Built fresh at every turn by the resolver (DESIGN.md §10.2) from the Spec, the registry and whatever the MCP catalogue currently offers, then narrowed.

Memory

model

class Memory(*, id: Annotated[str, MinLen(min_length=1), MaxLen(max_length=64)], key: psych_runtime.memory.port.MemoryKey, content: Annotated[str, MinLen(min_length=1), MaxLen(max_length=4096)], created_at: datetime.datetime)

One durable fact, remembered for one end user.

MemoryKey

model

class MemoryKey(*, tenant: Annotated[str, MinLen(min_length=1), MaxLen(max_length=256)], end_user_id: Annotated[str, MinLen(min_length=1), MaxLen(max_length=256)])

The isolation boundary for durable memory.

Frozen and hashable so it can be a dict key in an adapter's own storage, which is exactly how psych_runtime.memory.store_backed uses it.

InMemoryStore

class

class InMemoryStore() -> 'None'

A Store backed by plain dicts, guarded by one asyncio.Lock.

Implements the Store protocol structurally; there is no base class to inherit because the port is a Protocol.

InMemoryBlobStore

class

class InMemoryBlobStore() -> 'None'

A BlobStore backed by a plain dict, guarded by one asyncio.Lock.

Implements the BlobStore protocol structurally; there is no base class to inherit because the port is a Protocol.

DEFAULT_PRICES

value

A PriceResolver over a fixed dict.

Exact match first, then the longest matching prefix. The prefix fallback exists because providers version model ids by suffix (gpt-4o-2024-08-06), and a table that only matched exactly would go stale silently every time a provider dated a release. A prefix match is still a real match against a rate someone entered, never a guess.

ModelPrice

model

class ModelPrice(*, input: Annotated[decimal.Decimal, Ge(ge=Decimal('0'))], output: Annotated[decimal.Decimal, Ge(ge=Decimal('0'))], cache_read: Annotated[decimal.Decimal, Ge(ge=Decimal('0'))], cache_write: Annotated[decimal.Decimal, Ge(ge=Decimal('0'))], cache_write_1h: Annotated[decimal.Decimal | None, Ge(ge=Decimal('0'))] = None, currency: Annotated[str, MinLen(min_length=3), MaxLen(max_length=3), _PydanticGeneralMetadata(pattern='^[A-Z]{3}$')] = 'USD')

Four per-million rates for one model.

Attributes: input: cost per million uncached input tokens. output: cost per million generated tokens. cache_read: cost per million tokens read from the prompt cache. cache_write: cost per million tokens written to the prompt cache at the provider's default retention. cache_write_1h: cost per million tokens written at one-hour retention. Some providers charge more for the longer hold. When None the cache_write rate applies to those tokens too. currency: ISO 4217, uppercase.

StaticPriceTable

class

class StaticPriceTable(prices: 'dict[str, ModelPrice]', *, overrides: 'dict[str, ModelPrice] | None' = None) -> 'None'

A PriceResolver over a fixed dict.

Exact match first, then the longest matching prefix. The prefix fallback exists because providers version model ids by suffix (gpt-4o-2024-08-06), and a table that only matched exactly would go stale silently every time a provider dated a release. A prefix match is still a real match against a rate someone entered, never a guess.

CostPolicy

value

def CostPolicy(*args, **kwargs)

No docstring. That is a bug in the library rather than in this page: the reader who found this symbol in their editor sees the same gap.