bonsai

Context management

bonsai treats the context window as a managed budget: cache-stable prefixes, token estimation per provider family, continuous garbage collection, graduated compaction, and default-on task episodes that archive finished work into compact, recallable cards. /ctx shows exactly where the window is going and gives you manual control.

How context is assembled

The system message is: static persona prompt + a project context block + optional operator suffix. Project context is split deliberately:

Connections with rolling-history caching (and Codex always) instead receive volatile state appended to history, keeping the entire prefix append-only. Memory is deliberately not in the system prompt — recalled entries arrive as per-turn background data notes.

Tool cards, results, and persisted inspection evidence use UUID call identities. A separate session snapshot preserves each call’s original provider correlator, independently of discarded context or archived episodes. Transports restore the original provider ID on assistant calls and their results. OpenAI-compatible, Anthropic, and opted-in Codex native payloads have immutable response UUIDs and a session-owned store. Live rows and omitted originals retain those UUIDs when context-control IDs change, so repeated answer text never selects another trace. Opaque signatures and reasoning payloads remain unchanged. Unsigned compatible requests still deduplicate repeated wire IDs when the backend requires request-wide uniqueness.

Token accounting

A per-model estimator drives context-pressure decisions (billing truth stays with provider-reported usage): local tiktoken or Qwen tokenizers, the Anthropic count_tokens or Z.AI tokenizer API, or a chars/4 heuristic — each tagged with a confidence. Counting and sending share the same prepared request, including selected native history and resolved thinking settings. This applies to preflight, compaction candidates, and hidden summaries. Remote counts have a 30-second deadline, bounded response bodies, and cancellation support.

Local counts include native plaintext reasoning, with image and opaque payload bytes redacted at their protocol fields. Opaque reasoning gets a conservative allowance from the original measured completion or submitted output ceiling, once per selected response. Missing evidence reserves a full context window; ciphertext length is never treated as a hidden-token measurement. These allowances have low confidence but still prevent an oversized request. A successful remote count replaces the allowance. Ordinary over-budget rejection uses high-confidence estimates. See Models.

Thresholds use the input window left after reserving response headroom. Outside SMOL, the reserve is 16k tokens on windows of at least 64k and scales down to 25% on smaller windows (for example, a 12k window reserves 3k rather than 8k):

Trigger Default Smol
Continuous GC reclaim 75% 35%
Automatic compaction 95% 50%
Compaction target 50% 35%

Continuous GC

Every turn, existing stubs are re-asserted and pinned rows restored — byte-identically, so the provider prompt cache stays warm. Under pressure (the GC trigger), a reclaim pass introduces new stubs for:

Every rewrite must pay for itself: new stubs require ≥8,000 tokens or ≥8% saved (rewriting history invalidates the provider cache, so small savings are worse than nothing). Stubbed originals are restorable from /ctx.

Read reuse. A repeated read of unchanged content collapses to a compact pointer ([reused previous read]) instead of resending bytes — but an explicit re-request after that returns real bytes; reads are annotated, never withheld. Guards watch for pathological loops: repeated identical inspections, read storms on one target, repeated failed calls, and implementation stalls. Read-only loop guards reject the offending call and admit one explicit retry with a fresh budget; they do not terminate an otherwise healthy autonomous run.

Compaction

When the automatic trigger fires (or you run /compact), a graduated draft is built:

Always protected: the system message, the last three message groups, the two latest human user messages, and read-reuse pointer targets. Recalled memory is named background data and cannot replace a human request in this protection or anchor an episode boundary. Summaries carry a trust guard marking them lossy reference data. Every compaction records a telemetry event (tokens before/after, prefix hashes, cache impact) visible in /ctx.

Hidden summary generation has a separate bounded output allowance. Before a rewrite is committed, the actual candidate must save tokens and fit the input window after reserving response headroom. The default policy can replace an oversized generated summary with the deterministic fallback. A target below the protected content is best-effort; a candidate that still exceeds the hard window is rejected with guidance to inspect /ctx or use a larger window. Repeated compaction updates the live summary and new omissions; the complete originals are kept separately for restoration.

/compact [preview] [target=<tokens>] [fallback|provider|deterministic] runs or previews one manually.

Task episodes

Enabled by default; BONSAI_EPISODES=0 is the emergency opt-out (zero tracking, byte-identical disabled tool arrays, no persistence).

Smol mode

/smol (or the Context row in /settings) switches to a compact profile for small models: lower compaction thresholds and reserve, a reduced system prompt (bash/terminal/todowrite/set_session_title tools), no repo map or built-in skill/subagent/memory indices, 2 KiB steering cap, deterministic compaction summaries only, and memory recall off.

/ctx

The context modal breaks the window down node by node with three views (ledger / wire / turns): token counts, inclusion state, stub reasons, compaction events, and episode reports, reconciled against actual provider usage. Keys in the ledger view: p pin (never GC), d drop, s stub, r restore, Enter/arrows expand. The header’s context chip (fill bar + percent) is the always-visible summary; click it to open /ctx.

Native reasoning and local opaque allowances have separate ledger entries on their source assistant row. A framing/tokenizer delta explains differences between local row allocations and the prepared request total. A previous count is reused only when the actual request and native budget evidence match.

Dropping a tool row removes its complete call/result exchange for the next request, including reuse pointers that depend on those bytes. The stored transcript remains intact. Use s to shorten just one tool output while keeping the exchange in context.

Where this lives in the code

Concern Location
Preflight, thresholds, compaction src/agent/compaction/ (mod, draft, gc, summary)
Read evidence & reuse src/agent/read_persistence.rs, src/tool/read_evidence.rs
Episodes src/episode.rs, src/agent/episodes/
Project context block src/context.rs
Estimators src/provider/token.rs
Prepared request and native cost evidence src/provider/prepared.rs, src/provider/native_replay.rs
Smol profile src/smol.rs, src/agent/controls/smol.rs
/ctx report & modal src/context_view/, src/tui/widgets/modal/context.rs