Token Wiki

A reference on token optimization techniques used in production LLM applications. Based on analysis of four systems: Claude Code, Cline, Codex, and OpenCode.

Key Findings

• Four independent teams converged on the same 5-layer architecture: modular prompt assembly, multi-layer output truncation, message normalization, threshold-based compaction, and cache coordination.
• All four systems estimate tokens with length / 4. No tokenizer. It's consistently accurate enough for threshold decisions at zero cost.
• Tool results — not model output — consume ~45% of context in agentic systems. A single file read is 5-10k tokens.
• Prompt caching reduces cost by ~87% ($0.06/turn vs $0.45/turn on a 150k conversation), but any prefix mutation (timestamps, dynamic counters) invalidates the entire cache.
• Models advertise 128k output limits. 99% of real responses use ~5k. Reserving the full limit wastes 123k tokens of usable context per request.

Quickstart

QUICKSTART.md contains copy-paste prompts you can feed to any AI coding agent to audit your codebase for token waste and apply fixes. No background reading required.

Contents

Architecture

Every system studied implements the same defense-in-depth model:

1. PREVENT    Tool-specific limits at the source
2. TRUNCATE   Generic caps catch what slips through
3. CACHE      Prompt caching cuts repeat costs by 10x
4. PRUNE      Clear stale results before expensive operations
5. COMPACT    LLM summarization as last resort

Reference Numbers

Sources

Based on token management implementations in:

• Claude Code — 3-layer compaction architecture with prompt cache coordination
• Cline — Provider-agnostic token management with variant-based prompt optimization
• Codex — Multi-layer tool truncation with fork-based subagent isolation
• OpenCode — 5-pillar defense-in-depth approach with 20+ provider support