Office of AI Throughput & Cost Control

Institutional notes on removing avoidable idle time in AI/LLM inference pipelines to reduce cost per successful completion.

Pillar pages

• AI inference cost optimization guide
• ProvenanceOS: persistent browser workspace

Phone terminal funnel

• Phone Terminal Codex
• Join for $37/mo

Office notes

• Tokens per second (throughput vs idle time)
• HTTP connection reuse for LLM inference
• Bounded concurrency limits (in-flight caps)
• Why AI/LLM inference is slow in production
• How a phone terminal survives reconnects
• Terminal session streaming: minimal message shape
• Output buffering and replay for long-running sessions
• Session IDs and admin controls for browser terminals
• Mobile terminal rotation: resize without corruption

Office Briefings (stub)

Office Briefings are paid, evidence-backed decision aids for specific throughput and cost-control questions. They are scoped, non-promissory, and designed to be reviewed by a senior engineer before adoption.

• Office Briefings overview

Scope boundary

This index covers Office materials on throughput and cost control via idle-time elimination. That includes both:

• inference pipeline controls (connections, concurrency, backpressure)
• operator workstation surfaces that reduce recovery friction (persistent sessions, reconnect, replay)

It excludes: training and fine-tuning, prompt/content strategy, vendor pricing claims, and benchmark numbers that are not backed by a reproducible harness and raw measurements.