How Compaction Works in Pi

What people zeroed in on was the cost of doing this in real systems. The big constraint is prompt caching. Every time you rewrite history, you usually blow away the cached prefix and pay to reprocess the whole conversation. That makes many “smarter” schemes look worse in production than they do on paper. Progressive pruning, replacing tool output with pointers, or compacting every turn can all preserve more useful context, but they can also turn into a cache miss factory. That is why several comments pushed a narrower goal than “summarize the conversation well.” They want to surgically remove junk like MCP tool logs, test output, and exploratory dead ends while leaving the human intent and the working plan intact. Pi apparently already exposes hooks for this via extensions, and some people argued subagents are a cleaner version of the same idea. Others went further and said the best local-model strategy is often to avoid giant contexts entirely, keep sessions tight, and branch or restart instead of trying to carry everything forward. A second theme was that local deployments change the economics. On your own stack you can pause inference, rebuild the KV cache, and even juggle dual caches to hide some summarization latency, but compaction is still expensive enough that many people simply start a fresh session when they hit 128k. The consensus landing point was blunt: compaction works, but it is a lossy workaround for finite context and current caching behavior. The winning designs are the ones that isolate noise, minimize rewrites, and make cache breakage rare.