7e2cc56a · zzmq.dev@gmail.com/zz
Commit
7e2cc56ad469133477a7f49e22fe66797cc4139c
Make the system prompt byte-identical across agents/repos -- real ~35x speedup Measured Ollama's actual caching behavior directly: a prompt that shares a long prefix with a recent call costs ~0.15s to reprocess; one that doesn't share a prefix at all costs ~5-6s, regardless of OLLAMA_NUM_PARALLEL (tried 2, made no difference for this sequential- interleaving pattern -- parallel slots don't help here). The system prompt had "${agent}"/"${repo}" baked in near the top and again at the end, which meant literally no two calls -- not even two different claimed asks for the same agent on the same repo -- ever shared an exact prefix. Every single turn paid full reprocessing cost. Made the system prompt a plain constant, byte-identical on every call, and moved all agent/repo/ask-specific text into the per-call user message instead (which was always different per call anyway, so this loses nothing). Verified directly with the real prompt text: switching between two different agents now costs 0.145s instead of 5.6s -- roughly a 35x improvement on the part of each turn that was previously wasted entirely on cache misses. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Troy Howard <thoward37@gmail.com> -- 2026-10-10T20:25:15.000Z