The Static vs Dynamic Context Trade-off
Static Context
Static context is always included in the prompt: system instructions, tool definitions, and critical rules. Static context consumes tokens regardless of task relevance. As agents accumulate more capabilities (tools, skills, instructions), static context grows and crowds out space for dynamic information.
Dynamic Context Discovery
Dynamic context is loaded on-demand when relevant to the current task. The agent receives minimal static pointers (names, descriptions, file paths) and uses search tools to load full content when needed.
Dynamic discovery is more token-efficient because only necessary data enters the context window. It can also improve response quality by reducing potentially confusing or contradictory information.
The trade-off: dynamic discovery requires the model to correctly identify when to load additional context. This works well with current frontier models but may fail with less capable models that do not recognize when they need more information.
Pattern 1: Filesystem as Scratch Pad
The Problem
Tool calls can return massive outputs. A web search may return 10k tokens of raw content. A database query may return hundreds of rows. If this content enters the message history, it remains for the entire conversation, inflating token costs and potentially degrading attention to more relevant information.
The Solution
Write large tool outputs to files instead of returning them directly to the context. The agent then uses targeted retrieval (grep, line-specific reads) to extract only the relevant portions.
Implementation
```python
def handle_tool_output(output: str, threshold: int = 2000) -> str:
if len(output) < threshold:
return output
# Write to scratch pad
file_path = f"scratch/{tool_name}_{timestamp}.txt"
write_file(file_path, output)
# Return reference instead of content
key_summary = extract_summary(output, max_tokens=200)
return f"[Output written to {file_path}. Summary: {key_summary}]"
```
The agent can then use grep to search for specific patterns or read_file with line ranges to retrieve targeted sections.
Benefits
- Reduces token accumulation over long conversations
- Preserves full output for later reference
- Enables targeted retrieval instead of carrying everything
Pattern 2: Plan Persistence
The Problem
Long-horizon tasks require agents to make plans and follow them. But as conversations extend, plans can fall out of attention or be lost to summarization. The agent loses track of what it was supposed to do.
The Solution
Write plans to the filesystem. The agent can re-read its plan at any point, reminding itself of the current objective and progress. This is sometimes called "manipulating attention through recitation."
Implementation
Store plans in structured format:
```yaml
# scratch/current_plan.yaml
objective: "Refactor authentication module"
status: in_progress
steps:
- id: 1
description: "Audit current auth endpoints"
status: completed
- id: 2
description: "Design new token validation flow"
status: in_progress
- id: 3
description: "Implement and test changes"
status: pending
```
The agent reads this file at the start of each turn or when it needs to re-orient.
Pattern 3: Sub-Agent Communication via Filesystem
The Problem
In multi-agent systems, sub-agents typically report findings to a coordinator agent through message passing. This creates a "game of telephone" where information degrades through summarization at each hop.
The Solution
Sub-agents write their findings directly to the filesystem. The coordinator reads these files directly, bypassing intermediate message passing. This preserves fidelity and reduces context accumulation in the coordinator.
Implementation
```
workspace/
agents/
research_agent/
findings.md # Research agent writes here
sources.jsonl # Source tracking
code_agent/
changes.md # Code agent writes here
test_results.txt # Test output
coordinator/
synthesis.md # Coordinator reads agent outputs, writes synthesis
```
Each agent operates in relative isolation but shares state through the filesystem.
Pattern 4: Dynamic Skill Loading
The Problem
Agents may have many skills or instruction sets, but most are irrelevant to any given task. Stuffing all instructions into the system prompt wastes tokens and can confuse the model with contradictory or irrelevant guidance.
The Solution
Store skills as files. Include only skill names and brief descriptions in static context. The agent uses search tools to load relevant skill content when the task requires it.
Implementation
Static context includes:
```
Available skills (load with read_file when relevant):
- database-optimization: Query tuning and indexing strategies
- api-design: REST/GraphQL best practices
- testing-strategies: Unit, integration, and e2e testing patterns
```
Agent loads skills/database-optimization/SKILL.md only when working on database tasks.
Pattern 5: Terminal and Log Persistence
The Problem
Terminal output from long-running processes accumulates rapidly. Copying and pasting output into agent input is manual and inefficient.
The Solution
Sync terminal output to files automatically. The agent can then grep for relevant sections (error messages, specific commands) without loading entire terminal histories.
Implementation
Terminal sessions are persisted as files:
```
terminals/
1.txt # Terminal session 1 output
2.txt # Terminal session 2 output
```
Agents query with targeted grep:
```bash
grep -A 5 "error" terminals/1.txt
```
Pattern 6: Learning Through Self-Modification
The Problem
Agents often lack context that users provide implicitly or explicitly during interactions. Traditionally, this requires manual system prompt updates between sessions.
The Solution
Agents write learned information to their own instruction files. Subsequent sessions load these files, incorporating learned context automatically.
Implementation
After user provides preference:
```python
def remember_preference(key: str, value: str):
preferences_file = "agent/user_preferences.yaml"
prefs = load_yaml(preferences_file)
prefs[key] = value
write_yaml(preferences_file, prefs)
```
Subsequent sessions include a step to load user preferences if the file exists.
Caution
This pattern is still emerging. Self-modification requires careful guardrails to prevent agents from accumulating incorrect or contradictory instructions over time.
Filesystem Search Techniques
Models are specifically trained to understand filesystem traversal. The combination of ls, glob, grep, and read_file with line ranges provides powerful context discovery:
ls / list_dir: Discover directory structureglob: Find files matching patterns (e.g., */.py)grep: Search file contents for patterns, returns matching linesread_file with ranges: Read specific line ranges without loading entire files
This combination often outperforms semantic search for technical content (code, API docs) where semantic meaning is sparse but structural patterns are clear.
Semantic search and filesystem search work well together: semantic search for conceptual queries, filesystem search for structural and exact-match queries.
|