One shared daemon that pools your MCP servers, records and replays tool results, caches repeated API calls, and enforces filesystem/terminal policy for any agent framework. ~132 KB install. Python 3.11+ stdlib only.
$ pipx install toolrecall
$ toolrecall setup
View on GitHub →
Zero config mode: every toolrecall command auto-starts the daemon if it isn't running.
ToolRecall replaces N cold Node processes with one warm daemon — and 1 tick instead of 4 for cache hits.
| Feature | What it solves | When you need it |
|---|---|---|
| MCP Multiplexer | One shared, persistent pool of MCP servers across all agent sessions instead of N Node processes per session. 5 Claude Code sessions + 3 Cursor instances = 8x the RAM for the same tools. | You run multiple agents or sessions that need the same MCP servers |
| Replay Mode | Record an agent session's tool results, re-run it deterministically in CI. Agent developers currently cannot write reliable tests — flaky tools and non-deterministic APIs make CI for agents nearly impossible. | You write tests for agent behavior and need reproducible runs |
| Forward API Proxy | Repeated identical LLM calls cost $0 in dev/CI loops. Every byte-identical request returns from local cache — no API call, no token cost. | You iterate on prompts or tools that make repeated API calls |
| Security Gate | A policy layer (path allowlist, terminal allowlist, sensitive-file blocklist) that sits between any agent and the machine. Framework-independent, works even with caching disabled. | You need a guardrail between agents and your filesystem/shell |
| MCP Result Caching | Legitimate for slow, idempotent external calls (search, fetch, docs). | Your MCP servers make expensive or rate-limited external calls |
| File / Terminal Cache | Reduces redundant reads within a turn. Useful when paired with the Context Tracker. | Your agent re-reads files or reruns static commands frequently in the same session |
LLM context windows accumulate everything. Every file read, every command output stays in context forever. The result is predictable: quadratic latency growth and session-killing bloat.
A 10K-token file read 100 times + accumulated tool outputs = ~1M billed input tokens for the same content, session after session. Provider prefix-caching helps only for exact byte-identical prefixes — but every turn shifts the prefix boundary.
500K-token context → 250B attention pairs per turn. Every new turn adds prefill latency — the GPU slows down quadratically as context grows. This is a latency tax on your time, not your wallet.
File read once, then served from SQLite in sub-millisecond. Zero tokens for repeated reads. 73-91% fewer input tokens per session.
The Context Tracker tracks which files the agent has written (dirty) vs. just read (clean). The agent drops clean file content from context every N turns — keeping only what changed. In simulation: 93.3% O(N²) reduction across 1–20 agents.
Y-axis: cumulative context size (tokens). Data from benchmark_on2.py — 1 agent. Without TR (red): context grows ~52K/turn, reaching 1.56M tokens at 30 turns. With TR + Context Tracker (green): context stabilizes at ~268K after 5-turn warmup. Without TR: 67B → 2.42T attention pairs. With TR: capped at 72B. 93.3% O(N²) reduction (mathematical model, 1–20 agents).
| Failure Mode | LLM-Driven Cache | ToolRecall |
|---|---|---|
| Misclassification | LLM guesses STATIC → messages silently dropped | `ttl=0` means NEVER cache. Binary, no AI middleman. |
| Extra cost | Every new tool needs an LLM call to classify | $0 — SQLite, no embeddings, no API calls |
| Cold-start latency | Must analyze metadata before first cache decision | First call executes live, cached on return — zero overhead |
| Side-effect blindness | Relies on tool name/description | mtime-based auto-invalidation — file edited? next read is fresh. |
| Reproducibility | Non-deterministic — may classify same tool differently | Always byte-identical for same args + same mtime. 100% reproducible. |
ToolRecall doesn't replace your tools — it caches their outputs so repeat calls skip the OS entirely. Here's what changes:
Result: ~20 min less waiting per session, deterministic payloads enable up-to-90% provider prefix-caching. ToolRecall cuts OS execution latency by ~1000× on repeat calls and unlocks the up-to-90% provider prefix-caching discount through deterministic byte-identical outputs.
File cache: mtime-based. Every cached_read() checks os.path.getmtime() — if the file changed, it re-reads from disk automatically. No stale data.
Write invalidation: Every write_file/patch through the shim immediately deletes stale cache entries. Next read after a write is always a cache miss.
Terminal/MCP: TTL-based with configurable expiry. Static commands only (8 items in allowlist).
Forward proxy: Request-body hash — a different request always hits the provider.
Stale data cannot persist. File modifications, writes, and TTLs all independently trigger cache refresh.
A deterministic, security-hardened middleware layer for any AI agent — no SDK changes, no plugins, no vendor lock-in.
Files invalidate on mtime, commands expire by TTL, and `ttl=0` always executes live. Zero hallucination risk.
Standard `stdio` MCP. Drop-in with any MCP agent (Claude Code, Cursor, Cline, opencode, Crush).
Unix Domain Sockets only (no TCP). Cryptographic path resolution blocks traversal. Path allowlisting blocks unauthorized file access.
Hybrid LRU + SQLite cache with in-memory hot paths. First byte in ~0.6ms on repeat calls. No network, no GPU.
Pure Python >=3.11. Stdlib only — not even the `mcp` SDK (optional for multiplexer). Install with `pipx install toolrecall`.
Built-in full-text search over docs and notes. SQLite FTS5 with BM25 ranking, Porter stemming. Query in natural language.
Single daemon manages multiple MCP servers with lazy loading. Idle servers shut down after 15 min. Server-side prompt caching.
Dirty/clean checkpoint tracking breaks O(N²) context growth. Agent keeps only dirty files in context, re-reads clean files from cache.
Local dedup: repeat file reads at zero tokens (73-91% fewer). Provider prefix-caching: byte-identical payloads qualify for discount.
Cache full API responses before they leave your machine. The forward proxy starts automatically with the daemon — no extra command needed.
export OPENAI_BASE_URL=http://localhost:8569/v1 — point any OpenAI-compatible SDK at the forward proxy. Cache hits serve response instantly: zero tokens consumed, never reaches the provider.
| Provider | How to connect |
|---|---|
| OpenAI | Set base URL to http://localhost:8569/v1 |
| Anthropic | Set base URL to http://localhost:8569/v1 |
| Google Gemini | Set base URL to http://localhost:8569/v1 |
| DeepSeek, xAI, Mistral, Groq, Together, OpenRouter | Set base URL + X-Target-Host header to the API hostname (these are OpenAI-compatible; path routing can't distinguish them) |
toolrecall serve --port 9090
ToolRecall provides three integration layers. Choose the one that fits your workflow.
tr binary connects directly to the daemon over UDS. tr read, tr term, tr status — from any shell, any language, no Python runtime needed..pth shim patches open(), subprocess.run(), and Popen() for automatic caching — no code changes.# Aider (v0.86.2+)
aider --mcp-toolrecall
# Build Go client from source
cd go-client && go build -o /usr/local/bin/tr .
tr read/tr term/tr status from any shell# Python shim (opt-in)
toolrecall shim --install
ToolRecall's core caching engine is language-agnostic (SQLite + UDS daemon), with three integration layers. Here's the current state:
| Runtime | Agent | Auto Caching | MCP Bridge | Go Client (tr) |
|---|---|---|---|---|
| Python agents | Hermes, Aider, Codex CLI, custom scripts | ✅ OS-level shim | ✅ Works via MCP | ✅ tr from shell |
| Any Python process | Scripts, REPLs, CI runners, background tasks | ✅ .pth patches open() | ✅ Zero config | ✅ tr from shell |
| Node.js / Go | Claude Code, Cursor, opencode, Cline, Crush | ❌ No native shim | ✅ Add toolrecall mcp | ✅ tr any shell, no MCP |
Three integration layers for every use case. Python agents get the transparent shim (toolrecall shim --install). Non-Python agents use the MCP Bridge (one server entry, any agent) or the Go client (tr) (any shell, any language, no MCP). All three share the same daemon cache.
Record live tool outputs once, replay them on subsequent runs — enabling deterministic, offline, zero-cost CI testing. Think VCR.py for agent tool calls, not HTTP requests.
toolrecall replay record my-scenario — run your agent normally. Every tool call is recorded as a (tool_name, args_hash, response) tuple.
toolrecall replay replay my-scenario — matching tool calls are served from SQLite instantly. No real execution, no network, no API costs.
toolrecall replay export my-scenario > scenario.json — portable JSON commit to git and import in CI.
CI/CD agent tests — record in dev, commit JSON, replay in CI. No API keys, no network flakiness.
Debugging — record the exact tool trace that caused a bug, replay to reproduce every time.
Offline development — all tool calls replay from local cache. Work on a plane with zero API calls.
Tool-level only. Replay caches tool outputs — it does not control the LLM. For fully deterministic CI, pin the agent's model and temperature (temperature=0) during replay. Full docs.
Thin, framework-native wrappers that bring ToolRecall caching to popular Python agent frameworks. Each adapter is ~50-80 lines, zero new dependencies, and works with the existing daemon.
@tool functions that make API calls, search queries, or database lookups — code the shim can't see. Just add @cached_tool on top of @tool.set_llm_cache(ToolRecallCache()) — every LLM call checks the local cache first. The ToolRecallCallbackHandler caches tool results by name + args hash.tr binary — any agent in any herdr pane shells out to tr read/tr term/tr status. MCP bridge — add toolrecall mcp as the entry point.Thin wrappers, not deep integrations. They use toolrecall.client — the daemon manages the SQLite cache. All three adapters add ~180 lines, zero new dependencies. Browse →
ToolRecall sits between your agent and the OS. Agents connect through one of several bridges — each speaks a different protocol but all funnel through the same daemon cache over a Unix Domain Socket.
from toolrecall import cached_read. Zero deps, importable by any Python process.tr read/tr term/tr status from any shell.| Aspect | Before (3 processes) | After (1 daemon) |
|---|---|---|
| RAM usage | ~60MB (3× LRU) | ~25MB (1× LRU + bridges) |
| MCP startup | ~200ms per call | ~5ms (Python stdio → socket) |
| Cache sharing | SQLite only (7ms) — each cold | LRU + SQLite — always warm |
| Language binding | Python only | Any language via UDS |
| Fault tolerance | One dies → others live | Daemon dies → systemd restart |
Runs everywhere Python does. No database server, no node_modules, no Docker required.
# Recommended — isolated CLI, always on PATH
pipx install toolrecall
pip install toolrecall · From GitHub: pip install git+https://github.com/whiskybeer/toolrecall.git# Auto-detect & configure agents
toolrecall setup
Detects Hermes, Claude Code, OpenCode/Crush, writes MCP config, and starts the daemon — all in one command.
# Daemon status & control
toolrecall daemon --status
toolrecall daemon --stop| Platform | Transport | Status |
|---|---|---|
| Linux | Unix Domain Sockets | Tested in CI |
| macOS | Unix Domain Sockets | Should work (POSIX). Not in CI. |
| Windows | TCP localhost:8568 fallback | Experimental — not in CI |
Daemon auto-start (fallback chain): 1) systemctl --user start toolrecall-daemon on Linux with systemd, 2) os.fork() + run_daemon() in Docker/Codespaces/macOS, 3) subprocess.DETACHED_PROCESS on Windows. Every toolrecall command auto-starts the daemon if it isn't running.
| Command | What it does |
|---|---|
toolrecall setup | One-shot: config + systemd service + daemon start |
toolrecall init | Create default config.toml and .env |
toolrecall status | Cache status and stats (auto-starts daemon) |
toolrecall stats | Detailed cache statistics as JSON (auto-starts daemon) |
toolrecall invalidate | Clear all caches (auto-starts daemon) |
toolrecall restart | Health check + clean daemon restart (auto-starts daemon) |
toolrecall mcp | Start MCP Bridge (auto-starts daemon) |
toolrecall serve | Forward proxy — cache API responses (auto-starts daemon) |
toolrecall debug | Start debug/demo server (auto-starts daemon) |
toolrecall index | Build/update FTS5 knowledge database (auto-starts daemon) |
toolrecall config-set | Set a config value |
toolrecall daemon | Start/stop/manage cache daemon |
toolrecall shim | Install/uninstall OS-level cache shim (.pth) |
toolrecall nginx | Generate nginx config for reverse proxy |
toolrecall turso | Turso Cloud sync: init, enable, disable, status |
README — start here
How It Works — quick technical overview
CLI Reference — all subcommands
Forward Proxy — provider list, header setup
Normalizer — cache key normalization
Configuration — all env vars and TOML
MCP Multiplexer — server management
Replay Mode — deterministic CI testing
Knowledge DB — FTS5 indexing
Benchmark — measured performance
Real-Agent Debug Loop — edit-heavy session
Architecture Diagrams — system + sequence
Security Architecture — policy gate, trust boundary
Agent Compatibility — per-agent caveats
Troubleshooting — common fixes
libSQL Backend — multi-writer, vector search
Docker Deployment — containerized stack
Appendix — comparison tables, OSI model, ROI
git clone https://github.com/whiskybeer/toolrecall.git
cd toolrecall && make setup
make test (run tests) or make check (lint + format).Deterministic caching, zero dependencies. Works with Python, Go, and any MCP-capable agent. ToolRecall is open source and ready to use.
★ Star on GitHub →You're right — providers bill per token ($/M tokens), not quadratically. O(N²) isn't your bill; it's the latency tax on the GPU prefill phase. Every turn with a growing context takes longer to process. For a 500K-token turn that's an extra 5-15 seconds of wait time per turn. Over 50 turns, that's real friction — not on your invoice, but on your time.
It helps — but only for exact byte-identical prefixes. In agentic workflows, every turn appends new messages and tool outputs, shifting the prefix boundary. A 10K-token file read at turn 5 vs. turn 15 doesn't share a prefix — they're surrounded by different context, so it bills ~100×. ToolRecall's local dedup eliminates those repeated tokens entirely, regardless of provider.
ToolRecall caches file contents, terminal output, MCP responses, and (optionally) LLM API responses in a local SQLite database at ~/.toolrecall/cache.db (permissions: 600). The database is not encrypted — any process under your user account can read it. However, sensitive files (`.env`, `.ssh/*`, credentials, private keys) are blocked from being cached by the built-in blocklist. The cache never leaves your machine. Full SECURITY.md.
The Context Tracker prevents exactly that. It tracks which files the agent has written (dirty) vs. just read (clean). The agent only drops clean files — ones it read but never modified. Dirty files (its edits) stay in context. If a dropped file is needed again, cached_read() returns it from SQLite in ~0.6ms. Confirmed in simulation: 93.3% O(N²) reduction (mathematical model), 277 tests, 0 regressions.
Collisions are rare (each server in ToolRecall's multiplexer has a unique name and tool set) but when they occur — e.g., if both github and git registered list_issues — the first server registered wins. Its tool takes priority and a warning is logged. The second server's conflicting tool is silently skipped.