01 / a local cache for AI agent tool calls

ToolRecall — Deterministic Tool Cache

Repeat file reads and commands come back in under a millisecond — and stop bloating your agent's context.

✓ installed — zero dependencies, pure Python stdlib
02 / the problem

The Context Snowball

LLM context accumulates everything. Every file read and command output stays in context forever — even exact duplicates.

session context 0 tokens
Red blocks are repeat reads of the same file. You pay for them linearly — and the GPU pays O(N²) attention on every turn. That's a latency tax on your time.
03 / how it works

Miss once. Hit forever.

ToolRecall sits between your agent and the OS. First call executes live and is cached — every identical call after skips the OS entirely.

04 / context tracker

Breaking O(N²) growth

The tracker knows which files the agent wrote (dirty) vs. only read (clean). Clean content is dropped from context — and re-served from cache in ~0.6ms if needed.

main.py · dirty utils.py · clean config.json · clean api.py · dirty README.md · clean
Without TR — grows ~52K/turn With TR + Context Tracker — bounded
1.56M 500K 0 0 15 turns 30 1.56M · 2.42T attn pairs ~268K flat · −93.3% O(N²)
05 / why not an LLM-powered cache?

No LLM in the caching loop

LLM-driven cache
ToolRecall
Guesses what's static — messages silently dropped
ttl=0 means never cache. Binary, no AI middleman
Extra LLM call to classify every new tool
$0 — SQLite. No embeddings, no API calls
Blind to side effects; may misjudge freshness
mtime invalidation — file edited? next read is fresh
Non-deterministic — same tool, different verdicts
Byte-identical for same args + mtime. 100% reproducible
06 / results — 13h real session

Ready to cut your token costs?

204K
file-read tokens
before (13h session)
0
after — 73%
fewer, same session
0%
overall cache
hit rate (91% file)
~0.6ms
repeat reads —
~1000× faster
up to 90%
provider prefix-cache
discount unlocked
0
dependencies —
stdlib only
$ pipx install toolrecall
$ toolrecall setup
★ Star on GitHub →
← → space · click a segment to jump