Two ways to make code retrieval cheaper
There are two complementary approaches:
| Approach | Best for | How it reduces token consumption |
|---|
| Language server (LSP) | Exact symbol resolution | Uses type-aware operations such as go-to-definition, find-references, call hierarchy, rename, and diagnostics instead of making the model inspect search results. |
| Code graph | Structural and architectural traversal | Uses a pre-built, often AST-derived index to answer questions LSP is not shaped for: what depends on what, how does this flow, what would this change touch and returns citable file-and-line references. |
The rule of thumb: use a language server when you need precision about a symbol; use a code graph when you need the shape of the system. Most AI workloads benefit from both.
Tools that target different sources of token waste
These techniques extend beyond retrieval. Different parts of the workflow can be optimized with different instruments:
| Cost lever | Instrument | What it does | Evidence |
|---|
| Symbol resolution | agent-lsp (repo) | MCP server orchestrating real language servers for agents; keeps the workspace index warm | Publishes a reproducible benchmark — methodology, five named OSS codebases (15K–319K lines), and a go run command to re-measure on your own repo |
| Structural graph | Graphify (site) | Local tree-sitter parse into a queryable knowledge graph with file:line citations. Apache 2.0, no embeddings, no API keys, runs on-device | Open source; deterministic AST extraction — inspect the parse yourself rather than trusting a claim |
| Structural graph | codegraph (repo) | Local code knowledge graph consulted as an MCP server before an agent edits | Open source |
| Context | headroom (repo) | Compresses tool output, logs and retrieved chunks before they reach the model | Open source |
| Dev loop | RTK (repo) | CLI proxy trimming verbose dev-command output — git, tests, builds — before it enters context | Open source |
| All · Metering | codemie-code (repo) | Runs multiple coding agents under one command, emitting one usage event per invocation so spend is attributable per team, model and run | Open source |
A note on the savings claims: tools in this category often publish large percentage reductions, but those numbers are typically measured on vendor-selected workloads. Treat them as benchmarks, not forecasts for your codebase.
Independent research provides stronger evidence for the underlying approach. Code Isn't Memory compared structural indexing with both no-index and agentic-grep baselines across SWE-PolyBench Verified and SWE-bench Pro. It found better localization and resolution performance alongside lower cost per solve.
The important point is not the percentage saved on someone else's benchmark. It is whether retrieval reduces unnecessary consumption on your repository. You can test that quickly:
- Pick a commonly used symbol.
- Search for it with grep and count the matches.
- Ask a language server for actual references.
- Compare how much information the agent would need to read in each case.
That ratio gives you a much more useful starting point than a vendor savings claim.
Disclosure: codemie-code is built and maintained by my team and released open source. It is included because it is the metering layer we use, not because the framework depends on it. Any system that emits structured usage events per model call can serve the same purpose, including a gateway, observability layer, OpenTelemetry pipeline, or custom wrapper.
How to build an AI spend management framework?
The temptation is to start with an AI cost dashboard. But you cannot measure or price chaos. AI spend management needs three steps, in this order:
| Step | What it means | What breaks if you skip it |
|---|
| Define the process | Run delivery through explicit AI workflow phases and gates rather than ad-hoc prompting. | There is no repeatable unit to measure. Every run is a different shape, so every number is noise |
| Measure the process | Attribute every model call to the task, phase, model, owner, tokens, and cost. | You optimize anecdotes while maintaining AI adoption levels. The loudest cost story wins, not the biggest one |
| Economics on top | Forecast before the run, prove after, cap with guardrails | Your AI budget remains a top-down estimate rather than a defensible operating model. |
A dashboard bolted onto improvised prompting produces beautifully rendered noise.
There is one qualification: instrument early. If you have no visibility today, start metering calls immediately. Measurement will show you where the process is broken. But it cannot establish a defensible reference point for what a unit of work should cost until that work has a repeatable shape.
Put differently:
- Measurement finds the mess.
- Process creates the baseline.
- Economics makes it predictable.
A defined pipeline, with explicit phases and gates, makes engineering work countable. Without that repeatability, tokenomics has nothing consistent to measure or price.
How to forecast, measure, and control AI costs using tokenomics?
Once the process is repeatable, AI cost management becomes a closed loop: forecast before the run, measure after it, and control the variance between the two.
1. Forecast AI costs before the run
Before an agent starts, estimate what the task should cost based on its size, codebase, and verification depth. Break that estimate down by phase: discovery, specification, implementation, review, and verification.
The FinOps Foundation has run a working group on AI cost estimation since late 2025, yet roughly 80% of enterprises still miss AI cost forecasts by more than 25%. The problem is less agreement on the need for forecasting than the ability to do it reliably.
Two rules make forecasts more defensible:
- Give a range, not a point estimate: Agent benchmarks show cost per trial varying by an order of magnitude across configurations, without a strong relationship between higher spend and successful outcomes. A precise per-task figure creates certainty the underlying system does not have.
- Calibrate against your own codebase: A "medium" task in a clean, well-tested repository can become an "extra-large" task in a legacy monolith with poor test coverage. Use completed tickets from the actual backlog to establish your baseline.
The unresolved problem: normalizing task complexity
Cost per unit of work is useful only when the units are comparable.
A one-shot summarization and a multi-turn agentic refactor can both consume tokens, but they are not equivalent units of engineering work. T-shirt sizing helps. Calibrating those sizes against your own backlog helps more.
What does not exist today is a validated way to normalize task complexity across fundamentally different codebases.
So start narrower:
- Compare like-for-like work within the same codebase.
- Track cost trends by task size.
- Recalibrate when the codebase changes materially.
That produces a useful internal baseline without pretending that a "medium ticket" means the same thing everywhere.
2. Measure AI costs after the run
Forecasting establishes what a task should cost. Measurement tells you what actually happened. A monthly figure such as "we spent $12,000 on AI" is too aggregated to optimize. Instead, attribute spend to the ticket, branch, workflow phase, and model. If analysis costs ten times more than implementation, you immediately know where to investigate. A monthly total cannot tell you that.
Also record how each decision was resolved: deterministic rule, fast path, or model call. That makes zero-token decisions measurable rather than anecdotal.
The dashboard then needs only metrics that lead to action:
| Metric | What it tells you | Why it matters |
|---|
| Cost per unit of work | What does a task of this size cost? | Establishes a usable cost baseline. |
| Cost per phase | Where is spend accumulating? | Points directly to optimization opportunities. |
| Forecast vs. actual variance | How accurate are our estimates? | Measures cost predictability over time. |
| Cache hit ratio | Are we repeatedly paying for stable context? | Exposes avoidable input cost. |
| Model mix | Are expensive models handling routine work? | Tests model-routing discipline. |
| Decision resolution source | Which decisions actually require an LLM? | Makes zero-token savings auditable. |
| Cost per merged change | What are we paying for completed outcomes? | Excludes expensive work that never ships. |