AI Agent Capability Resolution: Why I Shipped It Empty

Every AI agent ships with a toolbox, and every toolbox has a hole in it. Sooner or later the agent meets an intent its installed tools cannot complete. What happens next is usually bad: it improvises with the wrong tool, it hallucinates a result, or it quietly gives up. And the most valuable part of the failure evaporates on the spot, because nobody writes down what was missing.

ResolveMesh

is a neutral capability resolver that AI agents call as a fallback: it takes a sanitized intent, something like "EU case law search with citations," and returns ranked, compatible, read-only capabilities across MCP, A2A, and REST.

Your agent connects to the winner directly and then reports back whether the job actually got done. When nothing matches, the miss is recorded as a privacy-safe demand signal. The brand line is "the missing node, found." The engineering line is longer and more honest: and when it is not found, that fact becomes data instead of disappearing.

ResolveMesh ranks capabilities with a deterministic score, not a model call. Semantic intent match carries 40% of the score, live availability 20%, capability freshness 15%, observed outcome success 10%, authentication compatibility 10%, and cost and latency fit the last 5%.

Every response includes the full score breakdown, so an agent builder can see exactly why a capability ranked where it did. And when a capability is too new to have outcome history, its score says INSUFFICIENT_DATA. The resolver does not invent a flattering neutral number for something it has never observed.

I have shipped plenty of systems that lean on models at runtime — Codelit's automatic fallback across AI providers exists precisely because a model call is the least predictable thing in a stack. Ranking is where I wanted the opposite: same input, same output, every time, with the arithmetic printed on the response.

The ResolveMesh V0 spec contains a written stop condition: if precision at three cannot clear 70% without heavy manual curation, the project stops. To keep myself honest there is a benchmark of 150 labeled intents across six capability categories, and it runs as a quality gate in CI — 86% precision at 1 and 89.3% precision at 3 right now.

Writing the stop condition down before writing the code changes how you build. You stop defending the idea and start testing it.

A stratified study of 100 MCP providers found that only 49 completed discovery at all, mostly because the rest require authentication just to enumerate their tools. Those 49 exposed 823 tools, of which only 151 — about 18% — passed a deliberately narrow read-only heuristic.

The supply side is where this project got interesting. The obvious move is to crawl the official MCP Registry and import everything into the catalog. I built that crawler. Then, before letting anything near production, I ran the study to see what was actually out there.

The MCP specification itself is blunt about the metadata: tool annotations are hints from the provider, and clients must not make trust decisions from them.

The ResolveMesh production catalog is empty on purpose: it launched with published precision numbers, a live API, and zero resolution-eligible capabilities, because no discovered tool has cleared the evidence bar yet.

That sounds like a confession. I treat it as the pitch: a resolver you can trust is worth more than a directory you cannot.

What replaces trust-by-default in ResolveMesh is layered evidence with an expiry date, where each layer proves only what it can: a registry record is inventory, nothing more, a tools/list response is triage, nothing more, and attestation, repeated health observation, and human approval sit above both.

Attestation is the layer a provider controls: a small same-origin manifest that binds its claims to the exact digest of its tool definitions, so if a definition changes by one character, the attestation stops matching and eligibility drops instantly. Verified health requires three successful, definition-stable observations spanning at least 24 hours, with the freshest under six hours old.

I piloted this against five real providers, one from each category in the study. Zero had published the attestation manifest, which is expected for a contract that is days old; that number measures adoption, not failure. Eight of ten health observations succeeded.

And one provider that had qualified for the candidate pool minutes earlier failed its first observation, which is the whole argument for continuous evidence: a successful crawl is not availability. A guarded worker now persists twice-daily observations to a private Postgres ledger, and it stays disabled for any provider that has not explicitly opted in.

The most valuable thing ResolveMesh produces is the record of what it could not resolve. Unresolved intents are normalized into demand clusters, the raw intent text is processed transiently and deleted, and only an allowlisted category, constraint tags, and a tenant pseudonym survive.

A gap cluster becomes public only after at least 20 events from five distinct tenants, behind a publication flag that fails closed.

Those are the same rules I apply to DateSafe and anything else I build that touches real people: collect the minimum, delete the rest, fail closed.

Nobody has this dataset. Everyone measures what agents do. Nobody measures what agents reach for and cannot find. If ResolveMesh works, that ledger of misses tells tool builders exactly what to build next, without hoarding a single prompt to get there.

AI has already turned ordinary people into one-person companies — that premise is the entire reason AbleMakers exists — and a one-person company cannot afford to spend a quarter building the wrong tool.

The ResolveMesh V0 spec has a "not doing" list I reread more often than the feature list: the mesh does not proxy execution, holds no provider credentials, retains no prompts or documents or provider output, and runs no marketplace and no paid ranking.

Requests and results flow between your agent and the provider, never through the mesh. Provider payments never touch organic order. Each refusal deletes a security boundary I would otherwise have to defend and a category of trust I would otherwise have to earn.

Every node on Codelit's architecture canvas is a component I defined myself, which makes trust cheap. ResolveMesh recommends other people's software to other people's agents, and that is a different problem with a different answer: own less, promise less, verify more.

ResolveMesh is a TypeScript monorepo: Next.js for the site and REST API, PostgreSQL, and Zod contracts shared by the API, the typed SDK, and the MCP adapter, with Stripe billing, digest-only API keys, and idempotent outcome writes.

It is itself an MCP server, so an agent can adopt it by adding one endpoint: resolve_intent and report_outcome live at /mcp. The discovery crawler runs behind DNS-pinned SSRF protection with strict response caps, because a service that fetches URLs on demand is an attack surface first and a feature second.

The ResolveMesh mark is a diamond lattice with one edge deliberately missing, and the only horizontal line in the whole identity is the lit route to the found node. Signal green may appear only where a resolution exists. Amber may appear only for unmet demand. Everything else stays quiet.

If you build agents, resolvemesh.com has the API, the MCP endpoint, and every number with its methodology attached. The catalog will fill at the speed evidence allows, and not one tool faster.