September 2026
Hosho reads your prompt text and returns a score, named findings on exact lines, and concrete fixes you can accept or dismiss. Get that structural read while you write (MCP), in a shared pass (Prompt Editor), or on the PR when more people merge prompt files (Git Bot). Keep the registry and eval harness you already trust - this sits beside them.
Same engine, three moments:
If a coding agent is doing setup, point it at the agent guide. Your human still mints a pfr_ API key at prompts.hoshoai.com (Google sign-in) - agents must not invent keys. The co-presence plugin already bundles MCP; installing both registers the same tools twice.
When prompting stays in solo chat windows, the same system prompt gets reinvented weeks apart. Tribal copies drift. Nobody can say which wording is live.
When prompts move into git and env sync, a huge piece gets better: which string is running where, and can you roll back. That discipline is real progress. What still sneaks through - even with clean history - is the text itself. Rules accrete. Edge-case patches stack. Instructions start to conflict or leave the output contract fuzzy. A one-line tweak can flip answers. A model switch can invalidate sections that still "read fine" to a human. Production breaks without an obvious signal.
Many teams then do the right heavy thing: automated tests, golden sets, CI on prompt changes, sometimes across models. That is excellent. Evals prove whether behavior changed. They do not always explain why the prompt text was brittle while someone was editing.
So the gap is not "versioning is useless" or "evals are wrong." The gap is a missing lighter layer: a structural read on the prompt text - conflicts, underspec, overloaded scope, weak output contracts, vague tool descriptions - so people can improve wording before the full suite runs, and so PR reviewers can see prompt risk when more of the org merges prompt files.
Related: The messy problem of prompting · Catch fragile prompts before you ship or switch models
| Layer | Typical job | Hosho's job |
|---|---|---|
| Registry / version control (e.g. Langfuse, PromptLayer, PromptHub) | Store, version, release prompts | Not a registry replacement |
| Eval harness / CI (e.g. Promptfoo, Braintrust, custom golden sets) | Score outputs on fixed cases; gate releases | Not an eval runner |
| Tracing / observability | See production behavior | Not a tracer |
| Prompt diagnosis (Hosho) | Score prompt text, name gaps, suggest fixes | This layer |
If you already version in git or a registry, and you already run evals - keep them. Hosho is the complementary read on why the text is fragile.
A review returns an overall score out of 10 with a verdict, four macro factors (Scope, Structure, Guidance, Coherence) broken into sub-factors, and findings tied to specific lines - each with the problematic text and a suggested fix. In the editor, findings are cards you accept or dismiss; decisions stick on re-review. In MCP, the same report arrives inline in the IDE. On a PR, Git Bot posts one comment with better/worse and before/after snippets.
You still own the frozen cases and the pass rules in your eval harness. Hosho names why the text is fragile. Evals prove whether outputs still pass. Tracing shows what happened after ship.
A worked example - fragile prompt, Hosho findings, and the fix - will land here next. Until then we are not inventing customers or before/afters.
Hosho is not a prompt registry, not an eval runner, and not a production tracer. It does not guarantee quality or promise that regressions will not happen. It does not run your tests or execute your eval suite. The Git Bot does not block merges unless you build that yourself. It does not autonomously rewrite and ship prompts without you.
If a finding looks useful, apply it, then validate like any other change: same inputs, your harness, your judgment.
When you already version prompts - or you already run CI on them - and you still want to know why a change is risky in the text itself, get a structural read first.
Also useful: Docs · Agent guide · How Hosho scores