An MCP server asks your agent to trust it. Agent QA checks it instead,
and remembers what it found. Grade any public endpoint on how it handles bad input,
how fast it answers, and whether its tools are clear enough for an AI to use,
then recall that verdict before you trust it.
Proof, not trust.
Evaluate an endpoint
Paste a public MCP endpoint URL. Agent QA opens a real protocol session and runs a
read-only probe, then remembers the verdict so any agent can recall it later. It sends
only input a correct server rejects before doing any work.
or try:
running probe00%
Probe failed
Grade
–
0 / 100
Category scores
Per-tool results · 0 listed
Defect log
No issues found · clean bill of health.
Reputation memory · Walrus
>_
AGENT-QA CONSOLE
read-only MCP probe · verdicts remembered on Walrus · recall before you trust
AGENT·QA DOCS
Proof, not trust
What Agent QA is
A reputation layer for MCP tools. Give it a URL, get a graded report on how the server behaves, and a remembered track record any agent can recall before trusting it.
An MCP server asks your agent to trust it. Agent QA checks it instead, and remembers what it found. It connects the way an agent would, runs five read-only checks, and returns a letter grade with the reasons behind it. Nothing it sends is meant to change state on the server, so it is safe to point at a server you do not own.
"Every listing on OKX.AI has to be tested before it goes live. Agent QA turns that manual step into one call."
Built for the OKX.AI marketplace
The five checks
Each check is scored on its own, then rolled into one grade.
Connection and handshake. Can the server open a session and list its tools at all.
Schema validity. Does every tool declare a valid input schema with its required fields. An agent reads that schema to build its calls.
Malformed input handling. When the input is bad, does the server reject it cleanly instead of crashing or quietly returning a wrong answer.
Latency. How fast the server answers over repeated calls, reported as p50 and p95.
Description quality. Are the tool names and descriptions clear enough for an AI to pick the right tool and fill the right arguments.
Reputation memory
Every grade is remembered as a compact, tamper-evident fact on Walrus, in one shared registry. Before an agent uses an unfamiliar server, it can recall what Agent QA found last time and trust the tool with proof instead of a shrug. A tool graded once is vetted for everyone who asks after. The memory is optional: with no account configured, grading still works and the panel reports that memory is off.
Using the bench
The bench is the box on the home screen. Three steps.
Paste the MCP endpoint URL into the field.
Press Evaluate and watch the probe run.
Read the report when it lands.
Reading the report
Grade
The seal
One letter from A to F, with the overall score out of 100 counting up beside it.
Scores
Category bars
A bar per check. A red bar is the one place a failure shows in colour.
Tools
Per-tool rows
Each tool the server lists, with a pass or fail chip for each check.
Issues
Defect log
The top problems found, worst first, written in plain language.
An endpoint that cannot be reached comes back as an F, because a server you cannot reach is a server you cannot trust.
Command line
The same engine runs from your terminal with the agent-qa command. Pass it the endpoint URL.
evaluate
agent-qa https://your-server.example/mcp
For output a script can parse, add the JSON flag.
json output
agent-qa https://your-server.example/mcp --json
The command exits with code 1 when the endpoint cannot be reached, so it drops straight into a continuous integration check and fails the build on a bad server.
ci check
# fails the pipeline if the server is unreachable
agent-qa https://your-server.example/mcp --json > report.json
HTTP API
The service is a thin wrapper over the engine. One endpoint does the work.
POST /evaluate
# request body
{ "endpoint_url": "https://your-server.example/mcp" }
It returns the full report as JSON: the overall grade and score, a score per category, a per-tool breakdown, and the top issues. A bad request body comes back as 422. An unreachable target still returns 200, with reachable set to false and the grade set to F, because the evaluation itself succeeded.
Other routes
GET /reputation?q=... returns a server's remembered track record, without grading it again. The query is the endpoint URL or a question about it.
GET /health returns a liveness check for hosting and monitoring.
GET / serves this browser interface.
As an MCP server it exposes two tools: evaluate_mcp_endpoint grades a server now, and recall_tool_reputation returns what Agent QA remembers about it, so an agent can check a tool before it trusts it.
FAQ
Is it safe to point at a server I do not own?
It is built to be. Every check is read-only. The malformed input probe sends only input that a correct server rejects before it does any work, and no payload carries a full set of valid-looking values. Tools that take no arguments, and tools the server marks destructive, are skipped rather than probed.
What does an F mean?
A score below 60, or an endpoint that could not be reached. Treat it as not ready to depend on yet.
Where is the reputation kept, and can it be edited?
Each verdict is stored on Walrus through MemWal and sealed, so it is portable across machines and cannot be quietly rewritten. It lives in one shared registry, so a server graded once carries that record for the next agent. The memory is optional: with no account set, Agent QA still grades every endpoint and simply reports that memory is off.
Why grade descriptions at all?
An AI picks a tool and fills its arguments from the name and description. When those are vague, the agent calls the wrong tool or passes the wrong values, even when the server code is perfectly fine.
Which transports does it speak?
It tries Streamable HTTP first, then falls back to HTTP with SSE, using the official MCP SDK so it speaks the real protocol.