Methodology rung1.v1
How we score an MCP server
A score you cannot check is a score you cannot trust. So here is the whole method, in plain language. Every retained number on MCP Verdict came from these rules, applied to evaluations that installed and exercised the server. The score came from behavior, not from a manifest.
The grade
Each entry gets a composite from 0 to 100 and a letter grade derived from it. The letter and the number always travel together.
The composite
The composite is a weighted blend of four sub-scores. Functionality and reliability dominate, because a registry first has to find what actually works. Security is heavy. Latency matters least, since a slower correct tool can still be the right call.
| Sub-score | Weight |
|---|---|
| Functional | 35% |
| Reliability | 30% |
| Latency | 15% |
| Security | 20% |
The four sub-scores
Functional
Does the tool do what its entry claims? A pass scores 100, a partial scores 60, a fail scores 0. We average across every evaluation.
Reliability
Repeated success over many attempts, recorded as a 0 to 100 score per run and averaged. A run with no reliability measure defaults to 50, so an incomplete test never quietly looks perfect.
Latency
Measured response time, scored on a curve. At or under 250ms scores 100. From 250ms to 1000ms it falls from 100 to 70. From 1000ms to 3000ms it falls from 70 to 35. Past 3000ms it keeps falling toward 0. A missing measure defaults to 50.
Security
Each evaluation starts at 100 and loses points for every flag, by severity. The result is clamped to a 0 to 100 range and averaged.
- infominus 2
- lowminus 8
- mediumminus 25
- highminus 60
One rule overrides the math. An unresolved high-severity flag, meaning a tool can leak credentials or data, run arbitrary code, or reach outside its declared scope, caps the composite at 40 no matter what the weighted score would have been, and the entry shows a warning. Resolved flags stop penalizing. Medium flags lower the security sub-score but do not cap the composite.
Automated scanners are one input here, not the whole of it. We draw on the open-source tools in this space (mcp-scan from Invariant Labs, Cisco's MCP Scanner) and add a human review on top, because static analysis alone is uneven. The number gets the machine pass. The verdict gets the human.
Confidence
A score is only as trustworthy as the testing behind it. Confidence runs from 0 to 1 and rises with the number of evaluations: it starts at 0.50 for a single run and climbs to 0.95. A great-looking score with one test behind it shows high and low at once, on purpose. The planned cross-community confidence model was never activated.
What we refuse to use
The method is as much about the inputs we reject as the ones we keep. The score does not use any of these.
- Vendor or model-provider identity. A tool's maker never moves its score.
- Marketplace popularity, stars, or download counts. Attention is not quality.
- Sponsorship or payment. Placement is not for sale at any rung.
- Raw community votes. Community scoring was designed but never activated, so no public score includes it.
The score, by API
The stable, machine-readable view of any retained entry's score lives at a single endpoint. It reports the historical score and methodology version; it is not a live certification signal.
GET /api/scores/{id}
{
"entry_id": "ddfe0583-e4ef-5e8c-bd05-23b3a2e5242f",
"composite": 98.1,
"sub_scores": {
"functional": 100,
"reliability": 95,
"latency": 100,
"security": 98
},
"confidence": 0.5,
"methodology_version": "rung1.v1",
"last_computed_at": "2026-06-07T00:00:00Z"
}Negative findings are not published. The registry is limited to eight passing entries retained from the initial filesystem review. Read the publication status.
This methodology documents how the retained scores were calculated. MCP Verdict is not accepting submissions or operating a re-test schedule.