Scoring methodology
Every Agentiquette entry is scored 0 to 100 on eight dimensions with published weights: reliability 25%, skill leverage 20%, documentation 15%, maintenance 10%, safety 10%, evaluation readiness 10%, composability 5%, and adoption capped at 5%. Sub-scores come from written rubric anchors plus automated signals; every manual score requires evidence notes.
The formula
Overall = 0.25 x Reliability + 0.20 x SkillLeverage + 0.15 x Documentation
+ 0.10 x Maintenance + 0.10 x Safety + 0.10 x EvalReadiness
+ 0.05 x Composability + 0.05 x AdoptionWhat each dimension measures
| Dimension | Weight | What it measures |
|---|---|---|
| Reliability | 25% | Likelihood of correct completion: trigger precision, workflow completeness, failure handling, verification, boundedness |
| Skill Leverage | 20% | Reliable capability added per unit of adoption effort |
| Documentation | 15% | Clarity, examples, install truthfulness (we run the instructions), completeness |
| Maintenance | 10% | Commit recency, issue responsiveness, release cadence |
| Safety / Governance | 10% | Boundaries, permission awareness, destructive-action handling, audit trail |
| Evaluation Readiness | 10% | Can success be measured: tests, checkable outputs, verification steps |
| Composability | 5% | Plays well with other skills and tools; no monolithic assumptions |
| Adoption | 5% (capped) | Stars, forks, dependents: log-scaled, capped, excluded from tier gates |
Each sub-score is the mean of 3 to 5 dimensions rated 0, 25, 50, 75, or 100 against written anchors. Reliability's dimensions, for example: trigger precision (fires when it should, only when it should), workflow completeness (happy path plus failure paths), failure handling (explicit recovery or abort behavior), verification (checks its own output), and boundedness (cannot run away).
Tiers
| Tier | Overall | Gate conditions |
|---|---|---|
| S | 85+ | Reliability 80+ AND Safety 70+ AND commit within 90 days |
| A | 70 to 84 | Reliability 60+ |
| B | 55 to 69 | none |
| Experimental | any | Under 6 months old: scored but flagged too early to tier |
| Deprecated / Risky | any | Unmaintained over 12 months, known-unsafe, or archived. Kept in the index with a warning; deletion breaks citations |
Worked example: obra/superpowers
| Dimension | Score | Weight | Contribution |
|---|---|---|---|
| Reliability | 84 | 0.25 | 21.00 |
| Skill Leverage | 90 | 0.2 | 18.00 |
| Documentation | 82 | 0.15 | 12.30 |
| Maintenance | 92 | 0.1 | 9.20 |
| Safety / Governance | 74 | 0.1 | 7.40 |
| Evaluation Readiness | 66 | 0.1 | 6.60 |
| Composability | 84 | 0.05 | 4.20 |
| Adoption (capped) | 92 | 0.05 | 4.60 |
| Overall | 83 · A-Tier (misses S: overall under 85) | ||
Process rules
Automated signals (nightly): commit recency, release cadence, issue open/close ratio, parsed booleans (tests, examples, boundaries, scripts), log-scaled stars and forks. These draft Maintenance, Adoption, and parts of Documentation.
Manual scoring (45 to 60 minutes per entry): the reviewer reads the artifact, runs the install instructions, exercises at least one skill end to end, rates each dimension against the anchors, and writes evidence notes. Scores without evidence notes do not publish.
Overrides require a written justification, are flagged publicly as "editorially adjusted," and are logged in the audit history.
Uncertainty: unassessable dimensions score null, never zero; composites renormalize; pages state how many dimensions were assessed. We never guess.
Frequently asked questions
Are GitHub stars a good way to rank agent skills?
No. Stars measure marketing reach and timing, not reliability. Agentiquette caps adoption at 5% of the overall score, log-scales it so 50k versus 5k stars is a small difference, and excludes it from tier gates entirely. A 19-star repo can outrank a 188k-star repo here, and one does.
Can a score be bought?
No score, tier, rank, or editorial verdict has ever been or will ever be for sale. Sponsorship, when it exists, is a separate labeled surface that never touches index order.
What happens when a dimension cannot be assessed?
It scores null, not zero. The composite renormalizes over the assessed weights and the page displays "Partially scored, N of 8 dimensions." We never guess.
How often are scores refreshed?
Automated signals refresh nightly. Editorial scores are re-reviewed every 90 days for S and A tiers, 180 days otherwise, and immediately on major releases or credible reported issues.