Superpowers vs baseline Claude Code on a fixed task set
Protocol pre-registered 2026-07-07. Results publish when all runs complete; the protocol below is immutable as of registration.
Hypothesis
A Claude Code session with the superpowers framework installed will show a lower false-success rate and higher completion on a mixed task set than a stock session, at measurably higher token cost per task.
Method
| Tasks | 10 tasks spanning bug fixes, small features, and refactors, drawn from real repos with test suites, selected before any runs. |
|---|---|
| Conditions | Condition A: stock Claude Code, default configuration. Condition B: identical except superpowers installed. Same model, same repos, same prompts. |
| Runs | N=5 per task per condition (100 runs total). |
| Scoring | Completion (tests/build pass), false-success rate, intervention count, tokens and wall time. Rubric anchors written at registration. |
| Pinned | Model version, superpowers commit hash, task corpus hash, and rubric pinned at registration; published with raw transcripts. |
Results
Empty until runs complete. This section is the only part of the page that will change, and the change will be logged in the changelog.
Limitations
Framework effects confound with prompt-length effects; token-matched controls are out of scope for v1. Single model. Task selection biased toward test-covered work.