The Honest Backtest Census: How Often Famous Strategies Beat Buy & Hold — the open dataset, every edition
An open, freely-licensed record of how often famous, named trading strategies actually beat simply buying and holding — every strategy run on every famous asset the same honest way: costs and slippage on, no look-ahead, measured against each asset's own buy & hold. The headline, from the current edition: of 839 judged backtests on 36 US stocks & ETFs, about 86% failed to beat holding. Each edition is frozen, dated, and kept immutable, so a citation always resolves to the exact number it quoted.
As of September 25, 2026, 25 famous, named trading strategies were each backtested on 36 liquid US stocks & ETFs — 900 strategy-asset combinations, of which 839 had enough trades to judge (61 were too thin and excluded). Of those judged, about 86% failed to beat simply buying and holding the same asset over the same 5-year window — net of costs and slippage, with no look-ahead. This is a survivorship-biased set of well-known assets, measured over the full period with no out-of-sample split, against each asset's own buy & hold, and is not significance-tested.
To cite the dataset across all editions, use the concept DOI 10.5281/zenodo.22974115 (it always points to the newest version); to cite a specific edition, use that release's own DOI and permalink below.
Full citation (APA & BibTeX), the data download, and every strategy's result — current edition →
Editions
Releases are append-only — a new edition is a new entry, never an edit of an old one. Only the current edition is indexed; earlier ones stay reachable so older citations keep resolving.
What changes between editions
This is the first edition. When the next release lands, this section will show what moved between them — how the headline rate shifted as the window rolled, and which strategies crossed the line between beating and trailing buy & hold. Because the headline is the composition-stable stocks-&-ETFs failure rate rather than a crypto-inflated blend, those shifts reflect real data, not a changing mix of assets.
How it stays honest
- • The headline is the failure rate, not a win rate. It is framed so it can't be re-quoted as an edge, and it is the stocks-&-ETFs cut — crypto beats buy & hold far more often on this small, survivorship-biased set, which is an overfitting artifact, not a demonstrated edge.
- • Aggregates only. Beat and fail rates by strategy and by asset class, plus drawdown percentiles — no per-trade records, equity curves or tuned parameters. The denominator is strategy-asset combinations, with thin ones (too few trades to judge) excluded.
- • Reproducible + reusable. Each edition names the engine commit and the open method that produced it, and the data is free under CC BY 4.0 — study it, re-run any cell in the backtester, or cite it.