How to spot a fake backtest — or a too-good track record
Someone's showing you a backtest, a signal service's track record, or an equity curve that only climbs — and asking you to trust it, or pay for it. Most impressive-looking results are fitted, not real, and the tells are consistent. Here's what to check before you believe a number, and each red flag is just the inverse of something an honest test does on purpose.
The returns are gross — no fees, no spread, no slippage on the fills. Frictionless trading doesn't exist, and costs sink a huge share of active rules.
The honest version: A real test charges fees and slippage on every fill. If the seller can't show the after-costs number, assume it's negative.
It shines on exactly one ticker over exactly one date range. Pick the asset and the period after you know the rule worked there, and anything looks brilliant.
The honest version: Judge a rule on every asset and the whole history — the whole glass, winners and losers — not the single chart chosen to sell it.
The rule was tuned and shown on the same data. Of course it fits — it was built to. That says nothing about data it has never seen.
The honest version: Split the history: fit on part, test on the rest (walk-forward). An edge that survives out-of-sample is the only kind worth anything.
An equity line that only goes up, or a Sharpe ratio above ~3, is almost never skill — it's an overfit, a look-ahead bug, or hidden leverage.
The honest version: Discount the Sharpe for how many variants were tried (a deflated Sharpe) and expect real drawdowns. Too good to be true usually is.
The universe is today's famous names. The companies that went to zero — the ones a rule most needs to sidestep — aren't in the sample, so every result looks rosier than reality.
The honest version: Say so out loud. Survivorship inflates every backtest; an honest one discloses it instead of quietly benefiting from it.
“90% accuracy” with no payoff ratio and no expectancy. A high win rate is the easiest number to engineer and the easiest to sell.
The honest version: Win rate alone decides nothing — ask for the expectancy. (See: does a high win rate even matter?)
You're shown a screenshot, not a method you can run. If you can't reproduce it on data you choose, it isn't evidence — it's marketing.
The honest version: A trustworthy result is one you can re-run yourself, on your own asset, for free — no sign-up, no black box.
Checking your own idea is a different job
These flags are for vetting a result someone else is selling. If it's your own backtest you want to trust, the question is subtler — a result you built honestly can still be luck, and there are four independent ways to check that. Real signal or just luck? walks through them.
Take the strategy someone's pitching and run it through the same honest test — costs on, no look-ahead, measured against buy & hold, no cherry-picking — and see what's left. Or browse the whole honest scoreboard: every famous strategy on every famous asset, winners and losers.
This is an educational guide to evaluating backtests and performance claims, not an accusation about any specific product or person, and not investment advice or a recommendation to buy or sell anything. Hypothetical / simulated backtest results are not a reliable indicator of future results.