How to spot a fake backtest — or a too-good track record
Published
A fake or fitted backtest gives itself away through a few consistent tells: returns shown before costs and slippage, a single cherry-picked asset over one flattering window, no out-of-sample test, an equity curve too smooth to be plausible, only surviving names in the sample, and a result nobody can reproduce on data they choose themselves.
An educational guide to the tells that mark a fitted or cherry-picked backtest — not advice, not an accusation about any particular product or person, and not a judgement on any specific strategy.
Someone's showing you a backtest, a signal service's track record, or an equity curve that only climbs — and asking you to trust it, or pay for it. Most impressive-looking results are fitted, not real, and the tells are consistent. Here's what to check before you believe a number, and each red flag is just the inverse of something an honest test does on purpose.
The returns are gross — no fees, no spread, no slippage on the fills. Frictionless trading doesn't exist, and costs sink a huge share of active rules.
The honest version: A real test charges fees and slippage on every fill. If the seller can't show the after-costs number, assume it's negative.
It shines on exactly one ticker over exactly one date range. Pick the asset and the period after you know the rule worked there, and anything looks brilliant.
The honest version: Judge a rule on every asset and the whole history — the whole glass, winners and losers — not the single chart chosen to sell it.
The rule was tuned and shown on the same data. Of course it fits — it was built to. That says nothing about data it has never seen.
The honest version: Split the history: fit on part, test on the rest (walk-forward). An edge that survives out-of-sample is the only kind worth anything.
An equity line that only goes up, or a Sharpe ratio above ~3, is almost never skill — it's an overfit, a look-ahead bug, or hidden leverage.
The honest version: Discount the Sharpe for how many variants were tried (a deflated Sharpe) and expect real drawdowns. Too good to be true usually is.
The universe is today's famous names. The companies that went to zero — the ones a rule most needs to sidestep — aren't in the sample, so every result looks rosier than reality.
The honest version: Say so out loud. Survivorship inflates every backtest; an honest one discloses it instead of quietly benefiting from it.
“90% accuracy” with no payoff ratio and no expectancy. A high win rate is the easiest number to engineer and the easiest to sell.
The honest version: Win rate alone decides nothing — ask for the expectancy. (See: does a high win rate even matter?)
You're shown a screenshot, not a method you can run. If you can't reproduce it on data you choose, it isn't evidence — it's marketing.
The honest version: A trustworthy result is one you can re-run yourself, on your own asset, for free — no sign-up, no black box.
Checking your own idea is a different job
These flags are for vetting a result someone else is selling. If it's your own backtest you want to trust, the question is subtler — a result you built honestly can still be luck, and there are four independent ways to check that. Real signal or just luck? walks through them.
Take the strategy someone's pitching and run it through the same honest test — costs on, no look-ahead, measured against buy & hold, no cherry-picking — and see what's left. Or browse the whole honest scoreboard: every famous strategy on every famous asset, winners and losers.
This is an educational guide to evaluating backtests and performance claims, not an accusation about any specific product or person, and not investment advice or a recommendation to buy or sell anything. Hypothetical / simulated backtest results are not a reliable indicator of future results.
What are the warning signs of a fake backtest?
The recurring tells are gross returns with no fees or slippage, a result shown on one hand-picked asset over one flattering stretch, no out-of-sample or walk-forward split, an improbably smooth equity curve or a Sharpe ratio above about three, a universe made only of names that survived, a lone headline win rate with no expectancy, and a screenshot that can't be re-run on fresh data. Each tell is simply the inverse of something an honest test does deliberately.
Why does a backtest that looks too good to be true usually fall apart?
Because the traits that make it look flawless are usually artifacts, not skill. An equity line that only climbs, or a Sharpe ratio far above what liquid markets tend to hand out, more often signals an overfit, a look-ahead bug, or hidden leverage than a durable edge. The honest counterpart discounts the Sharpe for how many variants were tried and expects real drawdowns along the way.
How can a trading track record be verified independently?
Independent verification means re-running the method, not trusting the picture. A trustworthy result can be reproduced on an asset and a date range chosen by the checker, with costs and slippage charged on every fill and part of the history held back as an out-of-sample test the rule never saw while it was built. A screenshot of an equity curve, by contrast, is marketing, because nothing about it can be re-derived.
What does survivorship bias do to a backtest?
It quietly inflates every number. When the tested universe is only today's famous, still-listed names, the companies that fell to zero — the very ones a rule most needs to dodge — are absent, so results look rosier than the period actually allowed. An honest test says so out loud and discloses the bias rather than quietly benefiting from it.