5 Backtesting Mistakes That Make Strategies Look Profitable

A backtest is a historical simulation with your fingerprints on it. If the equity curve looks like a product screenshot, assume you helped. The job is not to find a strategy that would have worked. The job is to find the ways your test is lying, then see what remains.

This is not a list of platforms. If you need a desk, use Algo & Automation and the backtesting collection. QuantConnect and NautilusTrader are the two names that come up when people are serious about research versus a replay slider. This article is about the mistakes that survive any of those tools.

1. Testing the survivors

If your universe is "the S&P as it looks today," you already deleted the companies that died, got acquired, or left the index. The backtest never had to survive those names. Live trading will.

Same problem in crypto (delisted alts), FX (pairs your broker stops quoting), and futures (contracts you would not have rolled the way the continuous series did).

Fix: use a point-in-time universe, or at least admit the bias in the write-up. If the platform cannot do point-in-time, your Sharpe is a souvenir.

On QuantConnect, this is a data problem before it is a code problem. On a homemade CSV, it is usually an ego problem. You downloaded what was easy.

2. Forgetting the bill

A strategy that "makes 40% a year" on close-to-close with zero commissions is a blog post. Fill in:

• Commission per side
• Spread or half-spread, not a fantasy tick
• Slippage that gets worse as size goes up
• Borrow fees if you short
• Overnight financing if you are on FX or CFDs
• Data and platform costs, because those come out of the same account

If you cannot name the broker you would actually use, you cannot name the costs. Start from a real ticket: Interactive Brokers or whatever you already clear through in the broker catalog. Then rerun the test. A lot of "edges" are a round-trip you never paid for.

Intraday mean reversion is where this shows up first. The curve is a commission harvest.

3. Letting the future leak in

Look-ahead is not always a `shift(-1)` bug. It is also:

• Using the daily high in a signal you would have had to trade at the open
• Building features from a file that was revised after the date
• Picking the "best" parameter after you saw the whole decade
• Filtering the universe with a metric that was not published yet

Walk-forward is the minimum adult behavior: train on a window, trade the next window, roll. If you only have one beautiful in-sample period, you have a curve fit.

NautilusTrader and LEAN-style engines make it harder to accidentally peek, not impossible. A Jupyter notebook makes it easy. If your research life is notebooks, write the date of every dataset in the first cell and treat a "quick filter" as a leak until proven otherwise.

Related: if you optimize on Sharpe and then report Sharpe, you reported the objective you hunted. Report a metric you did not tune.

4. A stop that only exists in the test

Stops that fill at the exact level, every time, through a gap and a thin book, are fiction. So are profit targets that never become market orders when the level is an iceberg.

Model the stop as it would live on the ticket: stop, stop-limit, or mental. Mental stops in a backtest are just another parameter. If you would not leave the stop with the broker overnight, do not give the test that fill.

This is why execution and research have to share a clock. If you analyze on TradingView and execute elsewhere, the bar you thought you exited is not the fill you will get. The stack article — How to Build a Trading Stack — is about that seam.

5. Calling a lucky path a strategy

One equity curve is a path. Change the start date by a month. Change the first trade. Change the cost model by half a tick. If the story dies, it was a path.

Minimum checks before you paper-trade:

• Same rules, shifted windows
• Same rules, all traded names, not the three that worked
• Same rules, costs stressed 1.5–2×
• A written invalidation: what live result shuts it down

If you cannot state the invalidation, you will "give it more time" until the account does it for you.

For discretionary traders who think this article is for quants: your replay in TradingView has the same five bugs. You just call them "I would have seen that." You would not have.

What to do instead of another optimize pass

Write the hypothesis in one paragraph. Name the market, the hold time, and the reason the edge should exist after costs. Then pick an engine that can say no.

• Cloud, multi-asset, Python/C#: QuantConnect
• Local, event-driven, Python/Rust: NautilusTrader
• Broker-native: TradeStation if you already live in EasyLanguage
• No-code experiments: Composer — useful for allocation rules, not for pretending you have a HFT book

The backtesting collection is the short list. Algo & Automation is the aisle.

When the test is no longer embarrassing, paper it on the same broker you will pay. Then journal the live drift. A strategy that only exists as a tearsheet is not in production.

A one-page test card

Before you share a curve with anyone — including yourself next month — fill this in:

• Market and session
• Universe rule (and whether it is point-in-time)
• Entry, exit, stop as they would sit on a ticket
• Cost model and the broker you stole it from
• Train window / test window
• Metric you did not optimize
• Live invalidation number

If a line is "TBD," the backtest is a draft. Drafts do not get sized. They get another weekend or they get deleted.

Keep the card next to the code. When you change a parameter, change the card. A repo full of notebooks with no card is how "the old version was better" becomes folklore you cannot replay.

Version the card with the code. A git tag without the cost model is how two of you argue about a Sharpe neither can reproduce. If you work alone, you will still argue with last-month you. Write it down.

FAQ

How long should a backtest be?

Long enough to include the regimes that can kill the idea — not "as much data as I could download." A trend system that never saw a two-year range is a tourist. A mean-reversion system that never saw a 2020-style gap is a tourist.

Is walk-forward enough?

It is the floor. It does not fix survivorship or costs. It only stops you from calling one optimized decade a strategy.

Can I trust TradingView strategy tester numbers?

You can trust them as a sketch. You cannot trust them as a brokerage statement. Replay plus commissions you typed in by hand is still a sketch. Move serious work to QuantConnect or NautilusTrader before you size up.

What if my edge is discretionary?

Then your "backtest" is a journal with rules you can grade. That is How to Set Up a Trading Journal You Will Actually Use, not a hidden optimize loop in Pine.