Backtests lie. Curve-fitting, look-ahead bias, survivorship bias, unrealistic execution assumptions - the ways a backtest can deceive you are well documented.
But if you're serious about systematic trading, you still need to test. The question isn't whether to backtest. It's how to do it without fooling yourself.
Here are the methodologies that actually work.
Start With Logic, Not Data
Before you run a single test, articulate why your strategy should work.
Not "because it worked historically." That's circular. You need a market-based reason: institutional behavior creates this pattern, human psychology produces this tendency, market structure generates this inefficiency.
If you can't explain the logic, you're data mining. Any "edge" you find is probably noise that happened to look like signal.
Good foundations:
- Cycles exist because capital rotates through accumulation and distribution phases
- Volume confirms direction because money flow reveals institutional intention
- Confluence works because independent systems agreeing reduces false signals
Write down your hypothesis before testing. This prevents retrofitting explanations to results.
Build In Realistic Costs
Most backtests assume perfect execution. Real trading has friction.
Include in every test:
- Spread: You buy at ask, sell at bid. Model this explicitly
- Slippage: Assume fills 0.1-0.5% worse than your signal price
- Commissions: Even "free" brokers have SEC fees and payment for order flow costs
- Market impact: Your order moves price, especially in less liquid instruments
A strategy making 0.1% per trade with 0.15% round-trip costs is a loser. Better to know now than after real losses.
The Out-of-Sample Protocol
Here's the most important technique for honest backtesting: out-of-sample testing.
- Divide your data. Split it into in-sample (say, 70%) and out-of-sample (30%)
- Develop on in-sample only. Do all your testing, tweaking, and optimization on the in-sample data
- Test once on out-of-sample. When you have a final system, test it on the out-of-sample data you've never looked at
- One shot. You only get one chance. If you test on out-of-sample, tweak, and test again, it's no longer out-of-sample
If performance degrades significantly on out-of-sample data, you've overfit. If it holds up, you might have something real.
Walk-Forward Analysis
Even better than simple out-of-sample testing is walk-forward analysis:
- Optimize on period 1 (say, 2015-2018)
- Test on period 2 (2019)
- Record the results
- Optimize on period 2 (2016-2019)
- Test on period 3 (2020)
- Repeat
This simulates what you'd actually do: periodically re-optimize your system using recent data, then trade forward.
If the "test" periods consistently perform well, your system is robust. If performance in test periods is much worse than optimization periods, you're curve-fitting.
The Monte Carlo Reality Check
Your backtest shows a 25% maximum drawdown. Comfortable, right?
But that's just one path through history. Random variation could have produced much worse.
Monte Carlo analysis randomizes the order of your trades (or samples with replacement) and runs thousands of simulations. This shows you the range of outcomes that could have occurred with the same trades in different sequences.
If your maximum drawdown could have been 50% with different luck, you need to size for 50% drawdown, not 25%.
The Bottom Line
Backtesting isn't proof of an edge. It's a way to disprove bad ideas - if you're honest about it.
A good backtest:
- Uses simple rules with market-based logic
- Avoids look-ahead and survivorship bias
- Includes realistic transaction costs
- Reserves out-of-sample data for final validation
- Stress-tests with walk-forward and Monte Carlo analysis
The goal isn't a beautiful equity curve. It's a realistic estimate of forward performance. Those are very different things.
Non-repainting indicators are essential for honest backtesting. If the system you're testing uses cycle analysis, volume regime detection, and momentum voting that all finalize on candle close, your historical review shows exactly what you would have seen live. No look-ahead bias. No phantom signals. What you're testing is what you'd actually trade.
Pentarch's non-repainting signals are designed for honest backtesting - what you see in history is exactly what you would have seen live, with no look-ahead bias.
Test with confidence →