We find weak spots and test improvements, then check every change on data it wasn't tuned on, so you get real gains and not a lucky backtest.
Testing many settings always turns up one that looks brilliant on the past. Settings worth trusting sit in a broad area of good results, where small changes don't break them.
| Stop × ATR ↓ / Period → | 20 | 30 | 40 | 50 | 60 | 70 | 80 |
|---|---|---|---|---|---|---|---|
| 1.00 | 0.96 | 1.05 | 1.11 | 1.14 | 1.14 | 1.11 | 1.05 |
| 1.25 | 1.05 | 1.14 | 1.20 | 1.23 | 1.23 | 1.20 | 1.14 |
| 1.50 | 1.09 | 1.18 | 1.24 | 1.27 | 1.27 | 1.24 | 1.18 |
| 1.75 | 1.09 | 1.18 | 1.24 | 1.27 | 1.27 | 1.24 | 1.18 |
| 2.00 | 1.05 | 1.14 | 1.20 | 1.23 | 1.23 | 1.20 | 1.14 |
| 2.25 | 0.96 | 1.05 | 1.11 | 1.14 | 1.14 | 1.11 | 1.05 |
| 2.50 | 1.92 | 0.91 | 0.97 | 1.00 | 1.00 | 0.97 | 0.91 |
The plateau: settings worth trusting
Good results across many neighbouring settings. Small changes don't break it, so it's more likely to hold up on new data.
The lone peak: probably luck
The best single number on the grid, surrounded by losing settings. It fits the past by chance; we don't pick settings like this.
Illustration: profit factor (gross profit ÷ gross loss) for 49 combinations of two settings.
Testing a strategy across a range of parameter values to see how performance actually responds.
Watching for parameters tuned so precisely to the past that they're unlikely to hold up going forward.
Re-testing optimized parameters on data they weren't tuned against, to check they still hold.
Understanding how much a small parameter change moves the results. Fragile strategies show up here.
Checking performance across different market regimes and time periods, not just one favorable window.
Optimizing for risk-adjusted return (Sharpe, drawdown, consistency), not just raw profit.
The same two settings from the grid above, tested on the data they were tuned on and on data they had never seen.
Plateau settings
Lone-peak settings
Illustration: profit factor. The dashed line is 1.00, breakeven. The best number on old data is often the worst choice for new data.
Tuning settings until they fit past data almost perfectly. The result describes the past well and the future badly. Fewer settings, broad plateaus and out-of-sample tests are the protection.
No. Sometimes the honest finding is that the original settings were already as good as they get, or that the strategy has no edge to improve. We report that plainly.
Risk-adjusted results: steady returns with controlled drawdowns, not the single highest profit figure.
Let's stress-test it properly before you trust it. Send us the strategy and the results.
