Backtesting Cycle Methods: A Primer for Honest Evaluation

Cycle analysis methods are particularly prone to a specific kind of overfitting: finding patterns that explain the past perfectly and predict the future not at all. The flexibility of the frameworks, cycles of any period, any phase, any amplitude, makes it easy to fit historical data after the fact.

A rigorous backtest is the only protection against this. This primer covers what a rigorous backtest for cycle methods looks like, what common shortcuts produce false confidence, and how to interpret the results honestly.

Why cycle methods are hard to backtest

Standard trading strategy backtesting assumes a fixed rule: "buy when indicator X crosses threshold Y." You apply the rule to historical data, count wins and losses, compute the expected value.

Cycle methods don't fit this structure cleanly. The cycle parameters, period, amplitude, phase, are typically estimated from historical data. If you estimate them on the same data you test them on, you get an optimistic bias. The cycle you identified is guaranteed to fit the estimation period; the question is whether it fits out-of-sample.

This is the data snooping problem. Gann analysts who study ten years of data, identify the cycles that worked, and declare their method validated have not produced a fair test. They've produced a curve-fit.

The correct methodology

Step 1: Separate estimation and testing periods

Divide your historical data into two non-overlapping periods. The estimation period (also called the in-sample or training period) is used to identify cycles. The testing period (out-of-sample or validation period) is held out and not examined until after the cycle parameters are fixed.

A rule of thumb: use 70% of your data for estimation, 30% for testing. Never look at the test set while you're doing cycle identification.

Step 2: Identify cycles on the estimation period only

Run your FFT, apply your Bartels significance tests, identify your Bradley siderograph parameters, calibrate your Gann Square of Nine scale, all of this on the estimation period only.

Step 3: Fix all parameters

Before looking at the test period, write down all cycle parameters exactly. This prevents the temptation to adjust parameters after seeing test results. If you change anything after peeking at the test set, the test is invalid.

Step 4: Apply fixed rules to the test period

Run the cycle model with fixed parameters on the test period. Record the outcomes. Do not adjust.

Step 5: Evaluate the test period results

The test period gives you an unbiased estimate of forward performance. If the cycle model had genuine predictive content in the estimation period, it will show positive expected value in the test period. If it was overfitted to the estimation period, expected value in the test will be near zero or negative.

What counts as a "turn" for scoring

Defining what counts as a turning point is a critical methodological decision that needs to be made before you look at data.

Common approaches: - Fixed threshold: a "turn" is a local extremum within a ±N day window where the peak-to-trough move exceeds X% - N-bar high/low: a "turn" is the highest high or lowest low over the next N bars - Drawdown-based: a "turn" is identified using a maximum adverse excursion criterion

Whatever you choose, fix it before the analysis and apply it consistently across the entire sample. Ad hoc adjustment of the turn definition to match your cycle predictions is the most common form of cherry-picking in cycle research.

The window matching problem

When you claim that a Bradley siderograph extreme "called" a market turn, you need a window definition. Did the turn happen within ±3 days? ±10 days? ±30 days?

A wider window makes your hit rate look better but has less predictive content, saying "the market will turn sometime in the next month" is a weaker claim than "the market will turn on or around this date."

The statistically correct approach is to specify the window in advance and then test whether the hit rate within that window is significantly higher than the base rate (the fraction of random days that fall within the window of any turn).

For a 3-day window and turns occurring approximately monthly, the base rate is about 20% (6 days / 30 days). A genuine signal should produce hit rates well above this, 40%+ is meaningful; 25% is borderline noise.

The Bartels connection

After identifying cycles and testing them out-of-sample, the Bartels significance test provides an additional check. If a cycle produces positive test-period results but fails the Bartels test on the full sample, you have a puzzle. Positive results but no statistically significant cycle suggests you got lucky in the test period.

If a cycle passes Bartels on the estimation period and produces positive test period results, the evidence is stronger, though still not conclusive, because the test period is itself just one realisation of a random process.

Walk-forward testing

For cycle parameters that are updated over time (rolling estimation windows), walk-forward testing is the right methodology.

Procedure: 1. Estimate cycles on the first N years of data 2. Apply those cycles to generate signals for year N+1 3. Record the results 4. Add year N+1 to the estimation set, re-estimate cycles 5. Apply to year N+2 6. Repeat

This simulates what a real trader using this method would have experienced. The walk-forward results are the most realistic estimate of live performance.

Walk-forward testing is more computationally intensive but more honest. Crohamhurst.app includes walk-forward infrastructure for uploaded time series.

Interpreting results honestly

A cycle-based system with positive out-of-sample expected value over 5+ years of walk-forward testing has earned some credibility. That doesn't mean it will work in the future. Markets change regime. Cycles that were robust before 2010 may be weakened by algorithmic trading that arbitrages away cyclical patterns as soon as they become known.

The honest conclusion from most rigorous cycle analysis backtests is that individual cycles have weak positive expected value. The edge is real but small, typically in the range of 2-5% annualised over a broad-market benchmark. Meaningful position sizing requires combining multiple independent cycle signals to achieve acceptable Sharpe ratios.

A single cycle with p = 0.04 in the Bartels test and a borderline positive walk-forward result is not a basis for concentrated trading. Ten such cycles, properly combined, might be.

Related: Bartels Cycle Significance Testing | FFT, Hurst, and Bartels: Cycle Analysis for Financial Data | Bradley Siderograph Complete Guide