Market validation

A statistical relationship is not yet usable market information.

We expose a mathematically validated hypothesis to constraints that reflect real markets. Simulation is not a standalone product. It is a control step used to test whether a result holds after timing, costs, liquidity, capacity and risk are considered.

Market hypothesis validation framework

We specify:

• when the information is generated
• when it is actually available
• how it is converted into a measurable signal
• which horizon the hypothesis is intended to explain
• what would invalidate the hypothesis

Information that was revised later or became available only at a later date must not be used as though it had been known earlier.

We examine whether the observed change is related to financial results, deviation from expectations, price behaviour, volume, liquidity or relative behaviour versus the market or industry.

For events such as earnings releases, we use an event study when appropriate to the research question and measure price behaviour against a relevant market or industry model.

We test when the signal begins to contain information and how quickly that value decays.

The same hypothesis may be relevant over a horizon of several days and have no informational value over several months. We therefore test multiple predefined time horizons.

The result must hold in a period on which it was not developed.

When working across multiple companies and periods, we use a panel structure. The number of tested variants must be proportionate to the amount of genuinely available data.

Before interpreting the result, we also assess statistical power. Selecting the best result from a large number of attempts without adjustment is not evidence.

A backtest must not assume that a trade occurs instantly and without cost. Depending on the market, we therefore account for factors such as:

• delay between signal generation and possible execution
• bid-ask spread and fees
• liquidity
• limits on trade size relative to typical daily volume
• estimated market impact
• turnover
• strategy capacity

We assess results before costs and after a realistic estimate of costs.

We do not assess risk separately from volatility, liquidity or uncertainty in the estimate itself. Depending on the market, the simulation may use approaches such as volatility targeting and limits on position size.

A mathematically optimal position under perfectly known probabilities is not the same as a reasonable position when the estimate itself is uncertain.

One historical sequence is only one realisation. We therefore test how the outcome changes under a different ordering of trades, blocks of high volatility, higher costs, lower liquidity or a change in market regime.

We focus on the distribution of possible outcomes, not only on a single historical maximum drawdown.

The more model variants we test, the greater the risk that one will produce a good result by chance. We therefore track the number of variants, stability outside the training sample, sensitivity to parameter changes and signs of structural change in performance.

We do not consider a model robust if it works only under one precise combination of parameters.

We define in advance what must happen for us to reject the hypothesis. Examples include the effect disappearing after controlling for a market factor, unstable lead time, the result disappearing after realistic costs, insufficient liquidity or a result that exists only within one narrow period.

Without conditions for rejection, the process is not validation. It is a search for confirmation.

Practical market experience

Applied mathematics shows whether a statistical relationship exists and how strong the evidence is. Practical financial-market experience helps determine whether a hypothetical result can be implemented at the relevant time and scale.

We consider factors such as information availability, liquidity, spread, expected slippage, feasible exposure, market regime and the result after costs.

Market validation is part of the research process, not a public trading or investment service.

Market validation output

Depending on scope, the validation output includes:

• definition of the hypothesis and signal
• the data and timing framework
• methodology
• the out-of-sample result
• results before and after costs
• sensitivity to liquidity and capacity
• the distribution of possible outcomes
• conditions of failure
• a conclusion: Accept / Monitor / Reject