Quantitative method
A validation process designed so that results can be trusted when they are negative.
The problem
Analytical results are easy to make look convincing and hard to trust. Most failure modes are ways of fooling yourself: fitting noise, testing until something passes, or measuring against the wrong benchmark. The engineering problem is a process that can return a negative answer and be believed.
Approach
Each hypothesis is registered in writing before it is tested, so a result cannot be reinterpreted afterwards. Nine consecutive hypotheses have been recorded and falsified.
Validation runs out of sample in three independent ways, including a holdout of instruments never used during development.
An empirical null control establishes what a false positive looks like in this pipeline before any true positive is trusted.
Presented as methodology only. No performance figures are shown and none should be inferred — a research process transfers between domains, a track record does not.