A research note on validation method — worst-case fills and walk-forward.
Every backtest is a story a strategy tells about a past it never actually traded. The distance between that story and live reality is where most systematic strategies quietly go to die.
We run CRYNOMAD as a small, solo-operated quant shop building and validating a handful of systematic crypto strategies. We do not manage client assets. Being solo has one uncomfortable consequence: there is no second research desk to catch our optimism before it becomes a live position. So over time we have moved the skepticism out of our heads and into the tooling. This note is about one part of that — how we reconstruct executions inside a price bar so a backtest assumes the worst reasonable fill instead of the best.
There are no performance figures in this note. There won't be. We explain why at the end, but the short version is that method is the only part of this worth publishing.
A candle hides more than it shows
Most backtests are built on OHLC bars: for each period you get an open, a high, a low, and a close. Four numbers. It feels like data, but it is a summary — and the summary throws away the one thing that most affects a trade's outcome: the order in which price visited those levels.
Consider a long position with a protective stop below and a take-profit above, and a bar whose range spans both. Did price rally to the target first and then sell off? Or did it wash down to the stop first and then recover into the close? The OHLC bar cannot tell you. Both paths produce the identical four numbers.
This ambiguity is not a rounding error. If a backtester silently assumes the favorable ordering — target before stop — every one of those bars becomes a small, invisible gift to the backtest. Do that a few thousand times and you have a strategy that looks robust on paper and bleeds the moment it meets a real order book.
Assume you got the worse fill
Our rule is deliberately pessimistic: when a single bar could have triggered either a favorable or an adverse level, we assume the adverse one happened first. For a long, that means we treat the drawdown extreme as reached before the profit extreme; a stop and a target inside the same bar resolve as a stop. We walk each bar as an intra-bar path in the order least kind to the open position, rather than trusting the tidy close.
None of this is exotic — it is just the honest reading of an ambiguous bar. But it is surprisingly easy to skip, because the optimistic version makes everything you build look better, and nobody enjoys shipping a tool whose main job is to make their own ideas look worse.
The payoff is that a strategy which only survives under generous fill assumptions never reaches a live book. We would rather kill an idea in a spreadsheet than in the market.
Walk forward, and only score what the model hasn't seen
Pessimistic fills fix one lie. Overfitting is the other, and it needs a different discipline.
The temptation in systematic trading is to tune parameters across an entire price history and then report how beautifully they fit. That result is close to meaningless — you have measured memorization, not skill. Instead we validate walk-forward: fit on an in-sample window, then test on the next window the model has never seen, roll the whole frame forward, and repeat across the history. Only the out-of-sample segments count toward whether a strategy earns its place.
Put worst-case fills and walk-forward together and you have a validation pipeline whose entire personality is refusal. Most ideas do not make it through, and the ones that do have already survived their least flattering assumptions. For a one-person shop, that harshness is not pessimism — it is the closest thing we have to a second opinion.
A lineup built to fail on different days
The strategies that clear this process are deliberately different in where their edge is supposed to come from, so they are unlikely to all fail on the same day: VOLCANO reads positioning extremes in perpetual futures; GLACIER harvests carry from funding and basis; HORIZON earns from defined-risk options structures; AURORA is a separate, completely free AI signal service with no paid tier.
We are intentionally light on the internals. What matters for this note is not the specific ensemble or the parameters — it is that each one has to clear the same unglamorous, adversarial validation before it is allowed anywhere near a live book.
Why there are no numbers here
You have read this far without seeing a single performance figure, and that is on purpose. As a matter of regulatory caution we do not publish them anywhere public — the method, yes; never the performance numbers. Performance data is shared individually with qualified investors on request.
We have made our peace with that trade-off, because the method is the part that decides whether any number would even mean something. A good-looking curve produced by an optimistic backtest is just a nicer way to be wrong. If you are going to trust a track record, the first thing worth interrogating is how honestly it was built — and that is the part we are happy to show.
We publish the reasoning, never the performance numbers. Performance data is shared individually with qualified investors on request.