Validation · #359741 · Cross-asset futures

Attention-LSTM deep momentum network

Trading with the Momentum Transformer: An Intelligent and Interpretable Architecture, 2022

Kieran Wood, Sven Giegerich, Stephen Roberts & Stefan Zohren · Oxford-Man Institute of Quantitative Finance / arXiv q-fin

Open in ColabColab

The rule

Show a neural network one year of each futures contract's trend signals and tell it, directly: pick the position sizes that maximise the portfolio's Sharpe ratio. There is no return forecast and no label — an attention layer lets the model look across the whole year at once and notice where a trend broke, which is the pitch for why it should survive regime changes the plain LSTM version misses.

Fails2.54→0.13claimed → measured Sharpe

2.54

Claimed Sharpethe Decoder-Only TFT on 50 Pinnacle futures, averaged over the 1995–2020 test windows — a 50% lift over the authors' own LSTM deep momentum network at 1.70

0.13

Measured Sharpethe same architecture retrained on the 40-contract roll-adjusted panel, 2017–2026. Provisional: independent fits span −0.04 to +0.40, and this comes from a 4-draw hyperparameter search against the paper's 50. A full-budget rerun is scheduled — see the note below

0.52

The benchmark it must beatthe plain 12-month trend rule the paper itself tables at 1.03 still scores 0.52 here — four times the Transformer, at every cost level

Asset40 futures — commodities, US equity, US Treasury, FX

StrategyDecoder-only temporal fusion transformer, long/short

Period2017–2026, daily

CostsTurnover-based sweep (breakeven ≈ 0.14bps)

BenchmarkLSTM deep momentum network + plain 12-month TSMOM

Assets tradedCLGCNGHGZCESNQZN6E6J…and 30 more roll-adjusted continuous futures: 26 commodities (energy, metals, grains, livestock, softs), 3 US equity indices, 4 US Treasuries, 7 FX majors. The paper's 50-contract Pinnacle universe adds non-US equity and fixed-income futures with no public source, so ours is US-only outside commodities and, like the paper's own panel, commodity-heavy.

The exact rules

Each day, per contract	→	Compute the 8 Deep-Momentum-Network inputs — volatility-scaled returns over 1 day, 1, 3, 6 and 12 months, plus three normalised MACD speeds (8/24, 16/48, 32/96) — from a price series winsorised at ±5 exponentially-weighted standard deviations
Feed the model	→	The last 252 trading days of those 8 inputs for that contract, plus the contract's own identity as a static covariate
Inside the model	→	A Variable Selection Network softmax-weights the 8 inputs at every timestep → an LSTM reads the sequence → interpretable multi-head attention (4 heads, causal mask) looks back across the whole year → a tanh layer emits the position
Position	→	A continuous size between −1 and +1 per contract per day — a conviction, not just a direction
Size it	→	Trade that size × 15% ÷ σ, where σ is the contract's 60-day exponentially-weighted volatility, so every contract contributes equal risk
Portfolio	→	Equal-weight the per-contract returns across the 35–40 contracts and rebalance daily
Train it	→	Mini-batch SGD (Adam) directly on the Sharpe ratio of the captured returns — up to 300 epochs, early stopping on the diversified validation Sharpe with patience 25, best weights restored
Refit	→	Expanding window — retrain on all history before each test block, then hold the model fixed through it

The backtest, re-run

Growth of $1 · log scale

Drawdown · deepest -14.0%

252-day rolling Sharpe

Inside the model

Position mix: 45% long / 55% short across 35–40 contracts a day, and effectively binary: 76% of positions sit past 0.9 of the maximum. The paper's model realises 1.54% volatility against our 4.07%, so its book was sized far more finely than ours — the one difference we would most want a longer run to settle
Win rate: 49.7% of days, 51.8% of months — a coin flip
Skew / kurtosis: −0.39 / 7.6 daily — a fat left tail on a strategy sold on drawdown control
Best / worst day: +1.04% / −1.89% · month +3.56% / −3.74%
Annual returns: 2017 −2.1 · 2018 +2.1 · 2019 −1.9 · 2020 −4.3 · 2021 +5.0 · 2022 +1.0 · 2023 0.0 · 2024 +7.6 · 2025 −3.8 · 2026 −1.0 — four positive years in ten, and the record rests on 2024
Does the timing beat a coin flip?: Permute the model's own positions in time 400 times, holding the long/short mix and every contract's exposure fixed, and the shuffled book beats its actual timing 56% of the time — the model's Sharpe sits at the 44th percentile of its own null. Run on the most favourable fit we obtained, so it is not a weak seed. Whatever the book earns comes from the exposure, not from the sequencing
Seed spread: Independent fits of the same selected configurations give −0.04, +0.07, +0.35 and +0.40; the LSTM gives −0.37, −0.10 and +0.19. The paper reports only the mean of its five repeats and never a range
Retraining is not reproducible: Retrained with an identical seed, configuration and epoch budget on the same machine, the Transformer does not come back the same model — the static-contract embedding's backward pass is not deterministic on this hardware, and early stopping turns that into a different chosen epoch. The LSTM, which has no embedding layer, reproduces to nine decimal places. A single reported Sharpe for an architecture like this is one draw, not a measurement
In-sample versus out-of-sample: The selected configurations score 1.99, 1.02 and 1.65 on the validation windows they are chosen on — the LSTM reaches 4.19 — and 0.13 and −0.09 on the test windows. The architecture trains perfectly well; everything it learns is specific to the window it was selected on. That is the whole result
Benchmark calibration: The paper's own TSMOM row reproduces on the risk side almost exactly — volatility 4.46% against 4.47% claimed, drawdown −6.8% against −6.3%, hit rate 53.2% against 54.2% — while its return halves. So the panel and accounting are calibrated, and the honest era discount is about 2×. Grant the Transformer the same 2× haircut and it should still measure 1.27

The validation ladder

C0	Replicate	The architecture retrains and the pipeline is provably causal — hand the model tomorrow's features and the Sharpe explodes from 0.31 to 75.3, exactly as a correctly wired backtest must. The magnitude does not survive: 0.13 against the claimed 2.54, return 0.43% against 4.01%, drawdown −17.3% against −1.32%. The benchmark ordering inverts too: the paper ranks Transformer above LSTM above TSMOM above buy-and-hold; we measure TSMOM first, buy-and-hold second, and the Transformer fourth.
C1	Honesty	Breakeven cost 0.14bps. Liquid futures cost 1–3bps one-way, so the strategy never clears its own friction. At 1bp the paper claims +1.01 and we measure −0.42. This is what separates it from merely overstated: gross-positive, money-losing as actually traded.
C2	Deflate	Deflated Sharpe 0.008 against the paper's own disclosed search — 50 random draws across 7 reported architectures, 350 configurations, 8,750 fits once its windows and repeats are counted. No configuration in the family clears multiple testing, and the best t-statistic in the whole family (1.78) belongs to the plain TSMOM rule, not the Transformer. PBO 0.43 passes: the family is stable, it is uniformly weak rather than cherry-picked.
C3	Crisis	The paper's flagship claim inverts. Its SARS-CoV-2 scenario — the 'naturally adapts to new market regimes' argument — is claimed at +1.22 and measures −0.59, with seeds spanning −1.88 to +0.12, while the naive trend rule makes +1.45 in the same window. The COVID crash itself: −6.40 Sharpe. Only the August 2024 yen unwind is strongly positive.
C5	Frictions	A daily continuous-position model with very high turnover. The cost slope replicates — the paper loses about 0.69 Sharpe per basis point, we lose 0.49 — so the turnover accounting is right. The level is not: it starts at 0.07 and crosses zero almost immediately.
C6	Decay	There is nothing to decay. Inside the paper's own sample era (2017–2020) it scores 0.05 against the 1.71 claimed for its 2015–2020 row. Post-publication (2022–2026) it scores 0.15 against the trend rule's 0.53 — marginally better after publication than inside the paper's era, because it was never working.
C7	Originality	No alpha over anything tested. Against the plain 12-month trend rule: α −0.26%/yr at t = −0.24, R² 0.07. Against its own LSTM: α +0.29%/yr at t = 0.26. And a pooled linear regression on the identical eight inputs scores 0.09 — statistically indistinguishable from the Transformer's 0.13, paired t = −0.05. Whatever the attention layer contributes, it is not measurable here.

Architecture retrainsPipeline causalMagnitude replicatesBeats its own LSTMBeats the naive trend ruleBeats a linear model, same inputsStatistically realCost-robustCrisis-robustSeed-stable

How we rebuilt it

Data: The roll-adjusted continuous-futures reference panel: exchange daily settlements (CME Globex / ICE, via Databento) assembled into continuous series under a declared open-interest roll rule and ratio back-adjusted at every roll, so returns are what a rolled position actually earns. The paper's own Pinnacle panel is back-adjusted the same way — the difference is history depth, not construction.
Method: The authors' architecture rebuilt layer for layer from their released TensorFlow code into PyTorch: per-feature embeddings → Variable Selection Network → LSTM seeded with static context → gated residual blocks → interpretable multi-head attention under a causal decoder mask → tanh position head, trained on the Sharpe loss with early stopping on the diversified validation Sharpe. Initialisation is matched to Keras — glorot kernels, orthogonal recurrent kernel, unit forget-gate bias — because PyTorch's defaults train an LSTM differently and would have handicapped the paper.
Universe: 26 commodities + 3 US equity indices + 4 US Treasuries + 7 FX majors, against the paper's 25 commodities + 11 equity + 5 fixed income + 10 FX. The five ICE softs clear their feature warm-up in 2020, so they enter in the final window — the same staggered membership the paper's own asset table uses.
Window: A daily strategy with no public daily futures history before 2010: the feature warm-up burns to late 2011, models train from there, and the out-of-sample runs 2017–2026 in three expanding blocks. 2017–2020 sits inside the paper's own sample era; 2022–2026 is true post-publication. The 1995–2020 average the headline quotes is out of public-data reach.

Deviations from the paper

PROVISIONAL MAGNITUDE — a full-budget rerun is scheduled. The measured Sharpe here comes from a 4-draw hyperparameter search per window against the paper's 50, with 3 seeds against its 5 and a 60-epoch cap against its 300. That is enough to establish the verdict, which rests on the cost breakeven, the inverted crisis scenario and the benchmark comparison — none of which depend on search size — but it is not enough to pin the number. Expect 0.13 to move when the full search runs; the finding that will not move is that the architecture is dominated by a one-line trend rule and loses money at any cost a real futures book pays.
Search size — the most important caveat, and it cuts in the paper's favour. We draw 4 random configurations per window from the paper's own grid; the paper draws 50. A larger search might find a configuration that generalises better and we cannot rule that out. Three things limit how much it can explain: the cost conclusion is structural (a daily continuous-position model's turnover is not a function of search size, and the paper's own cost table already shows the architecture is fragile), the benchmarks involve no search at all and retain half their claimed Sharpe, and the paper's own LSTM — searched identically here — also lands negative. Deflation uses the paper's disclosed 350 configurations throughout, never our own count.
Window: the paper's 1995–2020 average is unreproducible on public data — no public daily futures history exists before 2010-06. We measure 2017–2026, of which 2017–2020 sits inside the paper's own sample era.
Universe: 40 contracts against the paper's 50; the non-US equity and fixed-income index futures have no public source, so those sleeves are US-only.
Our retrain converges to a near-binary book — 76% of positions past 0.9 of maximum, 4.07% realised volatility — where the paper's evidently did not, at 1.54%. The paper's Sharpe of 2.54 comes from earning nearly the trend rule's return at a third of its volatility, which requires informative time-varying position sizing. The Sharpe objective is invariant to a global rescaling of positions, so nothing in the loss forces the difference; it is plausibly a consequence of training on 194–438 sequences rather than the paper's larger panel and history. This is an open question about the reproduction rather than a demonstrated flaw in the paper, and it is the result we would most want a longer run to settle.
Framework: the authors' models are Keras/TensorFlow, ours are PyTorch, matched layer for layer with Keras initialisation. Random-number streams differ, so individual fits are not bit-identical to theirs.
The changepoint-detection variant (claimed 2.62) is not implemented — it needs a Bayesian Gaussian-process changepoint detector run per contract per day at two lookbacks. It is not the claim under test: 2.54 is the no-changepoint row, and its 4.01% return and 1.32% drawdown match the extracted specification exactly. That variant remains untested here.
The four other transformer variants the paper benchmarks (canonical, decoder-only, convolutional, Informer) are not re-implemented — they are baselines the paper's own headline already beats, not the claim.

ProvenanceBeta

Engine: v1
Blocks: 2 new (the Lim-Zohren-Roberts deep-momentum feature pipeline; a PyTorch port of the decoder-only temporal fusion transformer and the LSTM deep momentum network, with the Sharpe loss, diversified-validation early stopping and the paper's random search), 7 reused (reference-panel adapter, metrics, multiple-testing / deflated-Sharpe / PBO gates, regime analysis, single-factor spanning)
Data: Roll-adjusted continuous-futures reference panel, vintage databento_glbx_2026-07-17 (CME Globex / ICE daily settlements, open-interest roll rule, ratio back-adjusted), daily 2010-06→2026-06; models trained from 2011-11, out-of-sample 2017-01→2026-06
Source: arXiv 2112.08534 ↗ · Authors' reference implementation ↗ · Databento CME Globex daily settlements ↗ · Databento ICE Futures US (softs) ↗
Tests: 12 known-value tests — Sharpe-loss closed form, padding mask, target alignment, feature causality under truncation, Keras-parity initialisation, max-norm output constraint, diversified validation Sharpe — including a causality probe on both architectures: perturbing an input at t+1 must leave every position at t unchanged. Plus end-to-end negative controls: a look-ahead trap that explodes the Sharpe to 75.3, a stale-features control, a lagged-position control, and a position-shuffle null. One control, a feature shuffle, failed as designed and is reported as uninformative rather than used
Reproduce: view code ↗