Relational Model Training: 305 Seconds on One GPU
A NeurIPS 2026 workshop paper proposes a speedrun-style leaderboard for relational foundation model training, with a 305.63-second result on one NVIDIA H200.
👁 114 reads
The paper appeared on arXiv as 2610.28855, was submitted on October 20, 2026, and was accepted to the CODEC-FM workshop at NeurIPS 2026. Its authors are Gabriel Wedson Mendonça de Souza Veloso, Luan Borges Teodoro Reis Sena, Denis Oliveira Correa, and Francisco Galuppo Azevedo. The paper is 9 pages long and contains 4 figures.
Why the authors want to change model training
Training large foundation models is expensive, so experiments are largely conducted by big labs. Training recipes are developed within those labs, documented according to internal protocols, and rarely accumulate as knowledge that is available to everyone. The authors argue that progress in this field depends on who can afford to run experiments - and that relational models, trained on data from relational databases, are no exception.
A speedrun-style leaderboard for AI training
The authors propose bringing the idea of video game speedruns - competitions to complete a task as quickly as possible - to AI model training. The rules are straightforward: the problem is fixed, the tools are open, and submissions are ranked by the wall-clock time needed to reach a specified quality threshold. The authors emphasize that a single GPU is enough to beat the record, which could lower the barrier to entry for independent researchers.
Their tool, modded-nanoRT, is described as the first leaderboard of its kind for Relational Transformers, transformer models designed to work with relational data.
Choosing a dataset for the benchmark
The authors point to a less obvious challenge: the dataset used as a benchmark reference determines whether a benchmark can distinguish a genuine improvement from random noise. To address this, they calibrated the benchmark by evaluating six candidates on how clearly a 45-minute learning curve separates real improvement from variation caused by random initialization, known as seed noise. Based on that evaluation, they selected one dataset as a frozen reference point.
What the 305-second training result involves
The authors open the leaderboard with their own recipe, combining three elements. The first is typed trilinear attention, a relational operator that conditions attention scores and values on the foreign keys traversed in the database. The second is the Muon optimizer. The third is the SIGReg technique. The paper does not explain the mechanics of Muon or SIGReg in detail; both are named without further explanation in the available abstract.
The recipe reaches its quality target in 305.63 seconds on a single NVIDIA H200 GPU, making it 8.11 times faster than the reference implementation. In tests outside the dataset used to develop the recipe, it won 13 of 15 pairwise comparisons.
Benchmark code and availability
The authors say they will publish the benchmark, its frozen artifacts, and the record runs. The code is already available at the address given in the paper. The repository is not named in the available abstract.
Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →What this result means
This is a short workshop paper - 9 pages - with no new architecture or large-scale results. Its practical value lies elsewhere: if the leaderboard works as the authors describe, it could give an individual researcher with one GPU a way to compete with a lab and contribute meaningfully to progress in training methods for relational models. It is an interesting organizational idea, though for now there is only one entry, from the creators themselves. It remains to be seen whether the wider community will use it.
Source: arXiv cs.AI: Speedrunning Relational Foundation Model Pretraining (document dated 2026-10-22). This article is based on the full paper, not on coverage by other media.
