Nash-MOPD: Balancing Training Across Multiple Teachers
Nash-MOPD uses a local minimax game to allocate training samples across domains in multi-teacher distillation, aiming to protect slower-learning tasks.
👁 114 reads
A paper submitted to arXiv (cs.LG / cs.AI) on October 19, 2026, by Siyuan Gan and six co-authors. The document does not explicitly name their affiliations.
How to divide a training budget among multiple teachers
Multi-teacher on-policy distillation (MOPD) trains one student model from several expert models, each specialized in a different domain, such as mathematics or coding. The challenge is that the student has a finite number of optimization steps and samples (rollouts), which must be divided among those domains.
The authors describe two existing baselines. The first, called Fixed, uses constant domain-sampling proportions based on corpus size, with no adaptation. The second, D3-MOPD, dynamically adjusts the proportions using the reverse Kullback-Leibler (KL) divergence between teacher and student, along with its most recent decrease. However, D3-MOPD favors domains that have improved quickly lately, which can leave slower-learning domains with too few examples.
How Nash-MOPD uses a local minimax game
Nash-MOPD addresses this problem by framing domain allocation as a local minimax game. The game’s equilibrium maximizes the smallest predicted proportional reduction in reverse KL divergence within a local model, protecting slower-improving domains from being underfunded. Put simply, the method gives more prompts to domains that need more examples to achieve a comparable improvement.
The authors emphasize that Nash-MOPD requires neither extra queries to teachers nor additional rollouts. It uses the same reverse KL divergence statistics already available during training.
Experimental results
The tests used two base models: DeepSeek-R1-Distill-Qwen-1.5B and Qwen3-8B-Base. The authors report results after 100 training steps. On DeepSeek-R1-Distill-Qwen-1.5B, after 100 steps Nash-MOPD surpasses the final results of both baselines, Fixed and D3-MOPD, in mathematics and coding. On Qwen3-8B-Base, after 100 steps it matches the final result of Fixed.
Across the reported metrics, Nash-MOPD leads in eight out of nine and is best or tied for best on all six accuracy metrics for mathematics and coding. The document does not provide the specific numerical values for these metrics; they are not available in the abstract itself.
What the document does not include
The available text is only the arXiv abstract. It does not include detailed results tables, descriptions of the teacher architectures, a list of the datasets used, an analysis of compute costs, or comparisons with methods outside the MOPD group. Full methodological details and ablation experiments may be in the complete PDF, whose contents were not provided.
Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →What the results could mean
This is an early, unreviewed preprint, so the results should be treated with caution. The idea of using a Nash equilibrium to protect slower-learning domains, rather than rewarding those that are already improving quickly, seems conceptually sound and computationally economical to me because it adds no teacher queries. If the full paper confirms an advantage after just 100 steps, that could be a strong argument for using Nash-MOPD when training one model on multiple specialized datasets. For now, though, we have only the abstract, and the absolute size of the differences is unknown.
Source: arXiv cs.AI: Nash-MOPD: Optimizing Task Trade-offs in Multi-Teacher On-Policy Distillation (document dated October 20, 2026). This article is based on the document, not on reports from other media.
