Salesforce study finds 42% compute savings by distilling RL-trained experts

A new Salesforce AI Research preprint suggests training specialists separately and merging them can match joint reinforcement learning at a fraction of the cost Training one AI model to be good at everything is expensive. Salesforce AI Research says there may be a cheaper route: train several specialists, then pour what they know into a single model. In a preprint released on November 5, 2026, the team reports that this approach matched the performance of joint reinforcement learning on a combined math-and-code setup while using 42% fewer total training FLOPs. What the researchers actually tested The paper is titled “Unpacking the Compute Efficiency of Strong-to-Weak OPD against GRPO” and is posted as arXiv:2611.08453. GRPO stands for Group Relative Policy Optimization. It is a reinforcement learning technique that trains a model using verifiable outcome rewards, meaning the model gets credit when its result can be checked as correct. OPD stands for on-policy distillation. The study pits these two approaches against each other on a simple scoreboard: how much held-out accuracy does each method deliver for every unit of training compute spent? To answer that, the researchers measure...