Carnegie Mellon research shows unified training method for AI models

A new paper argues pretraining, fine-tuning, and reinforcement learning can run as one algorithm instead of three separate stages Most foundation models are built like a three-course meal: pretraining first, fine-tuning second, reinforcement learning for dessert. A new paper from Carnegie Mellon University suggests the courses might work better served on one plate. The study, titled “Training Foundation Models with a Single Algorithm,” argues that blending all three types of training data into one unified method can beat the traditional staged approach. What the researchers actually did The paper appeared on arXiv on October 29, 2026. Its lead author is Yuda Song of CMU, who worked alongside collaborators from Johns Hopkins University and Aurora Innovation. The core idea is a reframing. The team treats every training stage as maximum-likelihood fitting of model-generated data. That shift moves the important decisions somewhere new. Instead of a rigid sequence of distinct phases, the framework puts the emphasis on data composition and scheduling. Static datasets, the kind used in pretraining and supervised fine-tuning, get folded together with the data management used in reinforceme...