Epoch AI’s training compute estimates get the New York Times visual treatment

The New York Times has turned Epoch AI’s research into a visualization showing just how much math goes into training a modern AI model. The piece draws on Epoch AI’s estimates of training compute and capability scores. The details: counting the calculations Epoch AI measures training compute in FLOP, short for floating-point operations. Each FLOP is a single arithmetic calculation involving decimal numbers. According to Epoch AI, training compute for frontier language models has grown at roughly 5x per year since 2020. That works out to a doubling roughly every 5.2 months. The longer view is similar. Some Epoch AI analyses find that notable AI models have seen roughly 4.4 to 4.7x annual growth in training compute since 2010. Epoch AI has also broken down where that growth comes from for frontier models after 2018. It points to three main levers: Hardware quantity: a median of roughly 1.69x per year, meaning labs are simply buying and wiring together more chips. Training duration: roughly 1.53x per year, as runs last longer. Hardware performance: roughly 1.41x per year, as each chip gets faster. Epoch AI puts the combined effect at roughly 4.27x annual growth. Background: the chip c...