By Tapio Schneider and Raffaele Ferrari.

In 2018, with support from the Heising-Simons Foundation and Charlie Trimble, we convened a series of workshops on the “Future of Earth System Modeling.” We asked leaders in the field a question: given the data and computational resources available now, what would you do if you had to build a new Earth system model from scratch?

Few if any wanted to do such a thing—understandably, since existing models embody decades of invested effort and whole communities of users depend on them. But the question was worth asking: no one had built a new Earth system model in a long time. Existing Earth system models trace their roots decades into the past, when the world’s fastest supercomputers had less processing power than today’s iPhones. Despite decades of refinement since, their errors in simulating the present climate and their uncertainties in projecting the future had remained stubbornly large. Core concepts in parameterizations of some of the most uncertain processes, such as clouds, date to the 1970s and 80s, when climate models had horizontal resolutions of around 1000 km. At those resolutions, it was reasonable to assume a separation of scales between what models could resolve and processes such as clouds, hurricanes, and ocean turbulence.

None of this held any longer. Along with a billionfold increase in computational power since the last time an entirely new model was conceived, the volume of Earth observations, from space and from the ground, had grown by orders of magnitude. Resolved scales approached the scales of the processes governing convection and clouds in the atmosphere, and turbulence in the oceans, so scale separation could no longer be assumed. Machine learning had arrived, and GPUs and other accelerators offered a new level of arithmetic throughput. We thought there was room for substantial progress by building on what was new. Not everyone agreed. At the workshops we also heard what could not be done: no single model, many said, could serve weather forecasting and long-term climate projections across resolutions, reach into past climates, and adapt to other planets.

From an atmosphere model to a full Earth system model

At first we proposed something narrower: an atmosphere model with new parameterizations of the most uncertain processes governing turbulence, convection, and clouds, combining a physical scaffolding with data-driven components. Thanks to the generous support of Eric and Wendy Schmidt (and later Schmidt Sciences), the Paul G. Allen Family Foundation, the National Science Foundation, and the Windward Fund, we could get started.

We quickly realized we could and should do more. A step change in Earth system modeling required not an incremental improvement of what existed but a re-imagining of what is possible given today’s resources. That meant software architectures built with modern practices — modular, composable, tested component by component. And it meant re-thinking the science of what goes into a model: using physics and process-informed causal models as far as they can go, and learning from data where they can go no further. The aim was to harness the best of both worlds: the deductive nature of physics, which generalizes out of distribution and can predict a climate and extreme events for which no data exist yet, and the expressiveness and adaptability of models learned from data.

This is no longer a contrarian view. A recent perspective from leaders at ECMWF and other European forecasting centres describes where they argue the field needs to go: modular codes with clear interfaces, componentwise testing, modern languages, GPUs. It is how CliMA has been built from the start.

Drawing on expertise at Caltech, MIT, and NASA’s Jet Propulsion Laboratory, we began to build atmosphere, ocean, and land models: a foundational model of the Earth system that put software first and aimed for a leap in predictions on timescales from days to decades, all in a performance-portable package that exploits modern computing architectures with GPUs and abstraction-based programming through a modern language, Julia. It became the model we were told could not be built: the same code runs on weather and climate timescales, and, because every physical constant and parameter lives in one place, it can be adapted to past climates or to another planet by changing a few numbers. It has also found users we did not plan for: private companies, drawn by its speed and ease of use, simulate with it the impact of environmental interventions in the ocean and atmosphere.

Time for a tour

That’s what CliMA built, and we think it’s time to show it. Over the coming ten weeks, we will introduce the CliMA software stack, component by component. We will show what each piece does, why it exists, and what makes it different. You will also meet the scientists who built it, many of them early in their careers, who saw the potential of starting from scratch. (The photo above shows CliMA students, scientists, and engineers at work in the early days, when the project stood at some 500 pull requests and 60,000 lines of code. The pull requests now number more than 20,000, and the code base is more than ten times larger.) Along the way, each installment will highlight something strange and beautiful about the Earth system itself: how changes in the tilt of Earth’s axis reshape the distribution of sunlight on Earth and pace the ice ages; how the full complexity of moist air follows from a few assumptions; how ocean currents are shaped by Earth’s rotation; what a Nobel-winning calculation from the 1960s looks like when it runs in minutes on a laptop.

Every piece comes with code ready to run. In minutes, it can teach aspects of the Earth system that a few years ago would have taken experts hours to weeks to calculate. That is one argument for a new model: you can run it and see for yourself.

The tour starts next week, at the top of the atmosphere, with sunlight.