Title of Presentation

“From Determinism to Probability: Development of the Ensemble Prediction Technique for Weather and Climate Forecasting”

Weather prediction has a long history, dating back to at least to Babylonian civilization. The advent of digital computers in the 20th Century has allowed weather prediction to be made using the laws of physics and formulated as a deterministic initial-value problem. However, despite mathematical sophistication, this approach has led to spectacular forecast failures, creating skepticism amongst the public about the scientific rigour of meteorological prediction.

With the advent of chaos theory has it been possible to understand not only why precise forecasts of the weather are impossible, but also why the skill of deterministic predictions can vary so much, from week to week and year to year. However, this in turn raises an important question: can we predict the likely skill of a weather forecast? Can we tell the forecast user when to be confident in a prediction, or conversely when any particular prediction must be treated with caution?

This question can be answered through the concept of “ensemble prediction”, where multiple integrations of a numerical weather forecast or climate model are made from slightly different initial conditions and with slightly different model equations. This transforms the prediction problem from that of a deterministic “best-guess” estimate of future weather or climate, to a probabilistic prediction. In situations where the atmosphere is stable and predictable, the forecast probability distributions will be sharply peaked around specific values for the weather variables. By contrast, when the atmosphere is chaotic and unpredictable, the forecast probability distributions will be broad and will encompass a range of specific outcomes.

The author led the teams which introduced ensemble forecasting into the UK Met Office’s operational monthly forecast system in 1986, and into ECMWF’s operational medium-range forecast system in 1992. To make forecast probabilities reliable, a number of innovative techniques were developed e.g. projecting initial uncertainties onto so-called “singular vectors” (non-modal finite-time instabilities), and developing stochastic generalisations of the model’s sub-gridscale parametrisations, in order to represent model uncertainty.

As well as making weather and climate prediction a more scientifically rigorous discipline, ensemble forecasts have added considerable additional value to weather and climate predictions. This can be made explicit using idealised decision-theoretic models where a user incurs a loss L if an adverse weather event occurs, but can mitigate against this by taking protective action at some cost C. When should the user take protective action? An ensemble forecast provides a much more valuable strategy for deciding when to take protective action (when the forecast probability of the event exceeds C/L) compared with the more traditional deterministic forecast.

Virtually all of the world’s primary meteorological services now run ensemble predictions. Indeed ensemble forecasting is now completely standard across the whole of meteorological science: from short-range weather forecasts one day ahead, medium-range forecasts two weeks ahead, seasonal and decadal climate forecast on timescales of years to decades, and predictions of climate change for the next century. Ensemble prediction has transformed the science of meteorology, making it the principal forecasting tool for both weather and climate in the 21st Century.

Profile

Web Site URL

http://www2.physics.ox.ac.uk/research/predictability-of-weather-and-climate

A brief Biography

Professor Tim Palmer CBE FRS is a Royal Society (350th Anniversary) Research Professor in Climate Physics (2010-present) at the University of Oxford where he leads a group exploring the predictability and dynamics of weather and climate. Tim has made a number of contributions to our dynamical understanding of the climate system, from the discovery of the world’s largest breaking waves in the stratosphere to uncovering the mechanism of long-term drought in the African Sahel. His research has consistently emphasised the role of nonlinear processes, e.g. in determining the climate system’s response to imposed forcing. Leading teams at the UK Met Office (1977-1986) and later at the European Centre for Medium-Range Weather Forecasts (1986-2010) where he was head of the Predictability and Diagnostic Division, Tim introduced a paradigm shift in the way operational weather forecasting is conducted: from best-guess deterministic prediction to probabilistic prediction using ensembles of integrations. As part of this he developed the concept of stochastic sub-grid parametrisation, around which much of his current research – e.g. on energy-efficient imprecise computing – is focussed. Tim has led two EU climate prediction projects (PROVOST and DEMETER): these pioneered the development of multi-model ensemble prediction for seasonal prediction. He is currently leading an EU Flagship proposal on the use of dedicated exascale computers for ultra-high resolution climate simulations. Tim has won the top prizes of the American and European Meteorological Societies, and the Institute of Physics’ Dirac Gold Medal whose previous awardees include Nobel Prize winners, and is a Fellow of the Royal Society and a Foreign Member of the American Philosophical Society. He led the team that won the World Meteorological Society’s Gerbier-Mumm International award for work on the use of seasonal forecasts for prediction of malaria epidemics in Africa. Tim has served as Chair of the International Scientific Steering Group of the World Climate Research Programme’s CLIVAR (Climate Variability and Predictability) project, was President of the Royal Meteorological Society (2010-2012) and has been a Rothschild Distinguished Professor at the University of Cambridge (2010). Tim’s PhD (1974-77) at Oxford University was in general relativity theory and he remains active in theoretical physics, particularly foundations of quantum theory. High profile lectures include the AGU Bjerknes Lecture, the Newton Institute Rothschild Lecture and the Dennis Sciama Memorial Lecture. Tim has sat on numerous UK Government Committees and in 2015 was made a Commander of the British Empire by Her Majesty the Queen.

Details of selected Awards and Honors

*Fellow of the Royal Society and 350th Anniversary Royal Society Research Professor
*Institute of Physics Dirac Gold Medal (Previously awarded to Nobel Prize Winners: first time awarded in the Geosciences)
*American Meteorological Society Rossby Medal (Top Prize) and Charney Medal
*European Meteorological Society Silver Medal (Top Prize)
*World Meteorological Society Gerbier-Mumm International Award
*President of the Royal Meteorological Society
*Commander of the British Empire
*Honorary Doctorate of Science University of Bristol
*Rothschild Distinguished Professor, University of Cambridge (during a period at the Newton Institute for Mathematical Sciences)
*Chair International Scientific Steering Group. World Climate Research Programme CLIVAR Project.

A list of selected Publications

(NB These relate specifically to the topic of my lecture, and are not representative of my full set of research output.)

Palmer, T. N., 2016: A personal perspective on modelling the climate system. Proceeding of the Royal Society A, 472, 20150772.

Palmer, T. N., 2014: More reliable forecasts with less precise computations: a fast-track route to cloud-resolved weather and climate simulators? Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 372, 20130391.

Palmer, T.N., F.-J. Doblas-Reyes, M. Rodwell and A. Weisheimer, 2008: Towards Seamless Prediction: Calibration of Climate Change Projections Using Seasonal Forecasts. Bull. Am Met. Soc., 89, 459-470.

Palmer, T.N.. A. Alessandri, U. Anderson and co-authors, 2004: The Development of a European Multi-model Ensemble System for Seasonal to Interannual Prediction (DEMETER). Bull. Am Met. Soc., 85, 853-872.

Palmer, T.N., 2001: A nonlinear dynamical perspective on model error: a proposal for nonlocal stochastic-dynamic parametrisation in weather and climate prediction models. Q.J.R.Meteorol.Soc., 127, 279-304.

Palmer, T.N., 2000: The prediction of uncertainty in weather and climate forecasting. Rep. Prog. Phys., 63, 71-116.

Palmer, T.N., 1998: Climate change from a nonlinear dynamical perspective. J.Clim.,12, 575-591.

Palmer, T.N., R.Gelaro, J. Barkmeijer and R. Buizza, 1998: Singular vectors, metrics, and adaptive observations. J. Atmos.Sci., 55, 633-653.

Palmer, T.N., J. Barkmeijer, R. Buizza and T. Petroliagis, 1997: The ECMWF Ensemble Prediction System. Meteor.Appl., 4, 301-304.

Palmer, T.N., 1988: Medium and extended range predictability, and stability of the PNA mode. Q.J.R.Meteorol.Soc., 114, 691 713.