In the context of an increasingly uncertain world, the Bank of England commissioned former US Federal Reserve Chairman Dr Ben Bernanke to review their forecasting and related processes. This increase in uncertainty has made, and will continue to make, forecasting ever more challenging while increasing the importance of independent forecasts, which are not constrained by legal remit and/or political pressure. As the United Kingdom’s leading independent producer of macroeconomic forecasts, NIESR clearly needed to think about these issues and respond to the Bernanke Report. We did this via an ESRC-funded research project and today are releasing the three papers that form the output for that project.
We wanted to use this project as an opportunity to improve the delivery of our UK macroeconomic forecasts, enhancing our ability to deliver impactful analysis both now and into the future. Importantly, it also enabled us to think about how we communicate our forecasts to the wider public, especially considering the high degree of uncertainty surrounding such forecasts currently.
The project consisted of three strands of work – hence, three papers!
Macroeconomic forecasts provide a consistent framework for analysing the economy and informing policy decisions. However, their usefulness ultimately depends on their predictive accuracy. Our first paper carried out an evaluation of NIESR’s forecast performance, with a particular focus on GDP growth and CPI inflation, produced between 1992 and 2023 across horizons of up to ten years. We documented forecast errors, dispersion and formal tests of bias, then evaluated NIESR’s forecasts against a hierarchy of benchmarks – including a random walk, autoregressive models and a Bayesian VAR – using Diebold-Mariano tests. We found that our GDP growth forecasts show a significant positive (over-prediction) bias that strengthens with horizon, suggesting that we are apt to expect stronger trend growth in GDP than we have seen in the data. Our CPI inflation forecasts, on the other hand, show no significant bias. Our forecasts for inflation outperformed the benchmark models we considered, at nearly all horizons, but for GDP growth our advantage is concentrated in the short-to-medium term and reversed beyond around five years, where we tended to be over-optimistic. Overall, these results suggest that the benefits of structural modelling depend on both the variable being forecast and the horizon over which the forecast is being made.
The second paper – on fan charts and their alternatives – examined how to communicate the economic uncertainty around a forecast in a way that is easy to understand, while being based on strong analytical foundations. We first assessed how well our fan charts have captured the distribution of past forecast errors, finding that they have some problems with this. We then examined different ways of constructing fan charts, seeing how different measures of uncertainty can be used to adjust the size of the fan in our forecasts. We would argue that fan charts based on historical forecast errors remain a useful way of communicating the degree of uncertainty around a forecast and argue that the width of the fan can be adjusted based on survey measures of uncertainty to provide a realistic illustration of the potential magnitude of forecast errors. Finally, we looked at how, in principle, the fan chart can be skewed to reflect the balance of risks. However, we would argue that there is no convincing way of calibrating such an adjustment.
The final paper – on implementation of scenarios – showed how we can use scenarios to illustrate the effects of specific risks to our forecast (as well as the general uncertainty we have about how the economy works), and to communicate the effects of these risks in a way that readers will find straightforward to understand. Building on the Bernanke review, we evaluated four uses of scenarios: assessing policy interventions, illustrating specific forecast risks, addressing model uncertainty, and decomposing historical forecast errors. We would argue that scenarios are most effective when used to convey distinct, narrative-driven risks around a central forecast, offering clearer insights than fan charts alone in this case. This is the case even where fan charts are still useful for communicating general uncertainty. Overall, we think that narrative-based scenario analysis is a powerful tool for communicating macroeconomic risks.
This project was motivated by a simple but increasingly important question: how can an independent forecasting institution remain both analytically rigorous and genuinely useful in a world characterised by persistent uncertainty? Taken together, the three papers provide complementary answers. They show that understanding forecast performance, communicating uncertainty clearly, and using scenarios to explore risks are not separate challenges but different aspects of the same task: helping policymakers, businesses and the public make better-informed decisions in an uncertain environment.
Perhaps the most important lesson from the project is that forecasting should be viewed as a process of continuous learning rather than a search for a single “correct” prediction. By systematically evaluating forecast performance, refining the way we quantify and communicate uncertainty, and developing more effective scenario analysis, we can improve both the quality of our forecasts and the value they provide to users.
Looking ahead, these papers provide a practical agenda for future work at NIESR. We plan to embed the forecast evaluation framework into a regular programme of forecast assessment and model development, helping to identify biases and areas where alternative modelling approaches may add value. The findings on fan charts can be used to enhance the presentation of uncertainty in our Economic Outlooks, ensuring that uncertainty measures remain responsive to changing economic conditions. The scenario analysis framework offers a foundation for more systematic exploration of key macroeconomic risks, allowing us to communicate the implications of major domestic and global developments more clearly and transparently.
In that sense, the outputs of this project are not an endpoint but a starting point. They provide a set of tools and evidence that will help NIESR strengthen its forecasting practice, improve how it communicates uncertainty, and continue to fulfil its role as the United Kingdom’s leading independent macroeconomic forecasting institution in an increasingly complex and uncertain world.