Key Concept

Is your company about to launch a new product or service into the market? If so, you know how crucial it is to predict as accurately as possible how it will be received; otherwise, you are putting at risk precious time and resources engaged. According to this Idea, there is a way to make an intelligent forecast that involves a combination of human and computer judgement. Faculty from Judge Business School and IE Business School explain how.

Idea Summary

Robots are taking over our jobs! Ok, maybe that’s not quite the case just yet, but as technology continues to develop swiftly, there is no doubt that machines are getting smarter, cheaper, and - in some cases- more effective than humans. In several fields, analytical models are already heavily relied on for forecasting; take the use of automated trading, for example. But when and to what extent should executives base their decisions on computer outputs, and when should they complement them with their own judgment?

This is the central question addressed by Professors Matthias Seifert and Allégre Hadida in their latest study on judgemental forecasting. Based on an experiment involving predictions of pop music charts, they put forward that the best approach is relying on both computer analysis and human judgement. However, they do not suggest using a 50/50 mix; rather, computer analysis should be relied on more in unambiguous contexts. But in highly-uncertain contexts, the opinions of three experts should be averaged, with greater weight given to their combined judgment than to a machine’s results. In other words, three humans plus one computer will give you the best forecast.

Methodology: Over the course of 12 weeks, Seifert and Hadida asked 180 people to predict the Top 100 positions of songs on pop charts in Germany and the UK by both established and new artists. Just over half of these people were music-industry professionals and the rest were graduate students with no particular knowledge of the music business. They tested their hypotheses by trying various combinations of human and computer predictions about the songs, finding the following: in the case of known/established music artists, the computer provided a more accurate forecast; for unknown artists, human predictions beat computer ones. However, in both cases, combining computer and human predictions produced the best results.
The one notable difference was that in the case of established artists, the level of the humans’ expertise was irrelevant to get the best forecast using a combination of human and computer predictions. But in the case of unknown artists, expertise mattered a lot; the higher the expertise, the more weight would be given to the human predictions.

Business Application

These findings provide food-for-thought for not just managers in the music industry, but even those working in organizations where decisions are frequently made based on predictions of product success; for example, ‘cultural’ industries, such as publishing and cinema. If the product is a new, groundbreaking one - something customers have not seen before - a computer can provide valuable insights, but according to Seifert and Hadida, you should lean more heavily on the judgment of experienced people. This is where the ‘three humans plus one computer’ formula can and should be applied.