4d.
Mixing Quantitative and Qualitative Methods

Overview

We are now turning to approaches that mix different methods, and thus rely on both quantitative and qualitive methods. These approaches are often elaborate, an outcome of the diversity of methods on which they rely. This section introduces three approaches and one decision-making framework, strategic planning, incorporating approaches and methods.

The Delphi Method

The Delphi method combines qualitative surveys with an elaborate and rigid methodological process which often involves quantitative steps. The name of this method refers to the Greek oracle we have discussed previously. The Delphi method was developed in the US in the late 1950s by the RAND Corporation, originally to explore military scenarios. It has since been relied upon extensively to forecast different aspects of the future. While it was adapted over time to different circumstances, the fundamentals of the Delphi method have remained the same. The Delphi method is probably the technique that is most often associated with futures studies, because it was developed specifically to explore future-oriented themes and, second, because it has been extensively used.

Delphi is a version of survey analysis. It consists in a repetitive questioning of respondents selected for their knowledge and expertise. It relies on a panel method and a pretest, post-test design and interrupted time series. As shown in Figure 4.5, there are nine steps to the Delphi method:

1) Specify the topic to be investigated. The topic is an aspect of the future. Delphi can be used to forecast this aspect of the future, formulate visions of a preferred future and/or consider means of achieving this vision.

2) Construct the questionnaire. The questionnaire should be attuned to the purpose of the Delphi investigation. The number of questions should be limited to allow respondents to develop their perspectives on each point that is raised in the survey.

3) Select respondents. The selection of the respondents is generally targeted at experts in a given field. Unlike the ethnographic surveys, it is directed at people who have expert knowledge of the questions under investigation. Because of this focus on experts, the number of respondents can be limited. While an ethnographic survey of the general population can involve large numbers of respondents because many different experiences and interpretations will be recorded, a survey of experts on a narrowly defined topic will soon reach “saturation”. Saturation happens when respondents start repeating each other’s statements. Anyone who has carried out survey research work with specific categories of respondents has run into the saturation phenomenon. In most cases, a survey of twelve experts is considered to be sufficient to provide diversity of views while avoiding saturation.

4) Conduct the first round of questions. These questions are generally formulated to elicit relatively long written (or sometimes spoken) answers.

5) Measure the responses to the first round of questions. Although the Delphi method is classified here as a qualitative survey, it can include quantitative steps. While answers to the initial round of questions generally invite respondents to provide statements expressing their knowledge and opinions, this information can be quantified for the next stages of the Delphi investigation.

6) Create report on the first round. The information emanating from the first round of questions is organized in a short report that is sent back to the respondents. The identity of the respondents is anonymous so as not to bias the reactions of the experts, some of whom could be influenced by the views of prestigious members of the panel. This report emphasizes areas of consensus and discord, asking respondents to adjust their responses in light of the aggregate results of the first round of answers. The implied purpose is to achieve consensus on the questions of the survey.

7) Conduct additional rounds of questions. The search for consensus leads to one or more additional rounds of questions. In each case respondents are asked to provide their feedback to the answers of the panel of experts. At these stages of the survey, the questions can be formulated in a way that invites Likert-scale answers to facilitate quantitative analysis.

8) Analyze results of the last round. Measurement and analysis of the information provided by the last round of questions.

9) Write the final report.

The Delphi method provides ways of achieving consensus when forecasting the future. In this sense, it can be useful to policy makers as it presents areas of expert agreement on policies. The same can be said of the application of Delphi within the business world. The method has, however, been criticized for sacrificing diversity of views to its effort to arrive at a consensus. As the future is inherently unpredictable, a consensus of experts may be proven to be wrong while it can be an outlying view that materializes.

First-Hand Experiences with Delphi Surveys

I have taken part in a number of Delphi surveys. In all cases, I was invited to participate and after accepting, was sent the questionnaire. A few weeks after sending my answers I received the compilation of how the panel had answered the questions. Of course, in all the surveys the participants were anonymous, so I had no sense of who else was participating. The nature of the second round varied according to the survey. In some cases, I was invited to rank on a Likert scale statements representing the view of the panel. In other cases, I was asked to write a few paragraphs as a reaction to these statements. In most instances, there were only two rounds of questions. Asking written responses for more than two rounds can be taxing for respondents, possibly causing a decline in response rates. And I would add that as far as I remember, I was never made aware of the final reports of these surveys.   

Gaming

The purpose of gaming is to verify how people will react in given situations, which may be replicated when making decisions involving the future. Games are thus a simulation of reality. They are especially well-suited to the exploration of trade-offs people would make in different contexts.

For example, one could conceive of a Monopoly-like game that would verify what aspects of a house are most important to potential purchasers. Players would be given the same amount of money and they would be required to bid against other players on different houses, each with different sets of attributes. Such a game could also be used to observe the competitive behaviour of the players and consider the strategies they take. In the fields of mathematics and economics, game theory is generally applied to find out how people strategize to optimize their interest in different situations.

Gaming has been digitized. For example, SimCity allows players to build a city, as if they were both urban developers and planners. They must balance the amount of housing, industry, offices, institutions, greenspace, while providing the required infrastructure and services. The speed at which development takes place and the balance between land uses must be calibrated with the infrastructure and services that are affordable according to different levels and forms of urban development. Built into the game are fiscal considerations municipal administrations must heed in order to avoid stagnation, decline or bankruptcy. The assumptions underlying this game share many similarities with the urban dynamics of Jay Forrester.

Social Experiments

Social experiments create laboratories of real-life situations to simulate the behaviour of people in circumstances that could happen in the future. The problem with such experiments is that they are expensive. They require the participation of a sufficient number of people to be representative of the population or of a sub-group. Then, conditions must be created to reproduce the circumstances to be tested. Their high cost and the complications of setting them up explains why reliance on such experiments is unusual.

Guaranteed Minimum Income Experiments

There have been a number of experiments about the effects of a guaranteed minimum income, also referred to as negative income tax. Such programs are generally supported by the left of the political spectrum because providing resources allowing the poorest people to live with some dignity. These programs are thus seen as reducing the inequality of society. Interestingly, some people on the right of the political spectrum have also voiced their support for guaranteed minimum income. For example, Milton Friedman (1912-2006), a strong proponent of neoliberalism and monetarism, was favourable to such an approach because it made it possible to dismantle the bureaucracies administering different welfare programs and, in the process, promote personal choice and individual responsibility.

Between 1974 and 1979, the Manitoba Annual Income Experiment took place. It consisted of a randomized trial in Winnipeg, rural Manitoba as well as a pilot project in Dauphin. Participants were paid an unconditional annual income. The guaranteed income was reduced in different proportions according to earned revenue in order to measure the disincentive effect the program would have on the willingness to work. The experiment did reveal a modest disincentive effect on work of the guaranteed income. This effect was for the most part limited to parents with young children. The program came to an end when Progressive Conservative governments took power in Manitoba and at the federal level in 1979. Although considerable data were collected, no final report was written because by then guaranteed minimum income was no longer in tune with political priorities.

There was a short-lived experiment of this nature in Ontario in 2018. The experiment involved 4,000 participants from Hamilton, Brantford and Brant County, as well as from Thunder Bay and surrounding areas and from Lindsay. The experiment came to an end when a change of provincial government took place in 2018. The Ontario experiment was too short for it to generate useful findings.

Strategic Planning

In the 1960s, concern about the future became an inherent component of the decision-making of organizations in both public and private sectors. Many such organizations adopted formal planning frameworks to plan for the future. These frameworks were generally variations on the strategic planning model. In some organizations, units were set up to carry out the research and management related to strategic planning.

Strategic planning originates from the military, where a distinction is made between tactics and strategies. For example, tactics can refer to the movements of troops during a battle whereas strategies can concern the type of armament being used by an army. In the context of non-military organizations, there are a few essential steps to strategic planning:

1) Mission and purpose: an organization needs to define its mission and its purpose. It must situate itself within the hierarchical structure in which it operates in the case of public sector organizations, or their competitive environment for businesses.

2) Objectives: organizations need to define their objectives and how they intend to reach them.

3) Future scan: organizations need to scan the future in anticipation of the conditions to which their strategic objectives will need to adapt. At this stage, the strategic planning process may engage in a forecasting and future visioning exercise, which will draw on one or more of the approaches we have discussed in this module.

4) Strategies: formulation of strategies that blend the objectives of the corporation, the scanning of the environment in which it operates and expectations about the future.

5) Action plans: adopt and implement action plans.

6) Monitor: monitor the strategic planning process and its outcomes for the organization.

7) Iteration: As strategic planning is supposed to be an ongoing process, this leads to a return to the first step and a repetition of the subsequent steps.

In reality, strategic planning ran into problems. The setting of functions related to strategizing, forecasting and visioning within one unit, had the unintended consequence that other parts of organizations paid less attention to these matters. It soon became clear that concern about strategizing and thinking about the future needs to be pervasive throughout an organization. In some cases, the units specializing in strategic planning expanded and produced abundant flows of information, often overlooked by the decision-making departments of the organization having to deal with day-to-day crises. And, given the uncertainty of the future, many efforts devoted to strategic planning proved to be maladapted to emerging conditions and therefore a waste of time and organizational effort. Organizational actions guided by strategic planning were a total failure when this planning generated forecasting that proved to be wrong.

Case Example

For example, imagine an airline corporation that, engaged in a strategic planning process, decided in 2018 and 2019 to borrow large sums to double its fleet so as to provide inexpensive services to tourists attracted to previously unknown Asian, African and South American destinations. This strategy was based on the extrapolation of an ongoing rise in air travel and the taste of millennials for new destinations as revealed in market research. All seemed to work well until the COVID-19 pandemic. As the airline was based in a tax haven and could not benefit from substantial government subsidies, bankruptcy became the most likely outcome.

Strategic Planning vs. Intuition

In 2013, Henry Mintzberg, a McGill University expert in organizations and management, published The Rise and Fall of Strategic Planning. The book is about the gap between the rigidity of strategic planning processes and the time needed to absorb the abundant information they generate, on the one hand, and the need for organizations to respond rapidly to shifts in their operating environment, on the other. Henry Mintzberg opposes the incapacity of strategic planning to respond to rapidly changing circumstances and their frequent unpredictability with successful CEOs’ ability to draw on their intuition when facing unpredictable circumstances. The book concludes that the intuition of CEOs provides more flexibility and superior outcomes that a clunky strategic planning process. Intuition is described in this book as the capacity to store in the mind a wide range of experiences and the ability to retrieve rapidly from this knowledge lessons that can be useful in rapidly emerging circumstances.