This article was originally published by War on the Rocks on 26 January 2016.
Wargaming is enjoying a renaissance within the Department of Defense, thanks to high-level interest in wargaming as a way to foster innovation. However, for this surge of wargaming to have a positive impact, these wargames must be designed well and used appropriately. For decision-makers with limited wargaming experience, this can be a daunting challenge. Wargames can be deceptively simple — many do not even use complicated computer models — so it is all too easy to assume that no specialized skills are needed for success. At the same time, wargames are hugely diverse: interagency decision-making seminars that involve conflict without fighting, crisis simulations adjudicated by subject matter experts, and operational warfare in which outcomes are determined by complex computer models. For sponsors who may have only seen one or two games, it can be hard to understand the full range of wargaming possibilities and the common approaches that underpin them all. How can a sponsor discern whether wargames and the resulting recommendations are actually worthwhile?
Writing aimed at the sponsors of wargames and the consumers of their results has been slow to appear and overly focused on specific historical wargames. For example, Micah Zenko, Gary Anderson, and Dave Dilegge wrote about the weaknesses of Millennium Challenge and offered some lessons to be learned from this famously failed wargame. In contrast, Deputy Secretary of Defense Robert Work and Vice Chairman of the Joint Chiefs General Paul Selva illustrated the potential of wargames by highlighting the famous Naval War College wargames played in the interwar years.
While these famous historical cases offer a compelling narrative of what wargames can be at their best and worst, they cannot illustrate the full range of contemporary wargaming that the department should strive to achieve. For political and military leadership hoping to use wargames to improve decision-making, a better understanding of how wargames can be helpful — and the ways in which they can backfire — is critical. Furthermore, when only looking at a single wargame or wargaming in a specific historical period, it can be easy to draw conclusions that may not be broadly applicable. Instead, consumers need more general guidelines to help them sponsor wargames that will provide useful insights into their problems.
To bridge this divide, here are eight lessons for sponsors about how they can best use wargames to explore analytical questions:
1. Not all problems can be helpfully analyzed with a wargame: Wargames are extremely flexible, but they are not suitable to all problems. Wargames work best when used to explore a problem involving human decision-making in conflict and generate new potential solutions. That makes wargames particularly powerful early in decision-making processes when the nature of a problem is still unclear, and where wargames can suggest new frames or approaches to guide subsequent analysis. For example, games can be used during an emerging crisis to consider how best to move forward or as part of long-term planning to generate potential strategies and concepts for future study. Games played once stakeholders are reasonably sure of their approach are less helpful, as the hypotheses generated in such games enter the conversation too late to change decisions.
Games can also provide a sense of whether ideas are feasible. Playing out a concept under competitive conditions can reveal critical vulnerabilities and suggest interesting variations for incorporation into future study. However, wargames will never provide reliable, granular understanding of weapon or platform performance. While these specifics can be brought into game play through the use of models and simulations, the focus of wargames should emphasize human decision-making. A simple, but helpful rule of thumb: If the answer you are looking for is a number, a wargame is not the best approach.
Furthermore, because wargames involve data about humans making decisions in artificial environments, they do not validate when used independently of other analysis. Validation ensures that the results of a model or simulation are consistent with real world. Validation is important when looking at physics-based models that show how often a gun can be fired or how fast a tank can move over rough terrain. When looking at human decision-making, there are so many variables of context, personality, and experience that validation becomes considerably more problematic. Wargames seek to mimic as many of these factors as possible to ensure that they are truthful, but they will never reach the level of accuracy and precision usually implied by validation.
What is more, often in games looking at the future, there is no real-world data available about weapons or opponents that do not yet exist. As a result, we are much better off when we try to learn from wargames, rather than prove with wargames.
A good designer should help sponsors determine if a wargame is actually the best way to get the analysis they need. If the answer to every question is a wargame, wargames are probably being misused.
2. Wargames should have a specific and relevant purpose and objectives: Once you have determined that a topic is appropriate for wargaming, the first step of all wargame designs is to determine the purpose and objectives of the wargame. Like good commander’s intent, this guidance should make it clear what needs to be achieved in the wargame, while still giving the designers space to apply their experience to the problem. Good wargame designers work as closely as possible with sponsors to ensure they understand what the sponsor is hoping to learn.
Wargames can be used to better understand very varied issues, from “How can the interagency better address a national security challenge?” to “How will my adversary react to new technology?” However, wargaming such different questions will get very different types of information. Broad, open questions are likely to generate areas for future analysis. More focused questions will often generate more specific hypothesis. Make sure that your wargame designer understands which type of information will help you, and which will hurt. Empowering designers to build a wargame that hews closely to your preferences is critical for success.
3. Wargame design should be shaped to meet purpose and objectives: Because a wargame’s design is tied to its purpose, wargames will often look quite different. Good designers study a problem and design wargames that highlight important actors, the context and background of the conflict, and decision-making rules. At the same time, they abstract away those elements that are less important to the problem at hand to ensure players are not overwhelmed with extraneous details. This process is as much an art as a science, and is where a good designer will show his or her worth.
Wargames resulting from this process will look very different. For example, some wargames may focus on the interaction between key stakeholders’ interests and resources. Such wargames may require no computerized models to determine the outcome of decisions as options; decisions and outcomes emerge organically from the discussion between different players. By contrast, a wargame focused on combat technologies may employ complicated computerized models and teams of players representing opposing nation states. Other games use mechanisms from commercial games such as hex boards, dice, and cards — or combine several approaches. No single model of wargaming is inherently superior.
While as many design decisions as possible should be driven by the wargame’s purpose, logistics will inevitably play a role in design. Coordinating player schedules, finding appropriate venues, and scheduling sufficient wargame time all impose limitations on designers. A good designer will help you understand the consequences of these inevitable impediments on the wargame results so you can decide how to manage those tradeoffs.
4. Blue losing is a sign of a fair game and a terrific learning opportunity: Much of the power of games comes from their ability to tap into players’ competitive instincts. As a result, games work best when players have competing interests. Often this is thought of in terms of friendly and adversarial forces (frequently called the blue and red teams, respectively), but there are other types of competition that can be just as important. Friction between different military services or aspects of the interagency process can provide plenty of conflict without stepping outside the U.S. government. Coordination with multinational coalitions of allies and partners or private companies is increasingly important, but often lead to fraught relationships. Wargames can explore these tensions to good effect.
Regardless of the source of competition, for a wargame to be helpful it should be a fair fight. Wargames should offer each team resources based on a common set of assumptions. If one country has access to weapons that would take 10 years to build and field, other country teams should have a force based on equivalent projections. Departments should have realistic restrictions not only in terms of resources, but also authorities and permissions. These efforts provide a necessary baseline for developing credible analysis from wargame results.
As a result of this level playing field, sponsors should expect that their blue team will sometimes lose the wargame. Far from being a failure, these losses often offer the richest opportunities for analysis and learning. The success of the wargame should be measured in whether it suggests new opportunities or identifies barriers that can be addressed is subsequent analysis. After all, some of the best lessons may come from placing friendly forces in intentionally asymmetric situations, a Kobayashi Maru of sorts. As a result, the success of the wargame should always be based on the insights they produce, rather than the outcome of the game’s fictitious battle or bureaucratic debate.
5. Wargame design isn’t over when the game starts: Often adjustments in wargame design will be required during play to mitigate sudden logistical challenges or to respond to unfolding developments. Even well-designed and well-tested wargames will turn up surprising results that make them helpful in finding innovative ideas.
Just as wargame design requires tradeoffs between practical considerations to ensure the wargame achieves its purpose, so too should in-game design changes balance these requirements. Holding rigidly to the original wargame design can lead to slow play, unhappy participants, and ultimately less useful data for analysis. However, on-the-fly changes can also wreak havoc with interacting design elements and data collection plans. Both courses can lead to wargame results that are not useful or believable. Instead, wargame designers should make course corrections thoughtfully and explain what impact the changes will have on findings.
In most cases, the rationale behind changes should be shared not only with other analysts working on the wargame, but also with the players themselves. Within the wargaming community, Millennium Challenge is often cited as an example of what happens when the rules of a wargame are changed without clearly justifying those changes — players lose faith in the wargame’s designers and the results are invalidated. A commonly understood justification for changes can mitigate these risks.
6. Those who learn the most from wargames are those who participate in them: While wargame results should communicate major analytical findings, they cannot replicate the immersive experience of participating in a wargame. The process of assuming a role (particularly one other than your current position) can teach players an immeasurable amount not only about the particular scenario being wargamed, but also about the interests, capabilities, and problems of other stakeholders. While it can be difficult to carve out the time to attend, participation in wargames provides excellent experiential education and should be encouraged when possible.
7. Transparency in wargame results is critical to justify faith in findings: Too often wargame reports document only the major findings of the wargame without outlining its design methodology. While this approach may be appropriate for “quick turn” analysis that is consumed by a small group familiar with the wargame’s design, as reports circulate to new readers outside the initial group, it can be increasingly difficult for them to determine if the results are credible.
Including an explanation of the wargame’s design and why the analysis drew the results it did from play vastly increases the rigor of post-game analysis. When designers are transparent about the limitations of design and their potential impact on the wargame’s findings, wargame results are more credible and easier for others to interpret. With proper documentation, it is also possible for readers to determine whether the results of a wargame should be applied to other problems. These considerations parallel the concepts of internal and external validity from the scientific method.
This transparency is particularly important when wargame designers have a vested interest in the wargame’s results; clear, logical justifications serve to keep the process honest. Many wargame designers are contractors with inevitable incentives to please current and potential future clients. Other designers reside within concept development shops where there may be pressure to develop analysis that supports a particular “party line.” The best way to ensure honesty and defend against accusations of bias is to ensure the designers’ logic is documented so that others can check their work.
8. Wargames are most valuable when they are linked to a “cycle of research”: Often, wargames are treated as one-off events instead of incorporated in a “cycle of research” that includes more wargames, other types of analysis, and exercises. The latter approach has the benefit of ensuring that wargame results do not simply sit on a shelf, but rather drive further analysis that can test intriguing hypotheses or dig deeper into specific areas of uncertainty.
Combining games with other approaches can produce stronger results. Different research approaches and methods have different strengths and weaknesses. For example, an approach might provide very specific granular information about a particular historical event, but do little to help generalize findings to other cases. By linking such an approach to a follow-on study using a method that seeks to understand what other cases share the same characteristics, a careful designer can get the best of both approaches.
Linking wargames to follow-on work can also alleviate the pressure on any one wargame to answer too many questions by dividing up the problem into more manageable parts. Instead of one big wargame that tries to do a lot, smaller wargames, studies, and exercises can be tailored to particular parts of the problem. Results of these more focused games can then be synthesized in later analysis to produce more comprehensive findings.
Yet increasing the number of wargames, studies, and exercises has real costs. Without careful planning and integration, results from different events can create confusing, contradictory results that feed interoffice disputes rather than provide clarity to decision-makers. There are also limits on the number of wargame designers, the availability of qualified players and technical experts, and the capacity of senior leaders for consuming analysis that combine to make developing larger numbers of quality wargames more difficult. As a result, careful thought about how elements of the cycle will build on both themselves and parallel efforts is critical for successful implementation.
As a sponsor, understanding what wargames can and cannot provide is critical to making good use of limited resources. Working with designers to ensure that games are used to tackle appropriate, clear objectives helps designers to pick the best design approach. During the game, welcoming blue losses, accepting the occasional last-minute change in design, and participating as much as your schedule allows will enable you to learn as much as possible from the event. Finally, post-game analysis should not just report what happened in the game, but also why the game’s design supports those results and how follow-on studies will continue to explore the issues raised in the game. These steps will ensure games are used appropriately and deliver on their full promise.
Elizabeth “Ellie” Bartels is a doctoral candidate at the Pardee RAND Graduate School and an assistant policy analyst at nonprofit, nonpartisan RAND Corporation.
For more information on issues and events that shape our world, please visit ISN Security Watch or browse our resources.