Sunday, April 2, 2006

Managing game design risk: Part I

GDC was wonderful. Unfortunately, I’m the sort of person who is absolutely miserable at copying notes in any literal way. As soon as I started writing, I became distracted by a thread connecting several intriguing talks on prototyping, tools and production risk. The result was a very long essay. This is the first part.

Managing Risk
Much of game production is about managing risk. We partake in a highly complex process that juggles people, technology, new forms of interactivity and rapidly changing market dynamics. Developing a single game is a high stakes effort in which the likelihood of failure is dramatically greater than that of success. One false move and the development team can face financial ruin or heavy restructuring at the request of their publisher.

It is a fascinating optimization problem that drives many of the practices in modern game development. Here are some rough notes describing different forms of risk as well as my initial thoughts on techniques for mitigating certain types of risk.

  • Financial Risk
  • Requirements Risk
  • Execution Risk

Financial Risk: The big daddy
All product development consists of the act of producing and delivering something of value to customers. In return, the customers give you money. If this process produces profit, then you can develop more products. If it produces a loss, you are heavily encouraged to stop making such products and do something more profitable with your limited resources. Economics abhors waste.

The likelihood that you will lose money is financial risk. It is the invisible hand that makes all other forms of risk meaningful. However, simply looking at ‘making money’ does not tell us enough to optimize our processes. We also need to look at what we make and how we make it.

Requirements Risk: What the heck are we making?
Requirements Risk is all about making the right product for our customer. It is split into three interrelated categories. Naturally, any one of these risks can increase the project’s overall financial risk.

  • Design risk: Risk of not including the right features that create a potent value proposition for your target audience.
  • Scope risk: Risk of not doing enough or doing too much. If you don’t do enough, you increase Design Risk. If you do too much, you increase execution and financial risks.
  • Market Risk: Risk that there will be changes in the market place that invalidate your value proposition. Market risk is external to your project.
Of these three, design risk is the most fundamental with the other two acting as constraints. In order to mitigate requirements risk, you must choose the minimal right features for your product. This correct set of features changes over time as external events influence your target market.

Execution Risks: How do we make this?
Once you understand ‘what’ you need to do, you need to execute on your plan. At this point, four additional risks raise their ugly heads.

  • People Risk: Risk that you do not have the right people with the right motivation and methods of work to execute. A subset of this is creative risk, in which you fail to create a product that you, as a creator, takes pride in. Often creative risk can be in conflict with the constraints of financial risk.
  • Technical Risk: Risk that major technology that you rely upon will fail. The obvious aspects of technical risk are things like rendering engines, path finding, etc. In games, technical risk also fluctuates wildly with the addition of new game mechanics. Since we are dealing with psychological interactions with real customers, minor game mechanic changes have surprising results. Even the act of adusting a character's jumping distance may not ‘work’ since it can destroy the fun factor for the rest of the game.
  • Schedule Risk: Risk of delaying the product and missing a competitive window in the market place. Games are products with a shelf life and strong market cycles. When you miss your dates, you dramatically increase Market Risk for certain types of established genres.
  • Quality risk: Risk of errors. A product that solves all other factors, but includes bugs and other malfunctions.

Mitigating risk: A difficult problem to solve
You end up with a web of constraints with no easy solutions. There is a wonderful section of Douglas Adams book “Dirk Gently’s Holistic Detective Agency” in which the lead character manages to get his sofa stuck in the stairway going up to his flat. He creates a 3D computer model of the situation to figure out how to get it unstuck, but can’t manage it. Dirk Gently, the world’s lateral thinking poster boy, solves the problem in seconds. All he has to do is remove one of the walls. Unfortunately, the solution to moving the couch would destroy the entire house. Woops.

Modern game development often finds itself in a similar situation. Let’s take the fundamental problem of choosing the right features for our product. An obvious engineering path is to add more features. Bridge builders might call this over engineering, a time tested technique. That way, you’ll ensure that you make a product that solves everyone’s needs since it contain more than enough features.

In game development, the secondary effects kick in. By increasing your scope, you increase the need for people, technology, time and quality. None of these scale linearly.

  • When you add more people to a team the communication processes within a team change dramatically. A 2 person team can sit in a room together and talk through problems. A 20 person team needs defined managers and process. A 200 person team often spends the majority of each individual’s time simply maintaining group cohesion and intergroup communication. Each stage is distinct and requires major cultural shifts to succeed.
  • As features increase in number and diversity, the need for new technology development increases. As game mechanics are added, the very nature of the system and its effect on the customer fluctuates radically. Each new feature has a very real chance of destroying the value of the entire product.
  • More features require more planning time and uncertainties can result in large schedule fluctuations. These are compounded by technology and people risks.
When you optimize for a single risk factor, you’ll often have unpleasant secondary risks that bring down the whole system.

The spectrum of process complexity
These high risk, heavily interconnected systems finally are getting their due. Building games is not like engineering bridges. In fact, folks have started to notice that many traditional project management are counter productive. When someone attempts to scale the project by turning one of the dials, the non-linear risk increases in unpredictable ways and the product fails.

The first step is to realize that there are different types of production processes and each one requires radically different management techniques. Here’s a diagram from the Scrum methodology that describes how uncertainty in both requirements and execution creates multiple classes of processes.

  • Simple Processes: When both requirements and execution are quite certain, then the projects can be managed with relatively simple processes. Often a simple checklist does the trick. The repetative steps that a single worker performs on an assembly line is a good example of a simple task.
  • Complicated: When the requirements and execution get out of hand slightly, you end up with a project that can still be completed with your typical check list. However, you need to increase the number of rule and steps necessary to accomplish the task. Bridge building is a good example of a complicated task.
  • Complex: Many projects fall into the dangerous middle ground where requirements and execution is somewhat defined, but rife with multiple layers of uncertainty. In these situations, high feedback, agile processes let team steer their way towards ago despite high risk and uncertainty. Software development is a good example of a complex task.
  • Chaotic Processes: When both requirements and execution uncertainty is very high, projects tend to devolve into unmanageable chaos. Success arises as often from luck as it does from any real process.
Game development is intriguing because it contains elements spread across all areas of the process complexity spectrum.
  • Simple: Creating 2D art
  • Complicated: Creating 3D character models:
  • Complex: Writing Code
  • Chaotic: Creating new game mechanics
If you end up focusing on a shoehorning all elements of gameplay into a single process, you are bound to leave gaps in your competitive strategy. With that thought in mind, I’ll be posting a couple more essays shortly that use this conceptual framework to examine two common risk mitigation strategies.
  • Data driven development: Reducing risk by simplifying the process of game development. This technique focuses on reducing overall execution risk.
  • Prototype driven development: Reducing risk by embracing change and finding the fun earlier. This technique focuses on reducing overall design risk.

Take care

PS: Part 2: Data driven development is posted. Read it now.

Descriptions of Scrum


  1. Thanks for a great read. It's always a joy to read when people can put such eloquent words to thoughts.

  2. Anarchy doesn't really need to be chaos although it can seem like that. Anarchy only means it has no enforced vision that everyone needs to work towards.

    Take the internet for an example, I would say it is quite an anarchy. Everyone does what they deem most valuable to themself. Yet there is an order of things.