Sunday, January 21, 2007

Project Horseshoe Report: Building Innovative Games that Sell

The 2006 Project Horseshoe reports are out! In the place of my irregularly scheduled essay, I'd like to point you toward the collaborative report that I've been working on with our appropriately named "PlayDough." We dug deeply into the issues around funding innovative games and ended up producing 18 pages of revolutionary white paper goodness.

Click here to read the report. Now for a bit of commentary on the topic of innovation and business.

We are bad at designing profitable games


If you've read this site for some time, you'll notice that I keep returning to the themes of measuring success and rapid feedback cycles so that you can make necessary changes early. Unfortunately, it became very clear that most players in the game industry do not apply these lessons to the business of designing games. It isn't purely a publisher issue, nor is it purely a developer issue, but the overall issues are rather clear:
  • We don't know what makes a successful game.
  • We don't measure our games regularly from the customer perspective to see if we are going down a successful path.
  • We don't act on information that our projects will be market failures.


A Solution
You might expect a good deal of venting at one of these designer retreats, but everyone was far more interested in fixing the problem. The game industry isn't the first to run into these issues and the solutions we found used in other industries are surprising palatable. Our group ended up exploring the adaptation of the Stage Gate new product development process to game develpment.


Stage Gate is a well known options-based portfolio management technique that focuses on improving product launch success rates. It is particular appealing to the game industry because it allows companies to try out lots of different ideas ranging from highly innovative project to product line extensions and then guide the most successful ones toward a market launch. Hundreds of companies in other industries use this model and it has been proven time and time again to dramatically improve success rates, reduce design and execution risk and also reduce time to market. These are all good things. :-)

Low hanging fruit
My basic belief is that innovative products are a highly desirable and profitable component of any publisher's portfolio. However, in order to make this obvious to all the egos involved, you need a process that provides hard data and a way to mitigate risk/fear. The Stage Gate process does this with surprising efficiency.

It is also obvious that a competitive market eventually punishes stupidity. Ignoring new markets while releasing products with a poor chance of success only works until someone figures out that there are better ways.
The first party story has some obvious suspects and I know at least one forward thinking 3rd party publisher has already started rolling a version of the stage gate process out across their teams. It will be fascinating to watch the techniques we talk about in the paper inevitably take hold over the next decade.

If you work in any organization that deals with multiple retail game titles, you are likely to save millions of dollars by taking the lessons of this report to heart.
Perhaps that is a big claim, but one that is well justified by the historical results from other industries. The opportunity to become the savior hero in your organization is immense.

Kudos go out to everyone in the PlayDough group. We had the perspectives of independents, studio presidents, and publishers represented. Everyone is looking for solutions and everyone who worked on the report feels quite hopeful about the future. It is a wonderful thing to be able to point to business best practices that hold the promise of dramatically improving the industry as a whole.

take care
Danc.

6 comments:

  1. Wow, that's really cool about the playtesting group.

    It makes me wonder if there could be an Internet community group like that available for all of us indie developers.

    ReplyDelete
  2. I just posted my thoughts on the Project Horseshoe report. I don't think stage gates are really the answer, mostly because , as you write above, game development is an iterative process. Stage gates are not an iterative model: They have a series of fixed stages that a product goes through from idea to release with fixed exit criteria for each stage.

    I think something like the Mario Club in the first comment plus an iterative process at the high level would be much more useful.

    Anyway, there's the link if you're interested. :)

    ReplyDelete
  3. Appreciate the comments.

    Re: Mario Club. This sort of user testing points out one of the big 'success criteria' that the market leaders have known for quite some time. Games that play test well tend to be pretty good games. :-) It's a rather obvious observation, yet many companies still struggle with not having testable games until very late in their development cycle.

    The best testing setups tend to involve watching users directly, but there is still a lot to be gained from remotely recorded play sessions. Indie developers with access to existing customers (say from a previous game) have some useful options with beta tests, forums and remote logging.

    re: Joe's essay
    Appreciate the link. I cross posted this comment on your site as well, but thought it fit here also.

    You've made a leap here and fundamentally mischaracterized a stage gate process as just another name for waterfall development methodologies. One has absolutely nothing to do with the other. You can use stage gate with waterfall. You can use it with agile. I personally feel it fits agile development much better and you'll see that more evolved stage gate process have a striking similarity to many agile technique even though you find them in non-software industries.

    As you may know from my other essays, I'm a huge fan of various agile process and put a big emphasis on both prototyping and rapid iteration. The stage gate process maps quite nicely onto practices like Scrum or XP and brings a lot of common lessons to the table

    Start small: Do the simplest thing possible that gives you critical information. You may learn something.
    Don't put all your eggs in one basket: Try out multiple ideas early on. Since you can't know which ideas will pan out, it makes a lot of sense to try several and then invest further in the promising ones.
    Test regularly: The whole point of the stage gate process is to have formal testing and validation *throughout* the development process, not at the end.
    Understand what success looks like: Even the most rapid iterative process is useless if you are making random changes that do no improve the product. By understanding what success looks like, you can iteratively guide your products towards that that goal.
    Invest late, not early: Product development is a learning process. You might as well invest money as you discover success mechanics, settings, etc instead of dumping a wad on hand waving specs early on.

    It is also worth noting that just because there are big formal stages that are used for large scale portfolio management, there is nothing in the process that rejects the hundreds or thousands of iterative loops within each development stage. In fact, I'd recommend it since it helps teams meet the gate criteria more successfully.

    Stage gate has some serious potential for helping us create innovative product. It brings portfolio management out of the darkness of Skull&Bones green lighting committees and encourages that smart companies set aside money to build new-to-the world, high payoff products.

    It is worth taking another look at the concept and imagining how it might fit into the iterative work flow you so rightly target as critical to success in our industry. I'm guessing you'll be pleasantly surprised at how the two compliment each other.

    Appreciate the article and the opportunity to start a conversation. :-)

    take care
    Danc.

    ReplyDelete
  4. All the games released from the game industry could be successful if they'd just produce about 80% fewer of them. :)

    --Ray

    ReplyDelete
  5. I think you are confused with Mario Club. Mario Club isn't a playtest group, but the entire testing dept. NOA has the Mario Club which does the daily testing of project, and then there is the Tree House which goes into more detail and design. NCL might have a different system.

    ReplyDelete
  6. This looks strikingly like the good ol' spiral methodology of prototyping.

    ReplyDelete