Saturday, March 18, 2006

Never innovate halfway

Double Fine’s Psychonauts was an original game that by all indications did poorly in the marketplace. This has led to the odd comments from fence sitters (and misinformed publishers) that innovation is a failed strategy. Bad monkeys! Bad!


We often talk about “innovation” in product development as if it were a miraculous substance that always benefits a game. I also imagine a scene in an idyllic game development kitchen with a game designer wearing a chef hat. “This title needs a dash more innovation!” he cries and the sous chef scurries off to the back room to see if they have any left.

If only it were this simple. First, realize that there are different types of innovation and not all types are equally beneficial. There are many places we can start on this topic, but we should first start with the classics. Back in 1991, researchers Cooper and Kienschmidt wrote a paper that discusses classes of innovation and their impact on market adoption. The studies looked at the market success of three category of products
  • Low innovativeness products: Modifications to existing product lines
  • Moderately innovative products: Product line extensions
  • Highly innovative products: New to the world products.
Each category was measured on three factors:
  • Success rate: What percentage of new products broke even?
  • ROI: What was the average profitability of the products that broke even?
  • Market Share: What percentage of customer in the markets adopted the product?
By looking at the patterns that occur across many industries, we can gain insight into how to build successful innovative games in our industry.

Interpreting this data in the game world
Let's map this general product model onto the specifics of the current game market.
  • Low innovativeness products: Sequels or expansion packs
  • Moderately innovative products: Games that attempt to blend genres or establish new brands in established genres.
  • Highly innovative products: Games that attempt to create new genres
The results
The results of the study are fascinating. Common wisdom would suggest that as the product became more innovative, the success rate would drop off. After all innovation is directly correlated with risk, right?


Instead, the data shows a U-shaped curve instead of a downward slope. Highly innovative products are actually mildly more successful than their low innovation counterparts. Moderately innovative products on the other hand do the worst out of any of the categories. This pattern repeats itself in the other areas.

Less surprisingly, ROI is strongest for low innovativeness products and quite reasonable for highly innovative products. It is again miserable for moderate innovation products. Market share is also low for moderate innovation products.

The immediate lesson is that both low innovation and high innovation products can do quite well. If you end up in the middle, however you are in big trouble. Let's dig into the strange U-shaped curve in more detail.

Explaining the U-shape
There are lots of reasons why sequels and direct expansions are successful. Several include
  • Lower R&D risks. The game mechanics in a sequel are well-defined and can be reused. Generally, game sequels add more content and mild technological innovation. These are both well understood production areas that can be budgeted for and scoped appropriately.

  • Built-in audience. With an established brand, sequels sell to both old players and players who did not pick up the previous title. This results in higher customer sales per marketing dollar spent.
An alternate set of reasons explains why highly innovative games such as the Sims or Animal Crossing are successful despite not having the advantages of low innovation products.
  • Tapping pent up market demand: Highly innovative products tend to get a burst of adoption as word of mouth from excited users spreads the news that someone has finally released a game that meets their needs. This happened with Nintendogs, where many women became avid promoters of the game to their friends.

  • Less need for polish: Because demand is pent up, people tend to ignore many damning issues. For example, Brain Training’s limited set of mini-games quickly become repetitive. Yet people are willing to put up a flaw that would have destroyed a less innovative title. The title offers a unique, high value experience that cannot be replicated with substitute products.

  • Lack of competition: When you are first to the market with a high value product, you can often just breeze in and acquire customers with no worries about competition. The Sims managed such a feat with gamers who preferred to create and explore relationships. The game industry had been blatantly ignoring this massive audience and the Sims reaped the benefits of being the only one who saw the opportunity.
Finally, we get to the moderately innovative games. These titles are cursed with the worst of all worlds.
  • Higher R&D risks: When you try out new game mechanics it is much more likely that you need to spend copious amounts of time balancing the title. If you do not have this time, there is a good chance the game will be released with poorly polished game mechanics.
  • No built-in audience: A mildly innovative game satisfies no one. It doesn’t tap into a new market of underserved users. It doesn’t appease existing fans of particular title or brand.
  • Strong competition: Often moderately innovative products are classified as part of an existing genre. The genre addicts have their favorite title already so why purchase a new one?
Where would Psychonauts fit into this mix? It is a competently executed, traditional platform game wrapped in a wonderfully quirky exterior. This rings some warning bells. It is more of the same, but different enough to be scary. It falls into the no-mans land of not familiar enough and not innovative enough to make a mark on the industry.

One step further
Psychonauts also points us in the direction of identifying the factors that matter less than we might expect when creating an innovative game. Psychonauts poured incredible amounts of blood and sweat into creating a delightful story, filled with new and interesting characters and sparkling with humor. All these elements were highly original and yet the title overall was not innovative.

These are not primary differentiators in the marketplace. In general, the game mechanics of a title and the ties into real world interests determine the value of a title. Well executed plot and characters merely support and augment the underlying game mechanics. In a novel, the story is the value. In a game, the activity that the player engages in determines the value of the experience. These are two very different value propositions that you confuse at your own risk.

If your core game mechanics are recycled from another genre, slathering a layer of original (and very expensive) content on your title will do nothing to push it into the realm of high innovation. It unfortunately, might be enough to push you out of the safe zone of comfy sequels.

Conclusion
If you are going to make a sequel, realize that more of the same is actually a good thing. You are in the gaming equivalent of the shampoo business, producing the same old bottle of Finesse over and over again. Sure, you can mix it up with a little sticker that says “Now with stronger cleaning power”, but don’t get too many ideas about changing the world. That isn’t what your audience wants. Give them more of the same: more levels, deeper stories and prettier graphics.

If you are going to be original, make a title with original game mechanics that taps into an underserved audience. Knock your title out into left field. Don’t look at other games, they aren’t your competition. Instead look at other activities outside of gaming. Don’t worry too much about plot, graphics or the polish of your title. Instead focus on generating deep value with people who don’t even think of themselves as gamers.

Never go half way. Don’t say “It is a shooter, but you fire ducks instead of bullets.” This is generating a half-assed value proposition by addition or exception. It is the sort of innovation that kills a game in the marketplace. You confuse the customers and suffer from comparison to existing products.

Developing a game title will often consume years of your life. Making a game that is only ‘moderately innovative’ simply is not worth the effort. Each project must choose its focus.
  • Are you a craftsman who lovingly polishes an established genre?
  • Or are you an innovator who creates new genres?
If you fail to chose, you risk being stranded in the no man's land that lurks between the two strategies.

Take care
Danc.

References
Comparison of first month sales for several titles: Pyschonauts: 12,000 units. God of War 200,000. Halo 2: 3.3 million. As a side note, 200,000 units for God of War are not great numbers considering the budget. They’ll cash in on this newly launched brand only if they do a sequel.
http://www.cs.northwestern.edu/~hunicke/blog/index.php?p=41166

More tales of Psychonauts
“According to the most recent NPD figures, Psychonauts has moved nearly 51,000 copies on the Xbox, not quite 23,000 on the PlayStation 2, and a little more than 12,000 on the PC.

Developed by Grim Fandango creator Tim Schafer and his team at Double Fine Studios, Psychonauts is something of an industry-standard cautionary tale about innovation. Psychonauts is about the story of a young child stowaway at a summer camp for psychics who must enter into the minds of his fellow campers and the camp counselors, and it was originally planned as a high-profile Xbox exclusive with Microsoft itself as publisher. However, Microsoft dropped the project in March of 2004 with no reason given.

Schafer shopped Psychonauts around for months before finding a new publishing partner in Majesco. The game was released on the Xbox, PC, and PS2 earlier this year, and met with critical praise and consumer apathy. Majesco (and a number of analysts, Pachter included) had great expectations of the game's retail performance. When the sales didn't materialize and another high-profile Majesco title flopped in Advent Rising, the company lowered its projected revenues for the year by a third, the CEO Carl Yankowski resigned, and a number of shareholders sued as the stock plummeted.”
http://www.gamespot.com/news/6141519.html
Posted by Daniel Cook at 12:31 PM
Labels: , ,

25 comments:

  1. Does the data you cite include all the highly innovative products that failed to find a market? Is it that it actually just averages out since the highly innovative products are more likely to become breakout hits?
    ReplyDelete
  2. The more I think about this study, the more it sounds like nonsense.

    Word of Warcraft falls squarely into the Medium Innovation model, using either your cross-genre reasoning or your product line extension definition. Everquest too; it's pretty much just a Diku with 3D graphics, no reason it should have done better than UO.

    Nintendogs was pretty much an upgraded Tamagochi (sp?), no new genre there. And am I the only person who remembers Little Computer People? The only person besides Will Wright, that is.

    Besides the fact that a number of counterexamples spring to mind, the "moderately innovative" category is far too subjective.

    Maybe if I had a look at the guts of the study the conclusions would make more sense.
    ReplyDelete
  3. The reference to the study was in a book that I've mentioned before called Winning at New Products by Robert G. Cooper. You can find the chart on page 18. The study it references is "The impact of product innovativeness on performance," Journal of Product Innovation Management 8 (1991): 240-251. E. J. Kleinschmidt and R. G. Cooper.

    I need to order the exact study since I've been unable to find full text elsewhere. The day Google crawls every scientific journal will be a happy one. In his other studies he tracks success rates of all projects, both failed and successful ones in order to avoid the inevitable 'success bias' that you sometimes see in many entrepeneurship studies. One factor that lets him do this is that he is working primarily with large companies that track projects from inception.

    So success rate should include failed products. Whether there is a major difference between the median and the mean of these numbers due to skewed distributions I do not know.

    Once you get into smaller companies, it becomes much more difficult to avoid the success bias. Many companies will begin and fail before they even show up on the radar. I make the assumption that many (though not all) of the factors that result in large company project successes are also important in small company successes (resources, cross functional teams, etc).

    It should be said that these surveys do not include game industry specific data. As such they shou be taken with a grain of salt since every industry has its own quirks. I come at it from the perspective that we have to start some place and a broadly based study of other products gives us a thought provoking reference point.

    take care
    Danc.
    ReplyDelete
  4. This is a very simple model intended to encourage discussion. It is guaranteed that it will not work with all cases. :-) Like many of the ideas posted on this website, they are most useful in helping you look at problems from a new perspective, not as the solutions to all gaming's problems. :-)

    By my definition of genre, Nintendogs certainly occupies a new genre. It is only peripherally related to Tamagotchi. It has different game mechanics, runs on a different platform, has a different input device and is targeted at a different audience. So it fits nicely in the 'highly innovative' category.

    Where does WoW fall? That is a very good question. It obviously provided a high value experiance with a strong brand. Yet it certainly jumped into a new genre. You'll see Nintendo do this same trick a lot with Mario. You could easily argue that some brands are powerful enough to act as a unique competitive advantage, especially in relatively new marketplaces like MMO or Console launch titles.

    It is also worth noting that studies deal in statistics. When you see a value for ROI, that is an average. There are some that do much better and some that do much worse. One or two exceptions rarely will prove or disprove a study.

    At their best, these sort of models help you play the odds. They do not guarantee success or failure.

    take care
    Danc.
    ReplyDelete
  5. The day Google crawls every scientific journal will be a happy one.

    You might try Google Scholar.
    ReplyDelete
  6. What about Earthbound?
    I believe you could describe it as a "competently executed, traditional" snes rpg "wrapped in a wonderfully quirky exterior". It could also be accused of pouring "incredible amounts of blood and sweat into creating a delightful story, filled with new and interesting characters and sparkling with humor", while the underlying game mechanics were 99% borrowed from other console/japanese rpgs.
    I believe it was quite successful?
    I'm not trying to dismiss your theory, I was just wondering what your thoughts might be regarding this particular case.
    ReplyDelete
  7. This comment has been removed by a blog administrator.
    ReplyDelete
  8. I would place WoW in the low innovation category. If you look at it from a series' perspective, going from an RTS to an MMORPG may be a departure, but just looking at WoW itself, it doesn't things that much differently than other MMO's, it just does them better. I wouldn't call using Warcraft's story as a backdrop all that innovative, that's just taking advantage of brand recognition.

    One addition I would make as to why highly innovative titles are succesful is the lack of standards to compare it with. If the highly innovative game lacks a feature, people see it as what they can do better the next time around (because if it's successful, a sequel is inevitable). If a non-innovative game lacks a feature and another game has it, people see it as what this game did wrong. In other words, people are less likely to hold a mistake or absence of a feature against you if your game is the first of its kind.
    ReplyDelete
  9. I see WoW as an interesting comparison point and possibly an outlier. When applying this concept to a game type.
    Psychonauts to platformers is very much like WoW to MMOs. Both bring an extremely stylized sense of play and environment while offering little innovation onto of the genera. Autoassaoult would be an example of a innovative concept to MMOs
    Where as Nintendo's approach overall is a good way to showcase this theory. They attempt to create a new niche environment followed up by a series of improvements on top of it.
    ReplyDelete
  10. I think this is the best article you've written so far. Thank you so much for writing what I was having so much trouble quantitizing about Psychonauts; I loved the game, and was so happy to have borrowed it from my friend and played it, and it had such wonderful characters and settings... but I just never got around to needing to buy it, the way I did with Katamari Damacy, or Shadow of the Colossus, or Rez.

    I think the difference between low and medium innovation is sometimes subjective, too; FFX did well overall because it was more of the same for all the FF fans, and thus a low-innovation title. I had been "out of the loop" since FF7 when I played it, though, so I approached it from a medium innovation perspective, and was disappointed when all I found was tired mechanics wrapped in super-pretty cutscenes.

    Likewise, when I found an old episode of consolevania where they gave a rave review to Bujingai, I ran out and bought a copy, only to conclude that it was a choppier version of God of War. Before God of War came out, I would've given it a rave review too, but afterwards it just felt "done better". I had the exact same reaction to Ocarina of Time and Half Life when I played them after their sequels, as well.

    I think this is also why "our kids" don't "get" older video games; we loved them because they were innovative, but now there have been countless games in the genres they pioneered.

    Of course, now you've gotten me worried that Pretty Pretty Bang Bang won't be innovative enough. I guess I'll just have to keep soliciting feedback from you until it is. :)X
    ReplyDelete
  11. Linda: I'm pretty sure, unfortunately, that Earthbound did poorly in the US, despite being packaged in an oversized box with a t-shirt. Hence no English version of Mother 2 (although hopes are high that we'll get Mother 3, which is currently in development).
    ReplyDelete
  12. Psychonauts was one of the few platform titles I've ever enjoyed since they went 3D, along with Mario 64 DS, Kingdom Hearts, and (perhaps oddly) Spyro the Dragon 2. I love the pacing, the detail, the difficulty curve (with the exception of the annoying last level), the bonus placement, the ratcheting, and pretty much everything else about. But when I make other people play it, they just say, "Eh, it's good, but it's nothing special."

    Psychonauts is, if anything, TOO brilliant, as what it does well is more in what it DOESN'T do than in what it does. I think people without any interest in game design probably don't realise the sort of game it could have been.

    Which makes me wonder if the key isn't in how MUCH innovation you have... but in what type. My experience would be that a general incremental innovation gets you nothing... but one single brilliantly executed narrative or mechanical innovation at the core of the game can catapult you to success and induce players to overlook any number of other flaws.

    But it needs to be done well. Take Forbidden Siren (Siren in the US). That's an innovative game that could have taken the world by storm... if it had just executed what it was doing better. Instead, its non-sequential narrative was confusing, the missions and episode system was poorly explained, the "sightjacking" mechanic was frustrating and restrictive, and the game on the whole was so hard you had very little opportunity to freely enjoy what it did well.

    Anyway, thanks for your great post on this modern industry conundrum. I just hope Psychonauts doesn't end up as the mythical "what not to do" for years to come.

    BTW - Little Computer People! OMG! I thought I was the only person who ever owned a copy of that game! I still miss the little guy... the way he used to sit at the typewriter and write me letters... Gawd bless 'is little heart... *sniff*
    ReplyDelete
  13. I think this article ignores the MASSIVE impact of game marketing and hype. Big franchise sequels get hyped to high heaven, while innovative titles from small companies barely get any media attention. Psychonauts didn't get much attention until well after the fact. Games like Halo 3 are so hyped it's unbelievable. I think if you want to look for a correllation between innovation and sales, you need to take into account how well the games were marketed somehow.
    ReplyDelete
  14. I tend to believe EarthBound was a market failure too, but it is the English version of Mother 2. What we didn´t get was the DS (or was it GBA?) remake Mother 1 & 2.
    ReplyDelete
  16. Hunty: You're right. I shouldn't have confused the two. Mother2 was a big success, Earthbound was a financial failure.
    But this brings me to another question: does the Low/Medium/High Innovation thing only hold true for the Western market?
    The more I think about it, the more it seems that everybody (gamers and developers, that is) is underrating the importance of theme and story. Yes, good gameplay is important, but only to gamers. As far as my elderly female relatives (the extreme case of non-gaming/casual audience) are concerned, the difference between a game with orcs/zombies/thong-clad vampires in it and an orc-free game is enormous. A diference between one set of gameplay elements and another is well … irrelevant, I guess. I'm not saying this is a good thing, but it is a fact that many (western) developers ignore. "Creating a delightful story, filled with new and interesting characters and sparkling with humor" might not be such a bad idea after all.
    ReplyDelete
  17. Linda: I certainly agree that this phenomena may be very much culturally related. However, I think at least in the case of Earthbound vs. Mother 2, their financial failure/success could be defined by their respective markets, in reletive terms. In the U.S., Earthbound may have fallen squarely in the middle category, accounting at least in part, for its poor sales. Mother 2, in its native Japan, could have certainly been considered more toward the 'low innovation' end of the spectrum. In a market where crazy robot fighting rpg dating sims are commonplace, I don't think this is much of a stretch.
    ReplyDelete
  18. "Which makes me wonder if the key isn't in how MUCH innovation you have... but in what type. My experience would be that a general incremental innovation gets you nothing... but one single brilliantly executed narrative or mechanical innovation at the core of the game can catapult you to success and induce players to overlook any number of other flaws."

    Greg T. pounded the nail in the board for me, as I was thinking about games like Resident Evil 4, touted as highly innnovative yet I get the sense that that it hasn't done anything really innovative. RE4 just took a couple features here and there from other games and reworked them into a tired franchise, thereby instilling a sense of overblown innovation. Don't get me wrong. I still believe it to be a great game, and it took these foreign elements and really made good use of them. But would RE4 really be lauded for its "innovation" if used outside the franchise?

    Video game enthusiasts often clamor for change and innovation, but as successes like God of War and Resident Evil 4 will tell you, they like it in small doses. Things that are too innovative are often presented with a much smaller window for being in the right place at the right time. Franchises and games with overall story appeal to the major market will have a much larger window. One such innovative title that did just happen to be at the right place at the right time was Katamari Damacy.
    ReplyDelete
  19. Wow, this makes a lot of sense. I often wondered why beyond good and evil did so poorly. It was a heartwarming tale, and really well-developed. But playing it gave people the impression that "meh, it was a simpler version of zelda"

    Same applies for prince of persia, sands of time. I think it had been too long since a POP game came out, people had forgotten what it meant to be a POP game, and they just saw it as a platfomer/brawler hybrid. Whithout realizing that prince of persia was a franchise that helped create that genre.
    ReplyDelete
  20. Kids want games that will make them feel like they've got a friend in the world. They want to feel popular, and the envy of all the others who play the same game. They want something that distinguishes them from the parents, and they want something that challenges them.

    --Ray
    ReplyDelete
  21. *Very* interesting!

    I am a game design student at the University of Baltimore and this reminds me of how many times my peers have come up with ideas that sound like what you described as moderately innovative. It always goes "it's like (pick a game) but instead of X you have Y." and somebody else goes "now *that's* crazy."

    It makes me want to look at the concepts I've come up with and see which ones step too close to that.
    ReplyDelete
  22. I randomly came across an intriguing review of Psychonauts last fall so I put it on my Christmas list. I have to say it's a cool game although at 57 I don't have the eye/hand coordination it takes to quickly get through some parts of it. I've considered hiring a 10 year old to get me past the tougher bits.
    ReplyDelete
  23. Jediborg: I'm not entirely sure if Prince of Persia: The Sands of Time could be deemed a failure since it did spawn two sequels. The franchise must be making money to warrant that happening.

    The game did innovate on certain aspects, especially with regards to character death. Being able to just go back in time for a few seconds rather than outright dying was pretty indenious. Sadly, I wish that the two sequels didn't have to go the dark and gritty route that so many other games have going for them. The whimsical and wisecracking nature of the first game was at least 1/4 of the fun.
    ReplyDelete
  24. Danc, great post. I didn't realize you were going to be at the GDC, else I would've come claimed a beer.

    If you haven't read it yet, I'd suggest checking out "Get Back In The Box: Innovation from the Inside Out", by Douglas Rushkoff. He talks a lot about the importance of social currency, which I think helps explain the results regarding the highly innovative titles.

    Social currency is essentially what you get from a product that you can then use in a social context, sort of like the "water cooler effect". I've definitely told more people about Brain Age than I have about Psychonauts, because you can get the new innovative idea of Brain Age across in about 20 seconds (and they immediately get it), whereas it would take me much longer to tell someone the plot of Psychonauts. All I can do is tell them THAT it is good -- at that point I'm just convincing them, not really transfering any social currency.

    Anyway, long post short, read the book.

    - Jeb
    ReplyDelete
  25. It really surprises me that Psychonauts wasn't a hit. But then again, maybe it shouldn't -- it was punished for being too good. I think it's one of the best games I've ever played, and certainly one of the most beautifully designed and executed. I felt the same about Grim Fandango, back on the PC. Psychonauts was innovative; that's what elevated it about the standard platformer. The writing was sharp and funny. It's the only game I've played on Xbox that I wish would have a sequel. I just hope whatever Schafer comes up with next is given a fair chance. He's my fave game designer. It would suck if he was punished for being too good.
    ReplyDelete

Subscribe to: Post Comments (Atom)