Sunday, May 29, 2005

How small game team efficiency compares to other media

Often people who deal with Serious Games ask very different questions than people in the game industry.
  • Game developers, driven by intense competition and the urge to have a king of the genre title will typically ask "How can I make the best game possible, no holds barred"
  • Business instead as "How much does it cost to get a basic job done?"
You end up with productivity comparison charts that look like this:

Getting the job done
The reason is pretty straight forward. Often simple fact that you have a system in place will bring the company benefits over not having the system. Some repair training is better than no repair training program. Business decisions makers focus on creating an improvement over the current state of the company. The subtle quality elements that are key determining factors for the success of games are far less important.

The most blatant example of this is the training world. Many (though thankfully not all) groups value quantity over quality. Creating thousands of pages of 'training' that detail procedures that few people will ever use or learn is preferable to creating learning experiences that actually teach.

Let's put some numbers around this discussion...

HPD for Training
In a previous post I talked about HPD, a metric that allows us to calculate hours of gameplay per developer year of production. We have some numbers for casual games (Fate) and commercial games (Diablo). Let's calculate HPD for training.

Training folks already have basic metrics that are comparable so we simply have to do some conversion. They measure hours of training per hour of authoring. In numerous conversations with training organizations I've found the typical range to go from 20 hours of development for 1 hour of training. Some very advanced groups that produce very specialized training with an advanced development pipeline are able to squeeze 1 hour of training out of each hour of development.

Here's the chart that shows the equivalent HPD for games and training. I've included the cost of making cut scenes as well for comparison.
  • Movie (Cut scenes): 0.22
  • Large Game Team (Diablo): 0.54
  • Small Game Team (Fate): 7.5
  • Training (20 hours of dev time): 104
  • Training (1 hours of dev time): 2080
If we plot this data on a log chart, we get a lovely straight line that shows how each category of content is approximately 1 order of magnitude more less expensive to produce.


This is where Serious Games has a great challenge. We can promote the benefits of Serious Games over other types of training, but we will always need to contend with the bean counters who are looking at metrics like HPD as a major determining factor of viability. When you have regulations that specify 'hours of training' as a key metric and cost per hour as an important evaluation point, Serious Games lose out when compared to PowerPoints and PDFs.

Most Serious Games groups that are heavily promoting the benefits of Serious Games over other types of training. Better retention, less drop out, etc. This is a very worthy promotional tactic. Another tactic that can help ensure the adoption of Serious Games is a focus on techniques and technologies that radically reduce the cost of production. The focus is on being competitive using standard metrics. Each developer should ask themselves, "How much is it costing to produce this 1 hour of content?" and seek ways to improve their HPD metric.

We also start talking the language of our business customers. We can stop saying "Dude, it's a game and games are expensive!" when someone asks us about production costs. We can instead say "Well, this past title gave us 7.5 HPD. If you are willing to cut down on the video sections by 50% and add in more gaming sections, we can keep the effectivness the same but increase our HPD to 12.6. That will save you $40,000."

take care
Danc.

4 comments:

  1. You do seem to have a very good understanding of how things sohuld work. (I'm a layman :])

    What was the development budget for Diablo II? Given a similar budget, what would you have done differently for a similar game?

    I would like to see you with such a budget. Lastly, are you confident in your ability to implement these things that you have posted?

    Dear me, this is sounding like some interview. XP

    Anyhow, good luck with your current job as an enabler (Game-designing tools designer?), and take care.

    ReplyDelete
  2. The public bits of knowledge I have to go on is the credits and my own experiences with game development. I'm making some assumptions about how full time people are since typically the artists are picked up near the end of the title and don't work the full time on it.

    I've been doing software development from the design and product management perspective for a good while. That gives me some painfully gained wisdom but by no means makes me a complete expert. :-)

    Diablo II is interesting because now you are talking a very substancial development team of over 200 people. Not all are full time on the project, but I'm guessing it cost in the neighborhood of $10 million. It's a 'big game' and that takes very different management skills than a serious game or a casual game. There are natural inefficiencies that occur once you get that large.

    *grin* From a pure design standpoint, I wouldn't mind getting my hands on a team of that size. The end results would be rather interesting at the very least.

    -Danc.

    ReplyDelete
  3. It is important to note that Diablo II's development was impacted heavily by Everquest. In that Diablo II's staff spent work time playing EQ.

    As far as I'm concerned, that would have been grounds to fire the lot of them (which didn't happen. They instead went to work on WoW). But that's why it cost what it did and took as long as it did.

    Personally, I'm of the opinion that professional game developers have no idea how to efficiently use a team larger than 25 people. It is certainly possible to do so, but they just don't have the skills to do so. Mistreatment of professionals by the industry keeps the people in the industry young, so they never have the time to actually develop the skills to effectively use a large team and multi-year development cycles.

    ReplyDelete
  4. Hey Dan,

    Some interesting metrics. Though you might want to investigate some of the real numbers for training development. A training program will most certainly exceed 20 hours per seat hour to design and develop. The implementation and evaluation efforts alone of a quality course will exceed that number. And don't even get me started on the costs of a seat hour of interactive media instruction:) 12,000 - 35,000 including media development is pretty standard in the industry. I've seen it done at lower cost, but the tasks involved are non-trivial. Contrast this with some rapid assembly of poorly thought out powerpoints and you can get nearer to the 1 hour development mark, but why bother:) The costs of quality development are staggering no matter how you look at it. One gets what one pays for. The real question is what is appropriate and what is the cost benefit for implementing a fluid interactive environment or world with a wider set of rules and variables over a fixed linear path information presentation? And is the arguement for an adaptive system for enhanced learner experience (when appropriately selected and properly designed) going to be enough to sway the money folks?

    Steve

    ReplyDelete