Saturday, May 7, 2011

A blunt critique of game criticism

Note:  This essay has gone through a couple drafts based off extensive feedback (which you can read below in the comments). I'm aiming for a version of this essay that is less likely to violently misinterpreted by a majority of readers. Apologies for altering the context of any of the comments below...an unfortunate peril of live editing.  Again, let me know where I'm wrong.  Let me know which portions makes sense. 


I read Ben Abraham's weekly summary of game criticism over at Critical Distance.  Unlike a decade ago, there is now an absolute deluge of essays being written about games.  I see reactions, counter reactions, and copious commentary. What is difficult to find is good writing that dreams of improving the art and craft of games.

There are three areas of improving writing on games:
  • We need better methods of filtering game criticism.  The types of writing about games have exploded.  With communities of writers attempting to support highly divergent goals and audiences, simply understanding if an essay is useful is a huge challenge. 
  • We need writers who are more deeply educated in the art, craft and science of games. The majority of "game criticism" tends to be informed by a narrow population of gamers, journalists and academics specializing in the humanities.  We are often missing experienced perspective from the sciences and the developers of games.  The vast body of game criticism is written by people that I would consider partial game illiterates.  They are dance judges who have watched Dancing with the Stars, but who have never danced. 
  • We need a defined class of game writing that focuses on improving games.  The existing community will continue writing about the experience of gaming. But what if there were a small group that wished to do more than talk about playing?  Imagine holding your writing to the standard that asks you to ratchet forward the creative conversation.  For this tiny crew, judge your writing on its ability to directly improve the art, culture and science of games in an incontrovertible fashion.  

The blossoming of shallow game criticism

When I started writing about games, there was hardly anyone talking about games in a thoughtful manner. At best, you had the chatter of more vocal gamers.  Even journalists were little more than gamers with a bigger podium.  The developers snuck in peer conversations once or twice a year in hotel bars and then went off to toil in intellectual isolation. An admittedly sad state of affairs.

Today,we've got the developer blogs on Gamasutra, dozens of conferences, the efforts of the Escapist, the rise of the intellectual game journalist and the slow blossoming of academic writing. The language has improved dramatically.  With the arrival of communities of like-minded bloggers and the co-opting of various university departments, writers find themselves encouraged to say what little they can say in increasingly wordy missives. Each week I find myself inundated with essays that appear on the surface to be fascinating treasure troves of insight.

When I invest my time digging past the fresh coat of erudite language, much of the content is a regurgitation of the same tired discussion from ages past.  Consider Adam Ruch's recent article "First Or Third Person – What’s Your Perspective?"  (I chose this example not to be cruel, but because it was at the top of Ben's recent list of game criticism.) Adam is introduced as "a PhD candidate, currently writing about Video Games Criticism" and "a pretty smart guy!"

Yet the essay is little more than a series of personal descriptions of how he feels when he plays certain games.  There is little insight that couldn't be gained by sitting down with a beer and a controller. There is no attempt at gathering empirical evidence. Adam could have saved everyone a vast amount of time with the TL;DR summary: "In 3rd person you can see (and thus empathize) with a visualized character and in 1st person, you can't." Once you strip away the laborious language, you have yet another bit of fluffy gamer opinion written by a young student.

There's a clear and obvious need for writing by young gamers attempting to think about their hobby.  Without such essays, you never gains the skills needed to write something better.  But there needs to be a better filter.

Classifying game criticism

To create a filter, it helps to ask "what is game criticism?"  This simple question results in a large range of definitions, each of which is vigorously defended by bespectacled tribal groups.
  • Traditional reviews: The stated goal is to inform players if they should purchase or try a specific game. Enough information is given to enable players to compare various games without actually wasting time or money playing them. Reviews cover games ranging from the latest Mario blockbuster to a smaller indie title deserving of attention.  
  • Playthroughs:  Where reviews are often (but not always) dry affairs that attempt objectivity, a play through seeks to describe the emotional experience of a game through a single player's eyes. Though I suspect many would disagree, I see the subjective descriptions of gaming found in New Game Journalism as a type of playthrough. 
  • Gamer culture: The impact of games on the culture and identity of the players.  
  • Connecting games with the humanities: An academic exercise in which various aspects of games are described as being part of an ongoing structure of philosophy, movie criticism, literary criticism, art history, rhetoric, etc. 
  • Connecting games with the sciences: An academic exercise in which games are analyzed using the tools of psychology, sociology, economics, etc.  
  • Industry analysis: A discussion of large scale trends in the industry such as platforms, new business models and the ever popular unexpected debacle.  
  • Game analysis: "Here's a working game.  Here's the experiment.  Here are the repeatable lessons I learned."
  • Meta-discussions of game criticism:  Discussion of the goals, best practices and changes in the broader field of game criticism.  This article is one example of such an article.   
Types of writers:  To complicate matters further there are several distinct populations of writers who come with their own goals and target audiences.
  • Journalists:  Writers paid to create content for a publication.  The larger goal of the publication is often to acquire readers that pay the bills which in turn has a strong impact on the style and content of the writing.  Typically journalists targets their writing at mainstream gamers or a sizable niche (such as PC gaming).  The goal is to inform, entertain and build a sense of community.  There is rarely any explicit call to make games better.  Rock Paper Shotgun is a good example of journalists engaging in reviews, playthroughs and the occasional piece of industry analysis. 
  • Gamer Hobbyists / Students:  People who come from a background of playing games and what to share their thoughts.  There is rarely a larger goal and just the fact that someone is reading what they write is often encouragement enough to continue.  The audience is often far narrower since there is no economic reason to broaden the reach. 
  • Academics / Intellectuals:  People who are attempting to build a larger tradition of analysis.  They exist in a self-contained, self referencing world of past papers, publishing, and tenure.  Their audience is other academics and the language is often hyper specialized.  External communication is rare and the bigger goal is the preservation and extension of existing systems of value.  There are rare academics that do original experimental research (thank you!). 
  • Developers:  People who make games.  Their audience is other game developers.  The higher goal is to improve the art and science of games so that games are alway become better: more expressive, more appealing, more efficient, more effective, more successful. 
None of this is clearly defined.  The types of writers mix together in unexpected ways.  They change roles over time.  They intentionally obscure their perspective.  For example, the writing of journalists for certain sites like IGN may mimic the writing by hobbyists.  Or a student might assume the role of an intellectual to give their writing stronger trappings of authority. Some of the writers for Rock Paper Shotgun have started making games.

Amusingly, all groups feel like they are in the minority.  Hobbyists feel that they must constantly burst forth in YMCA-style song about gamer pride or the Man will crush their love of games. Journalists feel no one appreciates their heroic efforts at balancing gamer passion, cultural translations and commercialization. Academics huddle in their isolated departments and wonder why no one listens when they speak the Truth (as defined by a philosopher from the 1970's). Game developers are too busy crunching or being fired  to write much and generally respond in grunts as a result.  'Touchy' is as good a description as any single segment for the entire crew.  Which makes even agreeing on goals, categories and terms difficult.

Here's an attempt:  If I were to categorize Andy's article:  He is a student acting as an academic, writing what is essentially a playthrough that in turn masquerades as game analysis.  The fact that he is a student writing a playthrough is fine.  The multiple levels of deception are what initially raised my hackles.

Given this, if you fail to disclose your perspective, you are very likely wasting the precious time of your reader.  If you deliberately obscure this information (as I've seen many student or indies tempted to do) you are being a dishonest member of our community.  Hey! Stop doing that...there is no shame in writing openly and honestly that you are a gamer expressing your love and appreciation for games.  Just don't obscure your intent with faux intellectualism.

Taking inventory

Given this classification system, what do we have in abundance and what are we lacking? Here is what I see: (and this admittedly may be biased by my own personal consumption habits):
  • Dominant Majority: Journalists and hobbyist gamers writing reviews and playthroughs make up the bulk of the writing on games.  There are very naturally more gamers than any other group so it is quite reasonably that gamers and those that serve gamers produce the highest volume of game writing. 
  • Growing Minority: Academics and intellectuals connecting the dots between games and the humanities are another major category and rally under the 'game criticism' label. 
  • Dwindling Minority:  Game analysis, and essays that connect games with the sciences are far less common.  There are a handful of trade sites like Gamasutra that keep the light alive, but in general it is a desert out there.  

The limitations of writing only by gamers

When I look at this distribution, something strikes me as odd: the vast majority of the rest of writers listed above do not make games, nor do they understand how games are made.  I can understand that there are many writers who are happy just to marinate in the warm communal bath of gamer burbling. I've heard many a gamer tell me that they have no need for any additional knowledge or perspective on games other than what they gain through the playing of games.

Yet I also imagine a mythical writer that wants to uncover additional insights into what makes games tick.  For these curious souls, having hands-on experience making games gives them the ability to observe nuances that no other gamer-only critic could manage.  For those of you instantly think of C++ when you hear the term 'making games', I am very specifically not talking about programming or technical skills.  By making games, be it board games, inventing new sports or making even the simplest of indie games, you gain insight into the fundamental structure of games and how they produce the end user experience that we all find so valuable.   You start to understand interaction loops, pacing, skill acquisition, randomness, how narrative supports mechanics, play styles and dozens of others of foundational game concepts that are difficult to derives from the experiences of being just a gamer.   These are not passing trends in engineering or technology.  These are the bones of what makes a game a game.

Consider the act of judging dances. Dancing (like making games) is a highly technical craft that may be enjoyed superficially or judged in a rigorous fashion. On one hand you have a trained dancer. On the other hand, you have someone who has watched Dancing with the Stars, but never fully engaged in the practical  mastery necessary to understand the foundations of the art.  I submit that if both have comparable skills of analysis and communication, the one with personal experience as a dancer would make the more informed critic.

(It needs to be said: The existence of educated judges does not obsolete the right of the audience to judge.  Dancing with the Stars would not exist if it wasn't for the people in the audience yelling out their own scores, filling message boards with thousands of comments, organizing around favorites and doing all the things that passionate members of a community do.  Games are the same.  An educated minority only add richness to the conversation.  It does not lessen the existing conversation.)

In general, game criticism tends not to be informed hands-on knowledge about what it takes to make a competent game.   In the past week of essays on Critical Distance, I found 1 writer of 12 had any declared experience making games.

This is all of course highly intentional on the part of the promoters of game criticism by gamers. When they look for role models in other media, they see no need for understanding the lowly techniques of creation.  Naive consumption without a deep understanding of form is seen by some as a means of recording a gamer's reactions without undue outside influence. Purely evocative media as music, video, writing or painting can often be reasonable well described using tools from the humanities and the personal reaction of an individual.  If I want to understand a novel, a single sample has limitations, but it can convey the essence of the experience surprisingly well.

Yet though games do possess evocative elements, they also are driven by a functional heart that resists being reduced to only the softest of sciences. Bridges, though undeniably aesthetic and cultural objects, can also be understood as functional or economic creations.  Playthroughs, aesthetics, rhetoric, literary theory, film theory, art history may be one set of valuable perspectives, but if you only rely on these, you will fail to paint a complete picture the babbling, whirring human-mechanical reality of a games.

There is so much about games that is missing from the majority of today's writing. Games have much in common with functional works involving mathematics, psychology, governments, economics or other complex systems. Given population A with skills B, we experimentally validate that we get result C. We have a rich tradition of design practice stretch across Miyamoto to Sid Meier to modern metrics-driven social games.  There exists game design theory stemming from folks like Chris Crawford, Eric Zimmerman and Raph Koster. The instinct of practicing designers alone is an immense iceberg of unwritten knowledge just waiting to be described and shared.

These are vast fields that are mostly untapped by today's writer. And for good reason.  You can only dig into them at the root if you devote a large hunk of your life to mastering them through direct experience.  This means making games in a thoughtful manner and then sharing those insights with those who will only play.  Such people are rare. We need to train more of them.

Wanted: Game analysis
I suspect that it is too late for the field of game criticism to ever again broadly mean 'critical thoughts about games'.  Somewhere along the line we imported wholesale too much baggage from media that long ago stagnated under the weight of navel-gazing divorced from practice.

Instead, we need a new field of discussion.  Let's expand up on the topic above I called Game Analysis.
  • Goal: Advance the art and science of games.  Simply looking at what exists is not enough.  Instead, we leverage what exists in order to to ask what is next and create the conceptual language and tools that get us there. 
  • Audience: Anyone interested in deeply considering how to improve games.   
Who can write on this topic?  Pretty much anyone. Your work will have more impact if you educate yourself in the following ways:
  1. Make games. Again and again and again.  Understand why games work by making games that work. 
  2. Study the fields of science that deal with complex functional systems. 
  3. Devour any and all existing writing both on games and on other unrelated fields to see if they might move the dial forward.  
  4. Share and discuss useful thoughts from your newly enlightened perspective.
Simply making games does not make you a good at game analysis. I have a friend who makes games, but publicly writes gamer-esque ramblings.  Then he wonders why no one pays attention.  A developer ranting about their personal, emotional experience with the controls in Super Meat Boy from the perspective of 'Dude, I'm a gamer just like you" no more improves the state of games than a 13-year old gamer engaged in creating entertainment for his blog. Think deeply about what you do and contribute meaningful writing. I love the visual of a ratchet. Every click advances and builds a foundation of steel that will not let the whole fall backwards.

For those with real world understanding of how to make games better, ask yourself the following questions about what you write:
  • Grounded: Are you basing your theories off empirical evidence?  Do not write something merely because you had a feeling to express.  
  • Aware: Do you know what other people have written in the past?  Do the research and be an informed commenter. 
  • Insightful:  Does your writing add a substantial new perspective or tool that moves the conversation forward?  Do not rehash the same old thing simply because you have an opinion on the currently popular meme.  
  • Actionable:  Does your writing identify a course of action that previously was obscured? Do not let an exploration of an idea wander off into vague hand-waving.  Ask the reader to perform an experiment that increases the knowledge of the community as a whole. 
There is a clear benefit when you follow these guidelines.
  • Your writing stands out from the muck.  The world craves a path forward and the intelligent people you attract by being a grounded, aware, insightful and actionable writer open doors that you would never otherwise find.
  • You improve the world.  Your small contributions build upon the work of others to create a mountain of human endeavor that builds our medium to heights we can only barely imagine. 
As a small closing note, I do realize that many writers are happy writing as only gamers or only journalists or only a specific sub-branch of academia and see no need to branch out.

But we can do more. I come at this topic with the personal belief that merely rehashing the works of others is not nearly enough.  As a creator, you have only a few short years to build something great that changes the world.  Hold yourself to a higher standard.  Be more than a gamer who is writing about personal experiences.  Be more than an academic trying to force games into a 200-year old history of criticism.  Take this weekend, grab some dice and build a game.  Play test it (you aren't building games unless you do).  Polish it.  Release it.  Ask yourself what this tells you about the nature of games and incorporate that critical perspective into your writing.  As years pass and you release your 10th or 20th game, reflect on what have you learned.   Share your journey with the world and raise the level of conversation.

take care
Danc.

Example game analysis 

Some game essays that fit the criteria above.  Heaven forbid I write an essay like this one without giving some positive examples. ;-)

Responses to common comments

  • Most game criticism is not for developers so none of this matters:  You are correct.  This essay is only for those rare writers who wish to improve their craft by mastering new perspectives that are fundamental to the art and science of games. 
  • Game criticism is not about improving games. It is about studying what exists: I understand that there are people who prefer to be historians, catalogers and masticators of culture.  There is still room for both catalogers and people who dream about the future.  Perhaps not under the banner of 'game criticism' but certainly within games as whole.  
  • But making games is engineering and that is dull and soulless:  No, it isn't.  Only a small portion of making games is the technical craft of drawing numbers on cardboard (if it is a board game) or getting triangles to show up (if it is a 3D video game).   Games are about building systems of rules, affordances and people.  They are art, science and community rolled up into one giant holistic act of creativity and play.  To make games well, you need to understand the whole picture.  I desire more writing from this holistic perspective, not from one narrow and highly uninformed perspective. 
  • How will game developers know what players are feeling if not for game criticism?:  Game developers are constantly looking at a vast range of  quantitative and qualitative data. The entire process of game development is built around observing players and adjusting the game (thousands of times!) till the system reaches a desired state of operation. Individual opinions are constantly taken into account.  I personally love watching players and asking them directly what they feel.  In light of this, having a piece of well written criticism is often interesting, but needs to be balanced against the weight of other (often more representative) players.  Since the critic almost never understands the systems underlying their experience, most notes on improvements or root causes are typically wildly off base.  This isn't the fault of the game critic.  They simply lack access to both the dozens (or thousands) of player data points and the intimate knowledge of the game mechanics.  Perhaps one out of a hundred provides a minor insight into a specific game. 

Monday, May 2, 2011

Game Design Logs


If you still practice or encourage the outdated practice of writing long design documents, you are doing your team and your business a grave disfavor. Long design docs embody and promote an insidious world view: They make the false claim that the most effective way to make a game is to create a fixed engineering specification and then hand that off to developers to implement feature by bullet-pointed feature.

Great game development is actively harmed by this assumption.  Pre-allocating resources at an early stage interrupts the exploratory iteration needed to find the fun in a game. A written plan that stretches months into the future is like a stake through the heart of a good game process. Instead of quickly pivoting to amplify a delightful opportunity found during play testing, you end up blindly barreling towards completion on a some ineffectual paper fantasy.

Yet, there is still a need for documentation.  Why?
  • We need a persistent repository of decisions: Teams include many people and conversation occurs asynchronously.  Without centralized documents, you end up with a fragmented conversation where many decisions made in one-on-one conversations are lost to the broader team forever.   
  • We need a shared vision: Documents also helps forge a common vision of the next iteration.   In a situation where everyone has strong and varied opinions, it is essential that someone can lead the team to by unambiguously stating what comes next.  Apparently even God needed documentation. 

Design logs

What I do now is write a little something I call a 'design log.' Game design is a process of informed iteration, not a fixed engineering plan that you implement.  The form of your design documentation should flow from this philosophy.


How to write a design log

  1. Start with a concept:  At the very bottom of the design log is the initial concept.   This is the rough idea started the design in the first place.  These are 2 to 10 pages long and contain just enough text, images and inspiration to start development.  I usually focus mine on the core interaction loop that we want to first prototype. 
  2. Build the prototype:  Design logs exist as a supplement to a working version of the game.  Make something you can play as soon as humanly possible.  Kill graphics, features, plot or anything that gets in the way of making a game you can react to on a tactile and experiential level. 
  3. Add a Daily Entry: After substantial fiddling with your prototype, add a daily entry above the concept to your design log.  This contains daily play notes, prioritized next steps and ideas.  The goal of the daily entry is to move the project forward. You are constantly trying to answer the question "How do we improve the current game?" 
  4. Repeat: Every day or two, you add a new daily entry and the old ones eventually roll off the bottom of the screen.  Much like a blog, the fresh stuff is at the top and the old stuff is at the bottom. 
For the daily entry, I try to keep to the following format, but it is really quite flexible.
  • Heading: The heading is today's date.     
  • Play Notes:  These are the designer's reaction to the day's working build.  I list what worked well and issues I ran into.  For every issue that's raised, I try to come up with a reasonable solution. 
  • Prioritized Next Steps:  If the list of issues is long, I'll call out the order in which they could be tackled.  Sometimes I'll work to create this list, if the backlog has gotten large.  For those agile folks out there, think of it as a Just-In-Time backlog. If you don't need this section at a point in time, don't add it.  
  • Tasks accomplished:  If work has been done, we mark it on the document.  Some teams add a new list of work accomplished to the daily entry.  Others just cross off notes directly in the doc.  
  • Experiments: Big, crazy experiments that move the game forward in big steps.  Without these, you cannot leap to a better local maxima. 

Tools for creating a design log

I personally love using Google Docs for my design logs.  Here are some of the advantages.
  • Real time conversation:  Multiple people can edit the doc at the same time.  I've had some very high bandwidth editing sessions with 3 people all adding and resolving comments like crazy.  No more passing documents around or managing versions. 
  • Comments tied to text:  People can comment on specific section of the text. They can also reply to the comments.  This keeps the conversation focused on specific details instead of hand waving.  No more long rambling thread that diverged from the original topic long ago. 
  • Ability to resolve comments:  Once a comment thread is finished and the resolution incorporated back into the doc, you can resolve it and hide it away.  This keeps the document clean and lets you know when it is time to move on. 
  •  Email alerts: When someone adds a comment, other users subscribed to the doc get an email.  This acts as a re-engagement system that brings the very busy people on the team back to the document. 
Some tools are poor choices for creating design logs:
  • Microsoft Word:  The lack of decent collaboration tools results in locked files, overwritten files and stagnant conversations.  Word is the single most used writing tool, yet it remains the worst possible choice for working designers. 
  • Email: Email lets you reply to specific issues nicely, but the thread of the conversation seems to splinter off rather rapidly. Or you get the counter productive 'epic email thread'. For short term issues it works reasonably. For designs that live longer than a week, email designs turn into an incoherent mess. 
  • Blogs:  I've been trying to get blogs to work as design spaces for years.  The inability to tie comments directly to text is the major failing as is the inability for others to edit the post simultaneously.  Many developers create developer blog, but most feel like one-to-many medium instead of a collaborative conversation that moves the game forward.  Consider these issues a challenge for those of you who love blogs. 

Some tips for using the design log effectively

  • Don't add too much in a single day:  It can be tempting for the designer to add dozens of pages of notes and ideas in a single day.  This just overwhelms the team.  A good rule of thumb is to keep the daily notes to a level that can be read in 5 minutes.  It is uncommon that even a large team will be able to accomplish more than a page or two worth of work in a day, so self edit and focus the writing on things that will make the biggest impact.   When it comes to design, there are no awards for quantity only quality.  Instead of pouring out a giant missive, take a walk and consider what really matters.  Your designs will improve. 
  • For larger teams consider having a handful of logs.  We have a design log and an art log for one project and that splits up the discussion nicely.  I intentionally work with small teams, so I'd be curious to hear how the concept works with production heavy teams that traditionally have difficulty iterating on and evolving their design. 
  • Make sure you have a conversation, not a monologue: Ultimately, a good design log is an ongoing conversation, not the rambling of an isolated individual.  By talking things through together, everyone internalizes the design and makes it their own. Without this conversation, you just have meaningless words on a page.  

Benefits

Here are the benefits I've noticed of the design log approach.  These are attributes you should look for in any healthy design process.
  • Real:  The design notes are heavily based off the last working build.  This reduces the tendency for the designer to wander off into la-la land imagining cool systems that don't tie back to the game you are actively growing. 
  • Actionable:  Each day there is a list of improvements that the team can work on next.  Very little about the design is theoretical. 
  • Communal:  Everyone can jump in and comment and make suggestions. The design notes often act as a lightning rod for directing comments and prompting ideas. 
  • Focused:   This is not a spaghetti wiki.  There is a clear thread of intentional design from the bottom of the document all the way to the top.  You can approach the document as a new team member and read the story of how the game has evolved.   
  • Fresh: The topmost items on the log are always new insights based off new learning from the latest build.   Stale items fall to the bottom of the doc.  This ensures that the document is meaningful to reads and encourages you to create an always living and evolving document. 
  • Agile:  As you learn more about the dynamics of the design, you can very easily steer towards the most promising opportunities.  For many teams, especially ones in preproduction, a design log can replace backlogs and task lists. 
Most importantly, the game design log fits the nature of design:  It is an essential quality of a game design that it evolves over time.  At the heart is a functioning product used by real people who have real reactions to what you've built.  You try new things.  You trim experiments that you imagined would work but didn't.  You double down on the delightful surprises that you could have never predicted upfront.  A design is not plan of execution.  A design is living process that grows a result organically from the journey that team takes together. It is an alchemical chain reaction of players, systems, teams, talents and design.  The starting point influences, but cannot fully define the end result.

There is no place for a dusty design tome in such a dynamic world of evolution.  On the other hand a design log fits. It helps remove the oppressive emphasis on completing preordained features.  Day-by-day, an active design log encourages the team to embrace the iterative spirit of great game development.

take care
Danc.