Saturday, May 7, 2011

A blunt critique of game criticism

Note:  This essay has gone through a couple drafts based off extensive feedback (which you can read below in the comments). I'm aiming for a version of this essay that is less likely to violently misinterpreted by a majority of readers. Apologies for altering the context of any of the comments below...an unfortunate peril of live editing.  Again, let me know where I'm wrong.  Let me know which portions makes sense. 


I read Ben Abraham's weekly summary of game criticism over at Critical Distance.  Unlike a decade ago, there is now an absolute deluge of essays being written about games.  I see reactions, counter reactions, and copious commentary. What is difficult to find is good writing that dreams of improving the art and craft of games.

There are three areas of improving writing on games:
  • We need better methods of filtering game criticism.  The types of writing about games have exploded.  With communities of writers attempting to support highly divergent goals and audiences, simply understanding if an essay is useful is a huge challenge. 
  • We need writers who are more deeply educated in the art, craft and science of games. The majority of "game criticism" tends to be informed by a narrow population of gamers, journalists and academics specializing in the humanities.  We are often missing experienced perspective from the sciences and the developers of games.  The vast body of game criticism is written by people that I would consider partial game illiterates.  They are dance judges who have watched Dancing with the Stars, but who have never danced. 
  • We need a defined class of game writing that focuses on improving games.  The existing community will continue writing about the experience of gaming. But what if there were a small group that wished to do more than talk about playing?  Imagine holding your writing to the standard that asks you to ratchet forward the creative conversation.  For this tiny crew, judge your writing on its ability to directly improve the art, culture and science of games in an incontrovertible fashion.  

The blossoming of shallow game criticism

When I started writing about games, there was hardly anyone talking about games in a thoughtful manner. At best, you had the chatter of more vocal gamers.  Even journalists were little more than gamers with a bigger podium.  The developers snuck in peer conversations once or twice a year in hotel bars and then went off to toil in intellectual isolation. An admittedly sad state of affairs.

Today,we've got the developer blogs on Gamasutra, dozens of conferences, the efforts of the Escapist, the rise of the intellectual game journalist and the slow blossoming of academic writing. The language has improved dramatically.  With the arrival of communities of like-minded bloggers and the co-opting of various university departments, writers find themselves encouraged to say what little they can say in increasingly wordy missives. Each week I find myself inundated with essays that appear on the surface to be fascinating treasure troves of insight.

When I invest my time digging past the fresh coat of erudite language, much of the content is a regurgitation of the same tired discussion from ages past.  Consider Adam Ruch's recent article "First Or Third Person – What’s Your Perspective?"  (I chose this example not to be cruel, but because it was at the top of Ben's recent list of game criticism.) Adam is introduced as "a PhD candidate, currently writing about Video Games Criticism" and "a pretty smart guy!"

Yet the essay is little more than a series of personal descriptions of how he feels when he plays certain games.  There is little insight that couldn't be gained by sitting down with a beer and a controller. There is no attempt at gathering empirical evidence. Adam could have saved everyone a vast amount of time with the TL;DR summary: "In 3rd person you can see (and thus empathize) with a visualized character and in 1st person, you can't." Once you strip away the laborious language, you have yet another bit of fluffy gamer opinion written by a young student.

There's a clear and obvious need for writing by young gamers attempting to think about their hobby.  Without such essays, you never gains the skills needed to write something better.  But there needs to be a better filter.

Classifying game criticism

To create a filter, it helps to ask "what is game criticism?"  This simple question results in a large range of definitions, each of which is vigorously defended by bespectacled tribal groups.
  • Traditional reviews: The stated goal is to inform players if they should purchase or try a specific game. Enough information is given to enable players to compare various games without actually wasting time or money playing them. Reviews cover games ranging from the latest Mario blockbuster to a smaller indie title deserving of attention.  
  • Playthroughs:  Where reviews are often (but not always) dry affairs that attempt objectivity, a play through seeks to describe the emotional experience of a game through a single player's eyes. Though I suspect many would disagree, I see the subjective descriptions of gaming found in New Game Journalism as a type of playthrough. 
  • Gamer culture: The impact of games on the culture and identity of the players.  
  • Connecting games with the humanities: An academic exercise in which various aspects of games are described as being part of an ongoing structure of philosophy, movie criticism, literary criticism, art history, rhetoric, etc. 
  • Connecting games with the sciences: An academic exercise in which games are analyzed using the tools of psychology, sociology, economics, etc.  
  • Industry analysis: A discussion of large scale trends in the industry such as platforms, new business models and the ever popular unexpected debacle.  
  • Game analysis: "Here's a working game.  Here's the experiment.  Here are the repeatable lessons I learned."
  • Meta-discussions of game criticism:  Discussion of the goals, best practices and changes in the broader field of game criticism.  This article is one example of such an article.   
Types of writers:  To complicate matters further there are several distinct populations of writers who come with their own goals and target audiences.
  • Journalists:  Writers paid to create content for a publication.  The larger goal of the publication is often to acquire readers that pay the bills which in turn has a strong impact on the style and content of the writing.  Typically journalists targets their writing at mainstream gamers or a sizable niche (such as PC gaming).  The goal is to inform, entertain and build a sense of community.  There is rarely any explicit call to make games better.  Rock Paper Shotgun is a good example of journalists engaging in reviews, playthroughs and the occasional piece of industry analysis. 
  • Gamer Hobbyists / Students:  People who come from a background of playing games and what to share their thoughts.  There is rarely a larger goal and just the fact that someone is reading what they write is often encouragement enough to continue.  The audience is often far narrower since there is no economic reason to broaden the reach. 
  • Academics / Intellectuals:  People who are attempting to build a larger tradition of analysis.  They exist in a self-contained, self referencing world of past papers, publishing, and tenure.  Their audience is other academics and the language is often hyper specialized.  External communication is rare and the bigger goal is the preservation and extension of existing systems of value.  There are rare academics that do original experimental research (thank you!). 
  • Developers:  People who make games.  Their audience is other game developers.  The higher goal is to improve the art and science of games so that games are alway become better: more expressive, more appealing, more efficient, more effective, more successful. 
None of this is clearly defined.  The types of writers mix together in unexpected ways.  They change roles over time.  They intentionally obscure their perspective.  For example, the writing of journalists for certain sites like IGN may mimic the writing by hobbyists.  Or a student might assume the role of an intellectual to give their writing stronger trappings of authority. Some of the writers for Rock Paper Shotgun have started making games.

Amusingly, all groups feel like they are in the minority.  Hobbyists feel that they must constantly burst forth in YMCA-style song about gamer pride or the Man will crush their love of games. Journalists feel no one appreciates their heroic efforts at balancing gamer passion, cultural translations and commercialization. Academics huddle in their isolated departments and wonder why no one listens when they speak the Truth (as defined by a philosopher from the 1970's). Game developers are too busy crunching or being fired  to write much and generally respond in grunts as a result.  'Touchy' is as good a description as any single segment for the entire crew.  Which makes even agreeing on goals, categories and terms difficult.

Here's an attempt:  If I were to categorize Andy's article:  He is a student acting as an academic, writing what is essentially a playthrough that in turn masquerades as game analysis.  The fact that he is a student writing a playthrough is fine.  The multiple levels of deception are what initially raised my hackles.

Given this, if you fail to disclose your perspective, you are very likely wasting the precious time of your reader.  If you deliberately obscure this information (as I've seen many student or indies tempted to do) you are being a dishonest member of our community.  Hey! Stop doing that...there is no shame in writing openly and honestly that you are a gamer expressing your love and appreciation for games.  Just don't obscure your intent with faux intellectualism.

Taking inventory

Given this classification system, what do we have in abundance and what are we lacking? Here is what I see: (and this admittedly may be biased by my own personal consumption habits):
  • Dominant Majority: Journalists and hobbyist gamers writing reviews and playthroughs make up the bulk of the writing on games.  There are very naturally more gamers than any other group so it is quite reasonably that gamers and those that serve gamers produce the highest volume of game writing. 
  • Growing Minority: Academics and intellectuals connecting the dots between games and the humanities are another major category and rally under the 'game criticism' label. 
  • Dwindling Minority:  Game analysis, and essays that connect games with the sciences are far less common.  There are a handful of trade sites like Gamasutra that keep the light alive, but in general it is a desert out there.  

The limitations of writing only by gamers

When I look at this distribution, something strikes me as odd: the vast majority of the rest of writers listed above do not make games, nor do they understand how games are made.  I can understand that there are many writers who are happy just to marinate in the warm communal bath of gamer burbling. I've heard many a gamer tell me that they have no need for any additional knowledge or perspective on games other than what they gain through the playing of games.

Yet I also imagine a mythical writer that wants to uncover additional insights into what makes games tick.  For these curious souls, having hands-on experience making games gives them the ability to observe nuances that no other gamer-only critic could manage.  For those of you instantly think of C++ when you hear the term 'making games', I am very specifically not talking about programming or technical skills.  By making games, be it board games, inventing new sports or making even the simplest of indie games, you gain insight into the fundamental structure of games and how they produce the end user experience that we all find so valuable.   You start to understand interaction loops, pacing, skill acquisition, randomness, how narrative supports mechanics, play styles and dozens of others of foundational game concepts that are difficult to derives from the experiences of being just a gamer.   These are not passing trends in engineering or technology.  These are the bones of what makes a game a game.

Consider the act of judging dances. Dancing (like making games) is a highly technical craft that may be enjoyed superficially or judged in a rigorous fashion. On one hand you have a trained dancer. On the other hand, you have someone who has watched Dancing with the Stars, but never fully engaged in the practical  mastery necessary to understand the foundations of the art.  I submit that if both have comparable skills of analysis and communication, the one with personal experience as a dancer would make the more informed critic.

(It needs to be said: The existence of educated judges does not obsolete the right of the audience to judge.  Dancing with the Stars would not exist if it wasn't for the people in the audience yelling out their own scores, filling message boards with thousands of comments, organizing around favorites and doing all the things that passionate members of a community do.  Games are the same.  An educated minority only add richness to the conversation.  It does not lessen the existing conversation.)

In general, game criticism tends not to be informed hands-on knowledge about what it takes to make a competent game.   In the past week of essays on Critical Distance, I found 1 writer of 12 had any declared experience making games.

This is all of course highly intentional on the part of the promoters of game criticism by gamers. When they look for role models in other media, they see no need for understanding the lowly techniques of creation.  Naive consumption without a deep understanding of form is seen by some as a means of recording a gamer's reactions without undue outside influence. Purely evocative media as music, video, writing or painting can often be reasonable well described using tools from the humanities and the personal reaction of an individual.  If I want to understand a novel, a single sample has limitations, but it can convey the essence of the experience surprisingly well.

Yet though games do possess evocative elements, they also are driven by a functional heart that resists being reduced to only the softest of sciences. Bridges, though undeniably aesthetic and cultural objects, can also be understood as functional or economic creations.  Playthroughs, aesthetics, rhetoric, literary theory, film theory, art history may be one set of valuable perspectives, but if you only rely on these, you will fail to paint a complete picture the babbling, whirring human-mechanical reality of a games.

There is so much about games that is missing from the majority of today's writing. Games have much in common with functional works involving mathematics, psychology, governments, economics or other complex systems. Given population A with skills B, we experimentally validate that we get result C. We have a rich tradition of design practice stretch across Miyamoto to Sid Meier to modern metrics-driven social games.  There exists game design theory stemming from folks like Chris Crawford, Eric Zimmerman and Raph Koster. The instinct of practicing designers alone is an immense iceberg of unwritten knowledge just waiting to be described and shared.

These are vast fields that are mostly untapped by today's writer. And for good reason.  You can only dig into them at the root if you devote a large hunk of your life to mastering them through direct experience.  This means making games in a thoughtful manner and then sharing those insights with those who will only play.  Such people are rare. We need to train more of them.

Wanted: Game analysis
I suspect that it is too late for the field of game criticism to ever again broadly mean 'critical thoughts about games'.  Somewhere along the line we imported wholesale too much baggage from media that long ago stagnated under the weight of navel-gazing divorced from practice.

Instead, we need a new field of discussion.  Let's expand up on the topic above I called Game Analysis.
  • Goal: Advance the art and science of games.  Simply looking at what exists is not enough.  Instead, we leverage what exists in order to to ask what is next and create the conceptual language and tools that get us there. 
  • Audience: Anyone interested in deeply considering how to improve games.   
Who can write on this topic?  Pretty much anyone. Your work will have more impact if you educate yourself in the following ways:
  1. Make games. Again and again and again.  Understand why games work by making games that work. 
  2. Study the fields of science that deal with complex functional systems. 
  3. Devour any and all existing writing both on games and on other unrelated fields to see if they might move the dial forward.  
  4. Share and discuss useful thoughts from your newly enlightened perspective.
Simply making games does not make you a good at game analysis. I have a friend who makes games, but publicly writes gamer-esque ramblings.  Then he wonders why no one pays attention.  A developer ranting about their personal, emotional experience with the controls in Super Meat Boy from the perspective of 'Dude, I'm a gamer just like you" no more improves the state of games than a 13-year old gamer engaged in creating entertainment for his blog. Think deeply about what you do and contribute meaningful writing. I love the visual of a ratchet. Every click advances and builds a foundation of steel that will not let the whole fall backwards.

For those with real world understanding of how to make games better, ask yourself the following questions about what you write:
  • Grounded: Are you basing your theories off empirical evidence?  Do not write something merely because you had a feeling to express.  
  • Aware: Do you know what other people have written in the past?  Do the research and be an informed commenter. 
  • Insightful:  Does your writing add a substantial new perspective or tool that moves the conversation forward?  Do not rehash the same old thing simply because you have an opinion on the currently popular meme.  
  • Actionable:  Does your writing identify a course of action that previously was obscured? Do not let an exploration of an idea wander off into vague hand-waving.  Ask the reader to perform an experiment that increases the knowledge of the community as a whole. 
There is a clear benefit when you follow these guidelines.
  • Your writing stands out from the muck.  The world craves a path forward and the intelligent people you attract by being a grounded, aware, insightful and actionable writer open doors that you would never otherwise find.
  • You improve the world.  Your small contributions build upon the work of others to create a mountain of human endeavor that builds our medium to heights we can only barely imagine. 
As a small closing note, I do realize that many writers are happy writing as only gamers or only journalists or only a specific sub-branch of academia and see no need to branch out.

But we can do more. I come at this topic with the personal belief that merely rehashing the works of others is not nearly enough.  As a creator, you have only a few short years to build something great that changes the world.  Hold yourself to a higher standard.  Be more than a gamer who is writing about personal experiences.  Be more than an academic trying to force games into a 200-year old history of criticism.  Take this weekend, grab some dice and build a game.  Play test it (you aren't building games unless you do).  Polish it.  Release it.  Ask yourself what this tells you about the nature of games and incorporate that critical perspective into your writing.  As years pass and you release your 10th or 20th game, reflect on what have you learned.   Share your journey with the world and raise the level of conversation.

take care
Danc.

Example game analysis 

Some game essays that fit the criteria above.  Heaven forbid I write an essay like this one without giving some positive examples. ;-)

Responses to common comments

  • Most game criticism is not for developers so none of this matters:  You are correct.  This essay is only for those rare writers who wish to improve their craft by mastering new perspectives that are fundamental to the art and science of games. 
  • Game criticism is not about improving games. It is about studying what exists: I understand that there are people who prefer to be historians, catalogers and masticators of culture.  There is still room for both catalogers and people who dream about the future.  Perhaps not under the banner of 'game criticism' but certainly within games as whole.  
  • But making games is engineering and that is dull and soulless:  No, it isn't.  Only a small portion of making games is the technical craft of drawing numbers on cardboard (if it is a board game) or getting triangles to show up (if it is a 3D video game).   Games are about building systems of rules, affordances and people.  They are art, science and community rolled up into one giant holistic act of creativity and play.  To make games well, you need to understand the whole picture.  I desire more writing from this holistic perspective, not from one narrow and highly uninformed perspective. 
  • How will game developers know what players are feeling if not for game criticism?:  Game developers are constantly looking at a vast range of  quantitative and qualitative data. The entire process of game development is built around observing players and adjusting the game (thousands of times!) till the system reaches a desired state of operation. Individual opinions are constantly taken into account.  I personally love watching players and asking them directly what they feel.  In light of this, having a piece of well written criticism is often interesting, but needs to be balanced against the weight of other (often more representative) players.  Since the critic almost never understands the systems underlying their experience, most notes on improvements or root causes are typically wildly off base.  This isn't the fault of the game critic.  They simply lack access to both the dozens (or thousands) of player data points and the intimate knowledge of the game mechanics.  Perhaps one out of a hundred provides a minor insight into a specific game. 

78 comments:

  1. I always value the opinion of game creators. But to say that they are the only source for criticism would be like saying that all movie reviewers must be movie directors. I don't think there are any of those out there.
    Games are certainly harder to make than most gamers think, but most professional journalists, I think, realize this.
    I think it's an interesting perspective and a nice dream, but I don't think it's going to happen.

    ReplyDelete
  2. This is more about getting more cream to rise to the top. We'll always have gamers writing about their experiences. That's a good thing. Let's just not pretend it offers more than a very limited and isolated perspective on games.

    And for the record, I don't think I would trust the literary criticism of someone who cannot write. When I speak of the parasite critics, I think those in the movie industry qualify quite nicely. I suspect most would agree that turning into the movie industry is *not* a desired outcome. :-)

    All the best,
    Danc.

    ReplyDelete
  3. It's true "game studies" academics have no experience making games, but the more serious indictment is that many won't even admit they have ever, themselves, even played a game, let alone enjoyed it. Evidently they view it as beneath them. I wrote at length about the uselessness of academic game criticism in my August 2006 Escapist article "Immersion Unexplained":

    http://www.escapistmagazine.com/articles/view/issues/issue_57/341-Immersion-Unexplained

    ReplyDelete
  4. Kudos on an excellent article that properly takes to task a lot of low-value writing and criticism going on right now. Thanks for sharing these "positive examples" of recent criticism, of which I've only read two - in fact I made a comment on the Gamasutra article praising it, so apparently we're on the same page here. :)

    I think you go a little far in implying that the value of criticism from other game developers is far higher than non-developers' criticisms. The tone here seems to imply that you simply aren't inclined to listen to criticism from people who aren't game developers. But there's two reasons why I'd value a non-game-developer's opinion on a game as high, if not higher, than that of a game developer.

    The first reason is that we developers are often too aware of, and too impressed by, the amount of work that went into developing a game. For example, as a programmer who has looked into natural language parsing, Facade can't help but impress me. But Facade's AIs still don't pass the Turoing test, and its characters are only slightly less unconvincing than "Eliza" or other clumsy and older attempts at conversation simulators.

    One example of this was Tom Chick's tepid review of Starcraft 2. As a developer, I'm a bit flabbergasted that one man can be so dismissive of a game that took over 7 years of work from over 100 of the most brilliant and creative developers in the industry. But part of what we need from criticism is honesty: an outsider's perspective that's not influenced by "inside baseball" as ours are.

    It's true that a non-developer could probably never make a brilliant mechanical treatise such as the "donkeyspace" theory laid out by Frank Lantz in your above example. However they can say a lot about the experiential levels of our games - which could and probably should be considered the most important layer in our games. After all, what is any artist attempting to create if not an experience for the recipient of the art? Reading about what these experiences are for thoughtful recipients (whether or not they have a deep knowledge of the design rules and development tricks that may have made this experience possible) has a great deal of value to me - so long as they're read with this perspective in mind.

    I agree, however, that criticism by other developers that offers insight and perspectives on other ways to solve problems is incredibly useful. I think what we really need is more of a "peer criticism" culture: I see too few "game design reviews" in which developers offer suggestions on other ways in which games could have been approached... and I'd agree with you that this kind of criticism is incredibly useful and constructive.

    (Oh, and so you know how to value my input here: I AM an experienced game developer, a programmer/designer - I would place myself as a journeyman - who has indeed spent the majority of his waking life attempting to make and release games for the last ~8 years, and for the last 1.5 years in particular... though unfortunately mine has been one of those careers in which the majority of games I've worked on will never be released.)

    ReplyDelete
  5. I don't think the effort and struggles that were required to develop a game should change anyone's opinion of a game or any other product. Either an opinion is well founded and insightful or it is not. Just because someone is inexperienced doesn't mean their opinions are without value. And what else is there than opinions? Even the most well-researched and factually based opinion in the world is still just an opinion.

    That you are willing to dismiss an opinion based on experience alone shows the vast ignorance of your own opinion on this subject.

    ReplyDelete
  6. As an aspiring game designer/critic I really enjoyed the article. Like your others it was well thought out, concise, and indeed adds a lot to the ongoing discussion. However, I had a couple of critiques.

    As someone who has read and understands a fair amount about both physics and systems, I think systems thought can be applied to and found in any field, not just science. Each field has their own unique perspective to bring to the overarching idea of systems theory, and biasing one over the other often precludes a very robust understanding.

    I would also echo the above poster that I don't think we should signify developers or non-developers as having more legitimacy. Everyone has a point to bring to the table, and saying that only one type of people are allowed to make that point seems rather limiting. As you said above game developers are often no better at this than any other. I would rather say that all critics should aspire to create works of writing that further the field of game design while being conscious of the biases their unique perspective skews them towards.


    However, excellent article. I always enjoy your unique view.

    ReplyDelete
  7. I think IQpierce is completely right. You should value an articulate player's experience with your game at least as much as any developer's, if only because you presumably made the game for him to play and enjoy! I mean, if all developers followed your hierarchy, they'd basically be making games for themselves.

    ReplyDelete
  8. Updated the doc to include feedback: A) Emphasized the importance of listening to players (something I passionately practice!) Tom Chick is still cool. B) Added a bit on the validity of different opinions C) Clarified that perspectives from other fields is good. D) Emphasized the need for game developers to write about games.

    ReplyDelete
  9. You've clearly articulated that you're mainly talking about whether games criticism is useful to you, and in that respect your argument is spot on.

    Unfortunately, you also imply in the strongest terms that everything else not useful to you is inherently worthless: "ignorant blather", "your spew of text", "what passes for insight" etc. That's muddying the waters. Most writing about games is crap, but game developers have no monopoly on the upper tier of the quality scale.

    As you asked for feedback, I suggest you ignore the fact keep the focus on your core topic: the purpose of game criticism.

    The fact is that criticism is not necessarily intended for creators, and is not subservient to their needs. It is neither design documentation nor instructional textbook. It is instead a consideration of the qualities, meanings and value of the art being produced. Its central questions are "what" and "why" rather than "how". Its purpose is, in part, to inform the community's thinking about the subject. This is clearest in the case of an overtly ideological site like The Border House, but is true of all critics to some extent.

    Living in Australia, where anti-game prejudice is enshrined in law, I'm acutely aware of the need for people who can clearly articulate what games are and what value games have to a wide audience. Game developers are often not the best people for the job, because their skill set is elsewhere (which is not to discount those like Will Wright who do make excellent representatives). It's valuable to have a Kieron Gillen (or critic of your choice) around to intelligently explain things like what a war simulation really means, especially when non-gamers are continually getting the wrong idea.

    It's not intended for the makers of the game, but if it informs their thinking, that's great too. In fact, I believe that designers who have the ability to step away from their role as creator and look at their creation from the perspective of its audience are likely to make better art. That means listening to the opinions of people not privy to the creative process that went into making it.

    Essentially, your argument strikes me as the old, traditional scorn of the engineer toward the humanities professor. "How does all your philosophy help me to build a stronger bridge?" scoffs the engineer, missing the point.

    ReplyDelete
  10. I have found myself struggling with a similar problem of filtering my game article intake. If I spent all my time reading articles and blog posts, I still wouldn't have time to read them all. And I find that the overall nutritional value of this intake is very low.

    Generally, I find that the best way to filter is simply to read the articles that I find through recommendations on Twitter, or Danc's Reading List. ;)

    ReplyDelete
  11. Hi Fraser! Appreciate you stopping by.

    I agree with the topics you raise. To reflect:
    - No, you are not subservient to creators. I've seen this comment in a couple of places and I'd be delighted if folks could help me identify the particular turn of phrase that seems to polarize folks so strongly.
    - Yes, it is helpful to have someone who can act as a translator to other media.
    - Yes, it is important for developers to look at their creation from the perspective of the audience. (This comment actively shocks me. What do you think game developers are *doing* with those metrics, surveys, play tests and thousands upon thousands of iterations on a game?)

    This last point is a good example of an area where knowledge of game development would improve criticism. Game developers tend to know immensely more about the breadth and depth of player reactions to a particular sequence than most writers ever seem to give credit. The iteration of development is a process and an experience that forever alters how you understand games. Once you experience what it actually means to listen to customers, you end up having a richer perspective on how your specific and highly personal feedback fits into the whole.

    Would it hurt your writing to have this more informed perspective? Probably not. And I suspect it would give much of the writing additional depth and authority.

    If your goal really is to inform the community, then most writing is missing some pretty major chunks of knowledge about games.

    Another metaphor: It is like there is a class of writing that spends thousands of pages discussing a person's individual experience with the texture of a single $1 dollar bill while ignoring anything having to do with micro and macro economics. Are you really broadly doing your job informing the community of the nature of a dollar bill if mostly you focus on the texture of that one example?

    Replace the texture of the dollar with a single person's experience and the micro and macro economics with game design and development (play testing, psychology, economics again, etc.)

    The comment about the engineer and the humanities professor is appropriate. However with games, we seem to have accidentally asked the humanities professor to write about the purpose and meaning of bridges independent of any knowledge of physics, transit or architecture. Instead, he walks across the bridge once, considers his research done and then spends the next three years writing a book about how his feet ache. ;-)

    To be clear
    - I'm not putting game criticism in a box.
    - I'm not claiming your soapbox for my own.
    - I'm not restricting meaningful discussion to only developers.

    Instead, I'm asking you to be better at your job. Be more informed. Understand games at a deeper level. Base your writing off a broader range of hands on experiences than just sitting down with a game and playing it. Yes, playing games is still valid, but there is a heck of a lot more out there.

    As a side note: Yes, I would also like more people to write about games in a way that moves game development as opposed to game playing forward. That's me being selfish. Believe it or not, the amount of game criticism completely swamps the amount of interesting writing by game developers. Sometimes I think gamers imagine that there are secret clubs where immense knowledge flows in edible chocolate streams. We try. But the conversation is limited and most developers are understandably focused on making games. So the art of game development inches forward mostly in a small sputtering bursts of conversation.

    In this respect, I admit to being a bit disappointed how academia and journalists have tackled games. The blossoming of writing that might help game developers never really happened. I suppose it is a simple matter of numbers: there are more gamers looking to write about games than there are game developers.

    take care,
    Danc.

    ReplyDelete
  12. Did Cliff Clavin write this article?

    ReplyDelete
  13. Could you possibly entertain the idea that writing may be the actual problem?

    Writing is far too simple and linear.

    ReplyDelete
  14. Unless you write game reviews you probably aren't in the position to critique game reviews.

    ReplyDelete
  15. Dan, it's really unclear to me who you're writing this article for? What exactly are you trying to improve?

    If you're writing it for game critics, you probably shouldn't start by calling them parasites, if you seriously intend them to read this piece and then start following your advice.

    If you're writing it to encourage game designers to write about design, then you're better off writing a how to piece on writing.

    I'm also confused to the 'positive' articles you're linking to. None of them are game criticism. They're all about game design, which is a completely different fish to the one you're talking about in the rest of the article.

    ReplyDelete
  16. The fact is, Dan, that most gamers simply do not care about the process behind making games. Just like most music fans (outside the obsessives) do not wonder what input a particular producer has offered or what an editor stripped out of a particular paperback.

    What your audience care about are a few things:

    - Is the game fun?
    - Does it work properly?
    - Does it look good?
    - Will it hold my interest?

    That's it. You should be happy that a lot of academic and feature writers bother to head behind the curtain and introduce the actual people who spend full years of their lives putting pieces together, tearing them down and putting them up again.

    The other part relates to how secretive a lot of non-indie production houses tend to be when it comes to their development process. I've had tons of interviews with both major and minor developers regarding their operations, and most aren't willing to digress further than a certain mark hense they will be muted by their PR rep or some other lakey from the publisher.

    Most copy is dribble, and that relates to an entire other problem when it comes to game critique - funding. If you want good prose, you either need to do this in your own time, or you need someone to back you with enough money to pay the rent. Since most gamers aren't willing to pay for copy anymore, most media rely on clicks, and those clicks come from, you guessed it, drama and quick to run reviews.

    I could go on all day about the dramas that those in the games media industry deal with on a regular basis, but the crux of it relates to what the audience wants.

    Until gamers care about what's behind their media, and developers are willing to spill their guts to the same extent, we're going to continue in this cycle for an indefinate period.

    On a side note: this whole situation links well to the brilliant article Geoff Knightley produced about Portal 2 - Valve gave him deep access to their studios and as a result, we got extraordinary access to the people and their development path that had never existed before.

    Once the developers begin to co-operate with the media, rather than heaping derision and making ridiculous claims (re: some of the points in your piece) then we might be able to work together.

    ReplyDelete
  17. I'll echo, as a game designer, what others have said.

    1. Experience making games has no bearing on the quality of writing about games. Yes, designers often have unique insight, but I've read some insightful writing by people who aren't developers. I think part of the problem is that there's a lot of poor writing about games from people who are wannabe game designers.

    2. You're confusing game critique with game design instruction. All of the "good" essays you linked are related to people sharing insightful instruction about game design. And, yes, this is often useful for game designers! But, this isn't critique.

    At the simplest level, critique is a "review" of a work. The best critique, however, is one that touches on the meaning and influence of a work. An essay about how controls for a game are sluggish because movement happens on KeyUp instead of KeyDown is design instruction. An essay about how the sluggish controls for a game build suspense and reflect the difficulty of the topic of the game is critique.

    3. Critique is not necessarily for developers. Good critique informs and educates the audience. Being game expereinced designers, we don't need to be informed or educated as much as others do. But, that doesn't mean the work is universally worthless. An essay that educates people about the emotional depththat we try to instill in our games or about how they have positive effects on bored housewives can help reflect the positive aspects we take for granted. Given that most of the most active designers are usually busy actually designing games, having smart and educated people write for the more general audience can help.

    My thoughts.

    ReplyDelete
  18. A music critic does not play a musical instrument yet he/she can criticize music.
    A movie critic does not make movies, yet he/she can criticize movies.
    A book critic does not write his/her own book, yet he/she can criticize other novels.

    There are certain constraints that each medium has. Band members can have differing visions, actors can be fickle, writers block can set in. But being familiar with these obstacles doesn't change the final product that is evaluated. If it is crap, being familiar with how crap is made doesn't change it from crap to gold. A medium such as video games can never be truly objective, it will always be subjective to a certain extent. To use the excuse "You guys just don't understand!" seems to be a cheap cop-out that will never be truly respected by the public. Even though many people that write about games might be immature, saying that "because you've never made a game, shut your mouth," doesn't seem like a step in the right direction.

    ReplyDelete
  19. The animosity between developers and game journalists exist because both camps tend to devalue key aspects of the other's job. Game critics tend to be uninterested in the development of the game, the difficulty of implementing features or simply understanding the limits and constraints of making a game. Game developers tend to under appreciate the perspective that can be provided by someone who is paid to play a plethora of current games and give an opinion on how that game fits into the current changing landscape.

    As a developer, it does bum me out when I read an overly snarky opinion from some dude who doesn't have the slightest idea how to implement his (most often unreasonable) complaints. But I'm also very busy working on our game and have two kids, which doesn't give me much time to play a lot of the games out there - so I do my best to value that perspective offered by critics who are paid to review games.

    ReplyDelete
  20. Dan: "No, you are not subservient to creators. I've seen this comment in a couple of places and I'd be delighted if folks could help me identify the particular turn of phrase that seems to polarize folks so strongly."

    Your entire argument is based upon the following:
    1) Creators' opinions should always be taken as a higher standard than non-creators, as explained in the sections where you have a chain of being where you judge someone's experience to whether you're going to pay attention to them or not.
    2) If criticism doesn't help game design, it's useless, as stated in the section where you list all the forms of games criticism and then reject all the ones which don't do that. Also, you admit that there's other readers, but fuck 'em.

    For the first one, I'll note that in Rossignol's years as a games journalist specialising in MMOs, he's never met a project lead who's spent any time playing Eve. They can talk about their experience, but their experience is so narrow and the medium is so large I would think it dangerous to draw too much from any one individual's experience.

    For the second one, I'd agree. Were I an engineer.

    KG

    ReplyDelete
  21. I would encourage you to integrate your comment to Fraser Allison in your main article, because it clarifies points that I had hoped you were making but weren't explicit in the writing.

    However, and a few people have mentioned this, but what you're referring to as game criticism isn't actually game criticism. What you consider "bad" games criticism is actually the definition of games criticism, and what you consider good criticism—"game developer analysis"— is not criticism at all, but instruction on the craft, which is completely different.

    Here is a metaphor: There are classes on creative writing, and there are classes on literary analysis. One tells you how to write, the other tells you how to understand what writing means and how it makes us feel. You want instruction on games as a craft, and you're looking for it in analysis of how games make us feel. No wonder you don't feel satisfied with current games criticism! By its very definition, it isn't what you're looking for.

    You are looking to games criticism for things like postmortems and analysis of technique. But this has never been what criticism is concerned with: as an example, a feminist critique of Hamlet will is in no way concerned with instructing the reader on how to write like Shakespeare.

    No one would ever read literary criticism as a instruction manual on how to write, because that isn't its purpose. Its purpose is to articulate the effects games have on us. That's why writers talk about their feelings, and why readers enjoy reading criticism—it enhances their own feelings and understanding of games, (or literature, in this extended metaphor). To be blunt, this writing is going to have little to no application to your design process and should never be expected to. I can tell you that as a writer, literary analysis is the last place I would ever go to get better at writing, just as a game designer will not find much instruction from Kotaku.

    HOWEVER, you've raised an extremely important point: not enough critics know about the craft of games. Roger Ebert may never have made a movie, but he knows damn near everything about the filmmaking process. In order to make more thorough analysis, critics will need to know what they're talking about.

    They still won't be any more useful to you, though, because criticism is concerned with analysis of response. We certainly need more collections of postmortems and analysis of how we make games, and that would be of benefit to critics as well.

    But please, redefine your terms: the craft of games is not the experience of games. If this is a draft, make this your number one priority.

    ReplyDelete
  22. In addition to Kieron's and Andrew's and Psychochild/Brian's comments, which I wholly agree with, I'll chime in that you don't seem to touch on criticism of social issues -- feminist critique and that sort of thing. I mean, I suspect you'll throw that out the window too, but you might want to add it to the list of criticisms you find invalid!

    ReplyDelete
  23. Danc: Thanks for the thoughtful reply.

    Does knowledge of game development make a writer better at game criticism? Probably, but it's not so simple. The developer's perspective is one body of knowledge among many that a critic can bring to their work to make it more valuable.

    The Psychology of Video Games article you linked is a good example. Jamie Madigan obviously knows a lot about game development, but it's primarily his knowledge of psychology that makes his analysis insightful.

    Or take this piece by Ben Medler on modernism in Mirror's Edge and postmodernism in Assassin's Creed. It provides a way of thinking about those games that has everything to do with their design but very little to do with the process of making them.

    I don't mean to devalue the creator's perspective, but to point out the benefits of other perspectives. And as IQpierce pointed out, developers have their own baggage: it's harder to see a game with fresh eyes when you know what went into making it. Does a food critic really want to know how the sausage was made?

    I'll try to answer the two big questions you posed: what was it you wrote that made people think critics should serve creators, and do readers understand the amount of user testing that goes into making a game?

    On the first point: it isn't one particular turn of phrase but a current that runs through what you've written. The section headed "Classifying game criticism" is the crux. By summarising the first four classes of writing in such a dismissive way, you imply that it has no value. Not all of the writing in each category is good, but some of it is, and it all has an audience. In each case, the audience is rating the work on different merits than what you're asking for here.

    On the second point: I hope most critics do appreciate the importance of user testing. I think developers have done a good job of conveying its central importance - particularly Valve with its developer commentaries. But while metrics, surveys and play tests are essential, they are all more or less quantitative forms of data collection, which can't provide the whole story. Chris Hecker has pointed out the danger of relying too much on this kind of feedback:

    "The problem is not that the data is wrong, the problem is that we tend to gather the data that is convenient to gather, we worship that data because it is at least some concrete port in the storm of game design and player behavior, and then we assume we can take a reasonable derivative from that data and hill-climb to a better place in design-space. But, as anybody who knows any math can tell you, hill-climbing is not a very good algorithm for optimizing complex functions, and game design is a very complex function indeed."

    Hecker says "intuition is the art" that goes into game development, but that intuition doesn't have to come from nowhere. To get a more complete picture, you need qualitative feedback - which you do indeed get, by talking to playtesters and players. In-depth game criticism goes deeper (if it's any good), but is essentially the same thing: an in-depth qualitative response to the game.

    Some new comments have appeared since I started writing this (I thought you had to say Gillen's name three times into a screen before he was summoned?), so I'll tie it up here rather than repeat what other people are already saying.

    Cheers
    Fraser

    ReplyDelete
  24. It sounds to me that you're asking less for a wholesale revolution in games journalism than for the creation of a professional journal of game development.

    In other words: a publication that covers issues in game design from the developer's perspective, perhaps with articles submitted by other developers. Something that does for game developers what The Lancet and New England Journal of Medicine do for medical professionals, what Nature and Science do for scientists, etc.

    Such a journal would provide the filter you're looking for, since by definition it would only include commentary by professionals working in the field. And since its audience would be developers rather than gamers, it could include submissions that focus on practical questions that wouldn't be of interest to the general population of gamers at large. (Just like patients don't read The Lancet.)

    So maybe this essay would be more productive if it were reframed from "why game journalism sucks" (which in the large it doesn't, since it's quite rightly aimed at the broader population of gamers than at the much narrower population of developers) to "why game developers need a professional journal."

    ReplyDelete
  25. Game criticism..

    These are not the Droids you are looking for...

    ReplyDelete
  26. I'll try to be brief.

    You seem to be saying that criticism should be about providing answers for game developers, and that they're the only ones who are sufficiently qualified to fulfill this function. I'd argue every single point of that statement.

    I'll start with the obvious. A good question can be just as useful to a creator as a good answer. Moreso, probably, as most creators like to find their own answers to problems. Outsiders are actually probably more likely to pose the best questions than insiders, simply because their perspective takes less for granted. Admittedly, this also means they're more likely to pose a lot of stupid questions as well.

    Now, the idea that criticism should be for the creator has been dealt with above, so I'll skip past, just saying that I didn't appreciate Far Cry 2 until good criticism made me reassess my vantage point.

    Finally, the idea that criticism can only be made by game developers is a bit of a nonsense. Game development isn't some set of mysteries handed out to a select few. It's a collection of skills, most of which have been appropriated from other disciplines: everything from cinematography to economics to physics to architecture to programming informs how a game is made.

    I'll try and show what I mean by quoting your response to Fraser, above:

    "This last point is a good example of an area where knowledge of game development would improve criticism. Game developers tend to know immensely more about the breadth and depth of player reactions to a particular sequence than most writers ever seem to give credit [for]."

    Now, I'll admit to cheating a little here, as I know Fraser personally while you may not. But seriously, do you think he's unaware of the QA process?

    You simply need to listen the Portal developer commentaries, or to listen to Bungie talk about the lessons they learned about warthog placement, or god knows how many other sources. It's not hard to get an understanding of the iterative process. Certainly, there's no doubt specific lessons can be learned by going through that process yourself, but to argue those lessons are necessary to talk about why players respond in certain ways or what causes certain reactions is simplistic, at best.

    A photographer is better qualified to talk about the guiding lines and colours in Mirror's Edge than someone who's worked in sound design on AAA titles for the past ten years.

    I'll round off my comments by pointing out that many of the best critics of the past few years have either dabbled in game design or crossed over, often with great success.

    They didn't need to go back to school to do so.

    (Oh, and because it doesn't want to show, my full name is Mr Ak. Well, Mark, really, but Marks are like water - we cover two thirds of the earth.)

    ReplyDelete
  27. @Little Fish ""because you've never made a game, shut your mouth,"

    It was not my intent to say this that that seems to be a common interpretation. Good to know. Gamers can and should talk about games! It is a valid perspective. But for that small portion of the writing population who wants to do more, I recommend that they learn about other perspectives and disciplines, namely game design and development. If you aren't inclined to do that...awesome. Keep writing. No one will stop you.

    @Mr "A photographer is better qualified to talk about the guiding lines and colours in Mirror's Edge than someone who's worked in sound design on AAA titles for the past ten years."

    I think this is exactly my point, but I apparently need to state it more clearly. Noted for the next edit. As far as I can tell there are very few writers of game criticism who are bringing a deep bench of domain specific knowledge to their writing.

    Re: QA Process and listening to Portal dev diaries.

    No, I'm sorry, you are misinformed. Merely reading about something that requires years of practice to master does not instantly translate into experience or understanding. There is a reason why dancing judges are past dancers. It gives a level of insight that the uneducated observer cannot match.

    A few side notes
    * The outsider / insider / engineer rhetoric is fascinating. I don't have this view, but it is good to know that it exists. My initial assumption was that we are all in this together.

    * Ebert did in fact participate deeply in the movie making process. http://www.imdb.com/name/nm0001170/ Though Ebert happens to not be one, I would state in general that idiot savants are poor role models. Just because there are critics who are ignorant of the art and craft of games doesn't mean you should strive to be one.

    @Jason
    A community of people who are interesting in moving the art and science of games forward would be a great step. Lately I've been irritated at academic journals since they seem more intent on maintaining a business model than encouraging the spread and advance of knowledge. But the core of the idea has merit. I suspect however I'm looking for something a bit broader than a box to collect game developers within.

    At one point, I hoped that some of the energy being expressed by promoters of game criticism (notably Greg over at Play This Thing) would yield a cross fertilization between people in other fields and game development. That hasn't occurred. Instead, under the staunchly defended label of game criticism, we have a small extension of traditional game reviews. And we have various academic departments that don't seem to be contributing much except to themselves.

    These are fine enough for what they are. Still, it doesn't hurt to ask for more collaboration and attempts at building hybrid philosophies.

    take care
    Danc.

    ReplyDelete
  28. The problem is that you're using a terrible example to support your point. The article you refence is indeed atrocious: it says nothing, is poorly-written, and wrapped in pseudo academic language which makes you want to puke.

    But instead of calling out bad writing as bad writing, you seem to be calling out all criticism written by non game designers. You make this weird analogy to critics having books read to them, when the better analogy is critics who read novels but doesn't write novels. In literature, music, and film, there is a huge tradition of amazing criticism produced by non creators. You haven't supported the position that this can't be the case for games.

    ReplyDelete
  29. Dan: The problem with what you have written is that you're not just "ask[ing] for more collaboration and attempts at building hybrid philosophies.", you're dismissing everything that isn't this.

    And you're doing it in a way that is pretty inflammatory. I don't think you're picking up on the angry reaction to what you've said from someone that many game critics considered was 'on their side' - mostly because as far as critics and academics go, the result is initially always going to be a polite and well written attempt at rebuttal.

    ReplyDelete
  30. Hi Dan,

    You solicited feedback from me, so here's my input.
    http://iam.benabraham.net/2011/05/a-response-to-dan-cooks-blunt-critique-of-game-criticism/

    ReplyDelete
  31. I agree that there's a lack of writing by developers that pushes the medium forward creatively, but I disagree with your assumption that writing by people who don't know how to make games is irrelevant and useless.

    I find as a game developer that my responses to the games I play are increasingly different to my non-developer friends' responses. My opinions and responses are what they are, but unless my audience is entirely game developers, I need to have an insight into how non-developers think.

    Regarding Gamasutra Blogs, I think there's a pretty good split there between the "Expert" blogs and the regular ones. I think the Expert blogs should be kept to the high standards that you propose (attempts to advance the games medium from a developer's perspective), but the non-developer blogs (and other enthusiast outfits like Kotaku) should continue their key writing as well.

    Perhaps we just need a different name for each type of writing? It seems at the issue at the moment is just that both groups are trying to co-opt the word "criticism".

    ReplyDelete
  32. Pretty much agree with e5d16eb2.... except the last part about ignorance.

    What I *think* (reading as I go along) you're trying to say is that game developers should write more?

    But I don't agree if someone has no experience in the game-making process makes there writing of less or no value. Maybe thats not what you're trying to say, but I did get a hint of that from what I've read.

    I definitely don't want criticism to be dominated by developers. And this is coming from a developer-in-the-making.... hopefully.

    ReplyDelete
  33. You asked for feedback, but this post needs more help than I have time to give.

    Go back to the drawing board. Take Ben's comments to heart. If you honestly want to draft something, solicit private feedback. Don't publish it on your blog, throw around reductive, divisive language (parasitic, illiterate, impaired, ignorant, wannabe, bozo, etc. & etc.), then ask for the "the particular turn of phrase that seems to polarize folks so strongly."

    ReplyDelete
  34. Interesting article. If you're looking for feedback, I'd say it's more useful as a sort of developer primal scream than it is as a step towards better game criticism.

    I thought "The Blossoming of Shallow Game Criticism" section was a great summation of the situation, but like the rest of the article it was sometimes a bit harsh:

    "Yet the essay is little more than a series of personal descriptions of how he feels when he plays certain games... Once you strip away the laborious language, you have yet another bit of fluffy gamer opinion written by a young student."

    As a piece that expresses the frustrations of a journeyman game developer who reads this sort of thing, you pretty much nailed it. As a piece that's meant to encourage people who write about games to gain perspective and be more thoughtful, it's probably a bit confrontational. There is some good concrete stuff here, I would love for articles to clearly state the author's background. You used the word "honesty" and i think it's a good one. I'm sure some game writers deliberately obfuscate this because they don't want their opinions to be dismissed, but it's not fair to the reader that they have to wade through three paragraphs on a screed about development pitfalls before they realize that the author is not talking from the perspective of someone who has ever actually shipped a game.

    I think the request for better filters is really the most productive part of the article. We all know where the bad writing is, but as developers our methods of compiling the good info is pretty haphazard. It's often a co-worker pointing out a specific GDC talk that can't be missed, or an article on a particular website that may not be worth reading regularly, or a podcast that only appears irregularly, or it's just someone relating an interesting conversation they had with another dev at a bar during some conference or convention.

    I've worked at a few studios now where people regularly read the this site, and and Tip of the Sphere has been doing good things since it started up. You mention the Escapist, which does some good things. I'm fascinated by #AltGameDevBlog but it's a bit like drinking from the fire hose. The NYU lecture series can veer into navel-gazing, but overall it's a huge step in the right direction. But in general I think it's a more reasonable to expect developers to put together a better set of filters than it is to expect millions of people with internet connections and very strongly held (if uninformed) opinions to gain better insight into development and the art form.

    You mention Sturgeon's Law, but the article reads as request to overturn it in the particular area of game criticism. There will always be high-calorie, low-protein content in all media. We don't HAVE to read the tabloid rack at the super market, or watch the local news, or read the group blog sections of certain websites, but we do it because we're masochists or rubber-necking a car wreck. The real solution is to make sure people who want quality discussion and criticism can find it in a place without having to spend huge amounts of time weeding through the fluff. Why not start collecting useful criticism here?

    ReplyDelete
  35. Also, would it be worth defining the difference between consumer reviews (a place where I think the, "It's not for you" argument holds weight), and game criticism? I took most of your article as a referring to the recent deluge of people writing about older or classic games (and some current games), and picking apart their mechanics, narratives, and form, but reading through these comments I think there's some confusion here.

    Does the difference between someone writing a consumer review ("This is worth/not worth your $60") and someone writing a piece of game criticism ("Here's is why having a silent protagonist worked/didn't work in HF2, and here's why it did/didn't work in these other games") only exist in my head? Is this a distinction worth making or is it just semantics? Because I feel like people may be talking about two different things.

    ReplyDelete
  36. Interesting article. If you're looking for feedback, I'd say it's more useful as a sort of developer primal scream than it is as a step towards better game criticism.

    I thought "The Blossoming of Shallow Game Criticism" section was a great summation of the situation, but like the rest of the article it was sometimes a bit harsh:

    "Yet the essay is little more than a series of personal descriptions of how he feels when he plays certain games... Once you strip away the laborious language, you have yet another bit of fluffy gamer opinion written by a young student."

    As a piece that expresses the frustrations of a journeyman game developer who reads this sort of thing, you pretty much nailed it. As a piece that's meant to encourage people who write about games to gain perspective and be more thoughtful, it's probably a bit confrontational. There is some good concrete stuff here, I would love for articles to clearly state the author's background. You used the word "honesty" and i think it's a good one. I'm sure some game writers deliberately obfuscate this because they don't want their opinions to be dismissed, but it's not fair to the reader that they have to wade through three paragraphs on a screed about development pitfalls before they realize that the author is not talking from the perspective of someone who has ever actually shipped a game.

    I think the request for better filters is really the most productive part of the article. We all know where the bad writing is, but as developers our methods of compiling the good info is pretty haphazard. It's often a co-worker pointing out a specific GDC talk that can't be missed, or an article on a particular website that may not be worth reading regularly, or a podcast that only appears irregularly, or it's just someone relating an interesting conversation they had with another dev at a bar during some conference or convention.

    I've worked at a few studios now where people regularly read the this site, and and Tip of the Sphere has been doing good things since it started up. You mention the Escapist, which does some good things. I'm fascinated by #AltGameDevBlog but it's a bit like drinking from the fire hose. The NYU lecture series can veer into navel-gazing, but overall it's a huge step in the right direction. But in general I think it's a more reasonable to expect developers to put together a better set of filters than it is to expect millions of people with internet connections and very strongly held (if uninformed) opinions to gain better insight into development and the art form.

    You mention Sturgeon's Law, but the article reads as request to overturn it in the particular area of game criticism. There will always be high-calorie, low-protein content in all media. We don't HAVE to read the tabloid rack at the super market, or watch the local news, or read the group blog sections of certain websites, but we do it because we're masochists or rubber-necking a car wreck. The real solution is to make sure people who want quality discussion and criticism can find it in a place without having to spend huge amounts of time weeding through the fluff. Why not start collecting useful criticism here?

    ReplyDelete
  37. As an academic teaching game design and writing criticism, I've struggled with the "you have never done this for a living, so how can you criticise it" question for a while. I acknowledge that a lot of your criticism is valid (although it could be expressed more politely if you want people to actually change, rather that just react angrily).

    I will however ask the question: on what are you basing your views on academic writing? You seem to be referring mostly to blog posts rather than academic conferences and journals. I blog, but I happily admit it is not my best writing. My blog is more of a forum for me to experiment with ideas or get things off my chest. If you are looking for the best in games scholarship, blogs are not the place.

    ReplyDelete
  38. My longer response: http://roguelikedeveloper.blogspot.com/2011/05/dear-dan.html

    ReplyDelete
  39. "Yet, games are not and never will be the same sort of purely evocative media as music, video, writing or painting."

    "Games have more in common with functional works involving mathematics, psychology, governments, economics or other complex systems."

    I have to disagree with both of these statements, though since I'm not and never have been a game developer, you may not value my opinion on the matter much. Still, I am a working creative, and there are parallels with my own medium.

    You make the comparison in your piece here between game design and disciplines like mathematics, physics, and structural engineering. And while it's certainly the case that there are scientific and technological aspects to game design and development, the fundamental purpose of game design is utterly different from these disciplines. The purpose of physics--the reason why we do it--is to explain and predict. The purpose of engineering is to build. The purpose of game design--that is to say, the purpose of games--is to elicit a human response. And most of the time, an emotional response. In a very important way, focusing the discussion of games on the "whirring, clicking mechanical reality of games in practice" is missing the forest for the trees.

    It's the same in my own medium, photography. There are scientific and technological underpinnings to the creation of a photograph that a good photographer has to understand in order to create a good image. Physical phenomena like the angle of reflection of incident light or the additive nature of light color are crucial to understand if one is going to create a good photograph. And yet, they are quite unnecessary to understand in order to discuss why a photograph works. You don't need to know what aperture or film emulsion a photographer used in order to talk about the resulting image. You do need to understand your own reaction to it.

    I absolutely find use in reading photographers' books, essays, and blogs, in order to learn about their methods and techniques. But that has little to no bearing on the utility of photographic criticism. The two are on almost completely separate axes of intellectual discourse.

    And, certainly, I have found that many of the most insightful analyses of photographic works have come from photographers, but that has little to do with their understanding of photographic techniques and everything to do with the fact that photographers are often experienced consumers of photography.

    By all means, be frustrated if you can't find the kinds of discussions you want to read and participate in. But understand that those discussions are at least as narrow and inutile as the kind you're complaining about.

    ReplyDelete
  40. For the people who couldn't (or just won't) take the time to read that entire diatribe:

    "If you aren't a game designer, you lack the knowledge to meaningfully criticize games and for all I know, you just might be a ranting 13 year old with a worthless opinion."

    The one-analogy debunking of this entire post:

    Do you have to be a professional chef to say that you absolutely hate the way something tastes when you eat it? I hope not.

    ReplyDelete
  41. You don't have to be chef to say you dislike the food, but you would have to at least know how to cook a little in order to offer some constructive criticism about the taste that would help to improve it.

    ReplyDelete
  42. I can appreciate that some are offended by the tone of Danc's post. I also appreciate that blogs tend to ruffle feathers to get a good conversation going.

    But I agree that games can only be pushed forward by developer-writers who critique and communicate effectively.

    I've been reading Danc's work since his Nintendogs game design deconstruction (http://www.lostgarden.com/2005/06/nintendogs-case-of-non-game-that.html). As a game designer, this article provided me deep insight into the mechanical workings of a game that I had dismissed as casual and gimmicky.

    This insight, along with others from different Lost Garden articles, have provided me with information I can use to further my own work. They're useful to me because they're presented by a peer.

    Generally speaking, we learn the most from those grappling with the same challenges.

    To further myself and my work, I attend conferences, read postmortems, read gamasutra features and the odd developer diary. I try to learn from others' mistakes and find inspiration in what they're attempting. But the message isn't always clear; developers don't always successfully convey their valuable insights.

    As for wider games journalism, I don't get any appreciable value from them. But that's fine; they're not writing for me.


    Cheers
    -Dan Vogt

    ReplyDelete
  43. Smart writing. I will, however, make a couple of contradictory points, 1) because I'm a writer for IGN (though rarely in charge of criticism) and 2) because I'm an asshole. Sorry.

    I always take extra care with my words when criticizing the literacy of others. "...but they are unable to read nor write" should read "but they are neither able to read nor able to write," and after this you can go ahead and call people illiterate.

    Also, if we follow the reasoning that only a person that makes games can write worthwhile criticism of games, would we not also have to agree that you'd have to write enthusiast-press game reviews before you could make worthwhile criticisms of them? For the record, I don't believe either of these two statements.

    I think there's a lot of value in what you're asking for. Much of it would probably sail over my head, but I know better than to call it worthless.

    ReplyDelete
  44. Danc: Thanks for the thoughtful reply. I left a follow-up comment yesterday but it seems to have been swallowed up before it reached the page, the way these things sometimes are.

    I won't rehash the whole thing, as others have now covered a lot of what I had to say. However, I would like to address the idea you put forward in your response to me, that writers fail to understand how empirical user testing is superior to criticism based on individual experiences.

    The types of feedback developers typically use - metrics, surveys, play tests and so on - are assessment tools. They answer questions. They can tell you if players are getting lost, if players are having fun, what players are confused by, what players are ignoring. This is a vital part of the design process, and an indispensable guide in reaching the project's goals.

    Criticism has a different aim, although the two may overlap. Criticism is a process of exploration and evaluation. It often asks questions that could not possibly be covered in empirical testing. See for example Troy Goodfellow's long-running series on National Characters in strategy games. There is no comparison between that kind of criticism and, say, the user testing Firaxis conducted for Civilization V. The two are different beasts.

    Talking about what he calls "metrics fetishism", Chris Hecker said "The metrics are the craft, and the intuition is the art." Without meaning to put words in Hecker's mouth, it seems a neat illustration of the difference. Empirical testing (including metrics) is part of the craft developers employ to achieve their goals. Creative thought (including intuition) is the process developers use to choose their goals. Criticism speaks to the art of game design; user testing to the craft.

    That's the kind of "looking at games" that I meant when I said developers would benefit from being able to step away from their role and see the game from another perspective. Not just the shallow-but-broad input of multiple game testers, but also the narrow-but-deep input of individual critics.

    To return to the bridge allegory: the engineer seeks to improve the design of his bridge. The geographer studies how the bridge will change the population distribution of the area. The historian places the construction of the bridge in a historical context. The poet writes a creative response to the bridge. The lawyer looks at how the road rules may apply to traffic on the bridge. They each have a different purpose, but that doesn't necessarily make them antagonists. They are, loosely, "all in this together", but they are not all pulling in the same direction, or indeed all pulling on the same rope. Would they benefit from a fuller understanding of each other's fields? Probably, but it's not essential to what they do.

    ReplyDelete
  45. The fact that the majority of games are bad means just culling the group to actual designers / artists / musicians / coders / writers who have published game material won't get you what you want.

    ReplyDelete
  46. Much like Objectivism is philosophy for people who are unaware of what philosophy is, this article is sort of the critical equivalent of a 3-year-old screaming at the top of their lungs "EXCUSE ME!!!!!" I.e., Danc has some vague concept of form, but seems cognitively incapable of relating it to its actual function. Perhaps if he read some actual, you know, criticism, it might help...though I obviously have my doubts. Nonetheless some starter recommendations: I Lost It At The Movies by Pauline Kael, Air Guitar by Dave Hickey, that new and utterly mind-blowing Ellen Willis anthology. Or just go to Robert Christgau's website and type whatever into the search bar.

    ReplyDelete
  47. @Fraser "Empirical testing (including metrics) is part of the craft developers employ to achieve their goals. Creative thought (including intuition) is the process developers use to choose their goals."

    A fascinating and false manner of segmenting art. I personally use both the empirical and creative tools of game design to achieve a result (very much in concert with the players). To separate empiricism from creativity in such a simplistic manner is to ignore the reality that scientific and logical thoughts are as deeply creative as the act of poetry. There is no 'hard part' or 'soft part'. They melt together and one without the other is devastatingly incomplete. Hence my original critique.

    One thing I've noticed over the years is that reaching this state of understanding often requires an grueling immersion in the practice before the art of it all becomes apparent. I'm not sure how to share this view other than suggesting you go down the path of making games yourself and see how things look on the other side. I'm all ears to less demanding solutions.

    "..see the game from another perspective. Not just the shallow-but-broad input of multiple game testers, but also the narrow-but-deep input of individual critics."

    Perhaps a slightly different flavor, but have you ever stopped by an MMO forum and read the dozens of pages of lovingly crafted encounter descriptions, art critiques, mechanics teardowns and oh, so much more. It comes from a rather different tradition, but the content contains surprisingly similar themes. All this, plus the articles, plus the long and highly personal emails all goes into the creative engine.

    It is rather difficult for game developers to avoid the "narrow-but-deep input of individual critics." :-)

    "Would they benefit from a fuller understanding of each other's fields? Probably, but it's not essential to what they do"

    Huh. As a builder of games, I cannot afford such siloing of knowledge. Every day pushes my skills and ideas to the limit. In order to create the thing that others have every right to insist on reacting to in a narrow fashion, I feel I must constantly be looking to integrate new perspectives.

    Even this discussion, as bloody as it might seem on the surface, has been immensely educational. 'Criticism' is certainly not my field, but I become better at what I do by engaging (however poorly) with the concept.

    I assumed the same of those who practice 'game criticism' but from your comments that seems not to be a shared value.

    In all of this I sense an odd fear. What is so dangerous about being an engineer-geographer-historian-poet-lawyer? I only see benefits to the community as a whole. The only risk is that individuals comfortable in their current niche might need to change and grow.

    take care,
    Danc.

    ReplyDelete
  48. Danc: "I cannot afford such siloing of knowledge" and yet nothing by a non-developer is worth reading? Did we flip positions at some point in this conversation? :-)

    If my comments suggested that critics should avoid knowledge, they were poorly written, because that's not what I mean at all. I mean to draw a distinction between essential knowledge for a field and other valuable knowledge. I dispute the idea that knowledge of game design is an essential requirement for a game critic, as you've suggested above. Rather, it is one of many bodies of knowledge that might enrich the criticism. (It may also have drawbacks, but others have argued that point before me.)

    I agree it would be preferable if everyone knew everything, but rarely do you get the chance to read a piece of game criticism by Doctor Manhattan.

    Re: "A fascinating and false manner of segmenting art." I'll take the compliment (cheers) but stand by my point. Craft supports art and art supports craft; that doesn't make them the same thing.

    ReplyDelete
  49. "I assumed the same of those who practice 'game criticism' but from your comments that seems not to be a shared value."

    Yes, instead of learning by posting a long, solipsistic, condescending, self-debunking screed that's "provocative" enough for others to feel compelled to explain to me everything I got wrong, I've always felt it more polite to actually learn something about the field I'm talking about first. Guess I'm funny that way.

    ReplyDelete
  50. Instead of focusing on the bad, I'd love it if people could look for what's good and point it out. A more constructive article to write as follow-up would be:

    "Examples of the Best Game Criticism on the Planet"

    Now there's an article worth writing!

    For those looking for the GOOD stuff, may I humbly recommend that you google:

    Emily Short
    Chris Hecker
    Ian Bogost

    I'd love to find more writers in the community that you think are top-notch. Let's find the good stuff and highlight it.

    Anyone have some examples of top-quality game journalism? Please post them!

    ReplyDelete
  51. For the sake of completeness, I thought I should link my response to this piece here, too: http://redkingsdream.com/2011/05/doing-things-with-critics/

    ReplyDelete
  52. Dan: there seems to be an easy answer to the need you feel to have "a new field of discussion" that supports "better methods of filtering game criticism" -- start your own peer-reviewed publication.

    If one of the problems is that a lot of new writers are unaware of the basic steps involved in game development, and another is not knowing or just rehashing a lot of the topics in MUD-DEV archives, why not found a new journal that helps educate and disseminate what you want to read? Be agile and test the market. People will either respond positively if you raise the bar, or else we'll be pretty much in the same situation. Except there will be even more academics and web comics.

    ReplyDelete
  53. Here's another, rather pithy, take on the piece, though it doesn't engage at all with the whole "game criticism sucks right now" angle because that wasn't the bit I felt was worth commenting on:

    Developers! Developers! Developers!

    ReplyDelete
  54. The most telling part of this article is the image you have at the top.

    "To escape criticism, do nothing, say nothing, be nothing."

    Your article frames this quote as an indictment against critics, implying that they stifle creation. The reality is that it is an indictment against creators who look for validation for their work in criticism.

    Basically, the job of a critic isn't to help creators, but to inform consumers. Pleasing them should not be your goal, just as helping you be better at creation should not be theirs.

    Granted, you each may be able to learn something from the other, but your jobs are not (and should not be) as intertwined as you seem to think they are.

    ReplyDelete
  55. In reading the post and the responses, it occurs to me that part of the issue is the lack of a commonly accepted vocabulary and structure for critics and creators. The critic praises a game for 'immersion' and 'replayability', terms so broad as to mean nothing to the creator.

    To quote Ben Abraham: "There are a myriad number of characteristics that go into evaluating a game on this criteria: variation, openness, procedurality, linearity, aesthetics, etcetera, etcetera. Notice here that I’m not avoiding replayability not just because It’s Not A Word but also because it obscures all these factors that go into what gets lumped under ‘replayability’."

    So the critic lacks the language to describe to the creator both what he loves and hates about the creation, leaving both sides confused on how to actually move the discussion forward. Certainly part of the discussion will always remain subjective; No amount of technical advice will turn Salieri into Mozart. But there will also always be a technical element that can be discussed in a common language.

    When two teams play in sports, we attribute part of their performance to 'heart, character and desire' - things not measurable on any stat sheet. At the same time, we have a commonly defined set of measurable metrics (batting avg, completion %, etc.) that provide insight to the team on how they can improve for the next game. Wouldn't it help both the critic and the creator to speak the same language so we can improve our next game?

    ReplyDelete
  56. Seeing as that I was cited here as being part of the problem, I present my response to this piece. Enjoy reading it, if you have time, that is.

    http://flickeringcolours.net/v2/?p=187

    ReplyDelete
  57. That was a well-written and insightful response, Adam. Thank you for the forbearance that I utterly lack.

    ReplyDelete
  58. Hmm…It’s an interesting idea but I think the bigger issue is how we're reviewing games, not necessarily who is reviewing them. The issue is that we're trying to determine what's "good" or what's "bad" which just doesn't work. Especially with such a variety in way people enjoy games now and the types of game enthusiasts there are. It seems every game affects everyone differently in varying degrees of course. I for one found a game like Borderlands completely enjoyable while some of my friends think it's just a boring "Shooter" with just "Grays and Browns" and "repetitive" game-play. I especially think scores and final judgments on the game’s quality are just counter-productive to the medium, industry, and creative merit of games.

    I think almost all games have that in common: They should be enjoyed. Though I'm sure if it's a sequel a developer is either trying to please their existing fans, getting new ones, or probably both.

    What I think would work is just measuring a reactionary experience, or even comparing what you got against what you expected. This isn't just my idea either; this is what UX testers look for when testing an interface against an audience. They watch their reaction while using it; if they're frustrated they are frustrated there's not much room bias or agendas that is exactly how the interface affected them at the time of use. This way just about anyone could review it and offer a valuable critique and the more there would be like that, the more accurate it would be for helping developers improving the game.

    I'm just a web designer & developer myself but whether you're designing games or websites, there are some similarities. When I want useful feedback for improvement, I look for reactions instead of judgments. I don't need people to try and understand what makes my site good or bad, what makes features work or not work. What I do need to know to make good decisions is how it makes them feel, like it doesn't matter how clever my commenting system is if no one enjoys using it or gets some satisfaction from using it to comment. And a designer/developer wouldn't even be the best source for criticism either as they've either seen it before or have enough experience to figure those interfaces more easily than the intended audience.

    ReplyDelete
  59. Even art is measured in a pretty similar way. It's either continually enjoyed by enough people to carry on or it isn't and it gets forgotten. While there have been many contests & competitions, those decisions are often long forgotten and sometimes a runner-up's work has been enjoyed more by society than it was by the judges.

    However, if you have to try and judge what's good or bad, then I completely agree that it should be someone who has some training or experience with what "good" is. But there's still going to be plenty of the same faults as different developers value different things. For instance, I just had a twitter conversation with one who thinks it's ridiculous that games are valued by their difficulty, and the specific game he was talking was Dark Souls. So while developers would certainly have a better, more technical grasp of what they're experiencing there would still be a ton of room for inconsistency.

    But don't get me wrong; I know this isn't a very realistic solution. It wouldn't be nearly as financially viable. If someone reads an IGN article on a game they're expecting good things from and the reviewer informatively explains their experience with it. There's not going to be any traffic coming to that site because the reader disagreed with the reviewer's judgment and threw a fit on Twitter which encouraged more people to visit that site and read that review.

    But I do think for the sake of gaming using more reactionary reviews, more than trying to determine what's good or bad and right or wrong with the design, will be more useful for developers, and even offer readers better insight as to what they could get out of the game instead of what the reviewer thinks the totality the game offers.

    ReplyDelete
  60. This comment has been removed by the author.

    ReplyDelete
  61. Dan, as a fellow practitioner, I appreciate you having the courage to tackle this.

    As I have slowly, jerkily learned more and more of the craft, I have become more and more dismayed at the almost complete lack of good design craft writing. It is nearly non-existent. What is present is mostly actively misleading.

    I think people like us have to recognize that the ecosystem of academia-style game studies really exists largely on its own, for its own purposes, which have less to do with the actual construction of games than we might think at first.

    Personally, I take a dim view of any thought with little application to reality. This does absolutely not preclude an appreciation for philosophy or humanities, it only requires a very strong filter while reading fields like this which are infested with navel-gazing. Philosophy, linguistics, etc do have applications when done properly. Writers like Nassim Taleb and Steven Pinker have proved this to me, when I had almost lost hope.

    Much - possibly most - game criticism is drivel. We should recognize, however, that there is a decently large space of writing which is worthwhile for purposes having nothing to do with design craft. The perverse economics of the education system have vastly inflated this sector beyond its optimal size, and allowed it to collect a mass of intellectually useless hangers-on. The sheer weight of these followers serves to obscure both worthwhile game criticism and worthwhile design craft. I think the same thing happens in any field where it is hard to judge competence without having it yourself. Without proper "global" filters, the chaff obscures the wheat until you can barely even perceive the wheat at all.

    My takeaway is that we need better filters.

    Finally, in response to the numerical lean of the responses to this article, just remember the selection bias which determines who has time to read these blogs and write in places like this.

    "It's hard to get a man to understand something if his job depends on not understand­ing it."
    - Mark Twain”

    ReplyDelete
  62. While, blogger just lost my long explanation of how your characterization of Adam is both inflammatory and inaccurate. The short version is that it's extremely insulting to say a doctoral candidate with a bunch of published papers is "a student acting as an academic". That might be an appropriate thing to say about an undergraduate, but by the time you're as far down the path as Adam has gotten, he very clearly is an academic.

    Additionally his blog post is not "masquerading" as anything, and it definitely isn't a playthrough (even by your own definition). Whatever you want to say his post is, there's just no evidence that he was trying to pass it off as something it isn't There are things to critique in that particular piece of writing, but your characterization of him and his post is both incorrect and basically insulting.

    ReplyDelete
  63. You should move your "draft status" disclaimer to the top of the piece until you're ready to remove it.

    Also, each new draft should be published as a new post. At the very least, that will push the edited content to feed subscribers. It also has the benefit of transparently preserving the evolving context of the discussion.

    ReplyDelete
  64. There is probably meaningful research about the role of critics in any field (ie painting, litterature). For sure they must have some knowledge of how the works are done.
    I would be interesting to compare the status of critics in videogame with those of other fields.
    However I must admit I often had the feeling that critics in videogame somewhat failed still missed the point in some way, either the point of videogame themselves and the way they are done, either the point of having a clear idea of what the game criticism should be.

    ReplyDelete
  65. We need more than criticism, we need proper history in the art of gaming.

    ReplyDelete
  66. Dan -- Is this now the final draft? Sounded like you might be planning a third?

    ReplyDelete
  67. Actually, I have a lot of sympathy with Dan here. As a game creator, it can get terribly frustrating the amount of effort that goes into analysing our output without any real attempt at identifying solid foundational problems and more importantly solutions.

    Basically, it feels as a creative, that all critique is negative. It isn't particularly surprising that Dan has this reaction.

    I'll give you a case in point:

    visit www.digra.org (the "digital games research organisation") and find me the papers that actually contribute to the field of making games. Contrast this with the number of papers that discuss the nature of games, or the effect of games on players, or on the theory and ontology used in game criticism etc.

    My point being that there is a lot of energy directed to the current output of games, without much thought on the direction of that output and on the fundamental contributions to it moving forward.

    Honestly, if you read many of the game design books on the subject I think there is a huge amount of effort needed still in this area (most game design books are shallow and almost entirely useless with a few key exceptions).

    So I think what Dan is suggesting (and I support) is that there is an imbalance currently where so much effort is given to the response to the extraneous view of games, rather than the creative aspect. Much of what is passed off as creative similarly has very little to offer but anecdotal evidence achieved via personal experience.

    I agree that perhaps Dan could have found a different term to use for the title of the post and that maybe game criticism isn't quite the term we're looking for. But I'm wholly on Dan's side in that I yearn for a situation as a creator where I can read useful critique of games that actually teaches me something that I could not grasp intuitively by merely playing the same games.

    I suspect the bigger issue is that the game industry itself has not figured out a method of critique that allows for development of design theories and critical appraisal as much as any problem with external review/criticism.

    I will however, offer that there are certainly efforts to move towards the type of critical analysis that Dan is suggesting, but it may take several generations for us to achieve the shift. My own design undergraduate programme has this goal as part of our curriculum and hopefully others are attempting to do the same. Eventually as the tools of constructive criticism become more engendered within the industry, we will see a more solid basis for moving the narrative of the creation of games forward.

    I'll be posting some of our research findings on altdevblogaday at some point, which hopefully will contribute to the kind of writing that Dan (and I) wish to see.

    ReplyDelete
  68. (This comment actively shocks me. What do you think game developers are *doing* with those metrics, surveys, play tests and thousands upon thousands of iterations on a game?)

    This might as well be where I jump in.

    Danc, while I appreciate your perspective, I can't help but apply it to moviemaking. You disparaged the comparison a bit, but is an equally technically challenging enterprise, and another relatively young medium of expression.

    Three things still stand out, if you go with that analogy.

    First, you seem to be claiming that "educated layman" film criticism isn't truly necessary, because we have focus testing and test audiences. That's what your "thousands of data points" comes down to when you apply it to television or film. Anybody with a lick of sense would realize that that's just silly. Focus (and play) testing is an incredibly useful tool, but many a movie has been absolutely ruined by too-slavishly following what was written on the little test audience cards, and there are certain films beloved (or at least admired) by critics that test audiences almost certainly despised.

    Second, you seem to see criticism as a scientific enterprise. That's not really what it's about. Connecting film (or game design) to sociology or economics is very nice, but any social scientist is going to tell you that it isn't actually going to necessarily be that fruitful. Sure, EVE needs an in-house economist to keep their byzantine in-game economy running, and a film studio is going to need to be mindful of the money, but an economist isn't going to be much good when it comes to sorting out what's any good or not. Criticism is supposed to be about discussing the medium as a form of artistic expression. Yes, that's going to get back to the humanities at some point, because the social sciences can't (and don't try to) answer that sort of question.

    After all, how the hell can anybody sort out how to "make it better" when nobody is discussing what better is?

    3)This still boils down to "film critics should be film directors". Game design is technical, but so is film direction; they just require different skills and backgrounds. Yet anybody who said that above sentence would get laughed out of the room! The most celebrated critics in film may have done a bit of writing here and there, but that doesn't mean that they were celebrated directors. In fact, there are practically no celebrated film critics who are also celebrated film directors, or at least none I can think of.

    Why would that be, if it weren't for the fact that creating and critiquing require different skill sets? Most game designers probably aren't good enough writers to be a top-notch critic in the first place, since writing is as much a craft and an artform as game design is.

    (Albeit one that requires a lot less math.)

    Finally, I think a bit of realism is called for here. The problem with game criticism is that there's little audience for it and no money in it. I'm sure we could go off on a very entertaining tangent about how publishers, developers, and audience alike have little appetite for real, honest-to-goodness capital "C" Criticism. The point is, though, that anybody who can make games will make games, if only because writing about the things is seen by too many as little more than an entry path into the industry.

    What you're prescribing is not better criticism, but more bloodless "game reviews" from low-paid freelancers.

    Thanks for an interesting piece. As someone who exults in the growth of "game criticism" as proof of the maturation of the medium, however, I can't help but hope that your attempts to dissuade fall on deaf ears. I want MORE, not less. Let the audience decide what they see as valuable.

    ReplyDelete
  69. It sounds to me that you're asking less for a wholesale revolution in games journalism than for the creation of a professional journal of game development.

    I wholeheartedly endorse this post. There is room for both discussion of development by professionals and woolly-headed criticism by tweed-choked academics. They just both need to be clearly collected and curated. The Internet isn't very good at that. A serious professional journal would be.

    ReplyDelete
  70. @CraigB

    "That's what your "thousands of data points" comes down to when you apply it to television or film. Anybody with a lick of sense would realize that that's just silly. Focus (and play) testing is an incredibly useful tool, but many a movie has been absolutely ruined by too-slavishly following what was written on the little test audience cards, and there are certain films beloved (or at least admired) by critics that test audiences almost certainly despised."

    This is probably worth it's own post.
    - Movies are a technically complex task
    - Making movies contains multiple iterative processes.
    - On the surface, this seems to make games and movies comparable in terms of the creative process.

    However they aren't. Playtesting is not just a 'useful tool'. It is a fundamental reality of making games. It is not a secondary process tacked on at the end by business people as it is with movies. A developer that does not do playtesting is 99.999% of the time going to make a broken game that is not fun, not evocative, not emotion and not functional. This goes for small games, big games, board games and video games. It is a basic property of games.

    Games build experiences by using carefully crafted stimuli to evoke a specific emotion in a specific circumstance. This results in building blocks of experience that are used to construct a scaffolding. If the base building blocks (really learning loops) are not reliable, the whole thing fails. However, you can't just 'imagine' a building block of a game. You run experiments on players. And you tune and tune and tune until the learning loop works reliably. An image, video or sentence can't 'fail'. A learning loop can.

    With movies, the primary cycle of editing involves the creator/editor as both the audience and the problem solver.

    With games, because they are functional hierarchically nested learning loops in which a person's experience changes as they play the game, the creator / editor is inherently corrupted and need to go outside to a representative audience in order to iterate honestly upon the game.

    This is a fundamental difference in the creative process. Unlike movies where 'play test' are excuses to compromise the creative instinct of the director, a 'play test' + iteration in a game is how fun and emotion are crafted.

    Every single game you have every loved has gone through this basic process of game design. You cannot make a playable game without listening deeply to players. (There are mythological exceptions in the boardgame space, but they tend to prove the rule.)

    Games are not movies. Nor books. Nor paintings. Nor plays. The creative process of making a game is nothing like that of such media. I've painted, written, been in plays and made games. Making games is a radically different form of authoring. We do a grave disservice to the process of understanding games by assuming the same rules and theory from linear, evocative, non-interactive media apply wholesale.

    We create systems of rules, affordances and players. We can't make games without players. It doesn't work.

    take care,
    Danc.

    ReplyDelete
  71. Danc, my post didn't get published. I think that's a bit of a problem, considering that people can't judge it for themselves.

    Playtesting is "a fundamental aspect of making games". Fair enough, though I suspect that there are indie games out there that haven't been playtested to anywhere near the same extent that a Valve game is—but then again, there are independent films out there that aren't shown to test audiences, whereas big budget productions are. Certainly your comment about "designer as inherently corrupted" does apply to pretty much any other medium, as others have pointed out. A writer without an editor will have plenty of "failed sentences".

    (Trust me. Sentences CAN fail.)

    But the real point, and it's been made by other people, is that neither the iterative nature of game development nor the existence of playtesting somehow exempts games from deep critical analysis.

    They are NOT engineering. They are NOT physics. Like photography, television, and film, they use these things, but the whole point of the enterprise is expression in some form or another. The fact that Lord of the Rings employs CG modelers, foley artists, and other technical experts to bring the vision to life does not mean that you should be fully conversant with any of these things in order to properly critique the film. Ebert isn't a director, nor is he a CG artist; but people still pay attention to what he says about how LotR affected modern film. You seem to think that the only person who could possibly do so is James Cameron. And, um, no.

    I'm not sure that you seem to know what criticism is for. Sure, it's subjective. But, to me, it's not about improving the craft. It's not even necessarily about sorting out what is or isn't art. It's about sorting out what a work is trying to express (whether from a ludology or narrative point of view), where it fits in with the other works in the form, and the extent to which it succeeds in accomplishing what it set out to do.

    That's one of the reasons I'd prefer a critic who has demonstrated that they know about the development of the form and the tensions within it, and how to properly write about it over someone who's an ace coder. If a coder has never played Civilization, System Shock, Doom, Super Mario Brothers, Fallout, or the other games that serve as touchstones for the form, he's going to be a terrible critic, just as a film critic who's never seen Citizen Kane and Taxi Driver is going to be a terrible film critic.

    (That's also why your MMO specialists aren't that useful. If your only experience is WoW, how can you judge any other MMO's worth as a development in the medium?)

    Again, what you seem to be hungering for is some sort of professional journal where people can develop the craft of game design, programming, etc. That's fine. That's great! I wholeheartedly support that. But there's also a role for the gaming equivalents of Pauline Kael, A.O. Scott, and Roger Ebert.

    Oh, and by the by: if you've never made a movie, how can you possibly judge how comparatively technically difficult it is? By your own reasoning, you can only judge those things that you make.

    ReplyDelete
  72. Hey CraigB...not sure what happened to your original post (it came into my email, but didn't make it to the site for some reason...I think blogger has been acting up lately.) If it is okay with you, I'll past it here:

    CraigB says..

    (This comment actively shocks me. What do you think game developers are *doing* with those metrics, surveys, play tests and thousands upon thousands of iterations on a game?)

    This might as well be where I jump in.

    Danc, while I appreciate your perspective, I can't help but apply it to moviemaking. You disparaged the comparison a bit, but is an equally technically challenging enterprise, and another relatively young medium of expression.

    Three things still stand out, if you go with that analogy.

    First, you seem to be claiming that "educated layman" film criticism isn't truly necessary, because we have focus testing and test audiences. That's what your "thousands of data points" comes down to when you apply it to television or film. Anybody with a lick of sense would realize that that's just silly. Focus (and play) testing is an incredibly useful tool, but many a movie has been absolutely ruined by too-slavishly following what was written on the little test audience cards, and there are certain films beloved (or at least admired) by critics that test audiences almost certainly despised.

    Second, you seem to see criticism as a scientific enterprise. That's not really what it's about. Connecting film (or game design) to sociology or economics is very nice, but any social scientist is going to tell you that it isn't actually going to necessarily be that fruitful. Sure, EVE needs an in-house economist to keep their byzantine in-game economy running, and a film studio is going to need to be mindful of the money, but an economist isn't going to be much good when it comes to sorting out what's any good or not. Criticism is supposed to be about discussing the medium as a form of artistic expression. Yes, that's going to get back to the humanities at some point, because the social sciences can't (and don't try to) answer that sort of question.

    After all, how the hell can anybody sort out how to "make it better" when nobody is discussing what better is?

    ReplyDelete
  73. CraigB says (continued...)



    3)This still boils down to "film critics should be film directors". Game design is technical, but so is film direction; they just require different skills and backgrounds. Yet anybody who said that above sentence would get laughed out of the room! The most celebrated critics in film may have done a bit of writing here and there, but that doesn't mean that they were celebrated directors. In fact, there are practically no celebrated film critics who are also celebrated film directors, or at least none I can think of.

    Why would that be, if it weren't for the fact that creating and critiquing require different skill sets? Most game designers probably aren't good enough writers to be a top-notch critic in the first place, since writing is as much a craft and an artform as game design is.

    (Albeit one that requires a lot less math.)

    Finally, I think a bit of realism is called for here. The problem with game criticism is that there's little audience for it and no money in it. I'm sure we could go off on a very entertaining tangent about how publishers, developers, and audience alike have little appetite for real, honest-to-goodness capital "C" Criticism. The point is, though, that anybody who can make games will make games, if only because writing about the things is seen by too many as little more than an entry path into the industry.

    What you're prescribing is not better criticism, but more bloodless "game reviews" from low-paid freelancers.

    Thanks for an interesting piece. As someone who exults in the growth of "game criticism" as proof of the maturation of the medium, however, I can't help but hope that your attempts to dissuade fall on deaf ears. I want MORE, not less. Let the audience decide what they see as valuable.

    ReplyDelete
  74. Hi Danc,

    My favourite part of the current article is the qualification of a new term: Games Analysis.

    I'm an experienced software engineer, on the first steps toward becoming a well rounded game designer. I have picked up many of the best books (Jesse Schnell comes to mind) and follow some of the most interesting blogs (Click Nothing), but separating technical analysis full of insight from subjective criticism has always been a challenge.

    Gamasutra obviously features post-mortems very heavily, and has a healthy offering of game analysis, but beyond that I look to trade sources such as Game Developer Magazine and GDCVault, or direct from experienced authors, such as Chris Hecker and Jon Blow.

    It is clear from the comments and from the general community that has arisen around Game Criticism that the term is now quite loaded, in the sense that is has a shared and broad meaning, and often tends to the subjective, rather than systemic. The emotive language in earlier drafts clearly challenged that shared meaning for many people.

    Game Analysis instead suggests a much more analytical and rigorous approach, which better reflects the conceptual models and deconstruction that game designers and creators use in furthering our understanding of the craft.

    By being clear in the distinction of these two separate forms of writing, your advocacy for more analysis is less confrontational, rather than suggesting "criticism" has more value (to the medium) when it is technically/systemically based.

    ReplyDelete
  75. Huge apologies for jumping into the comments without reading the preceding ~60 comments, but I just wanted to say: This article is amazing, thank you so much! I'm hoping to write 'case studies' of games over the next few weeks (at http://immutablestrings.blogspot.com/ ) that might, if I'm lucky, count as critical analyses.

    ReplyDelete
  76. Hi Dan, after reading your initial post + (most of the) follow-up comments, I just wanted to give you a virtual thumbs-up for tackling such a far-reaching and politicized topic.
    In-line with your Game Criticism delineations, I've now added a disclaimer to my essay repository (found at www.metakiosk.wordpress.com) to more effectively capture what type of writer I am (Student) and also to better define my essays' purpose (Connecting with Humanities + Industry Analysis).
    Thanks,
    /D

    ReplyDelete
  77. Good points. Thought provoking. And thanks for the positive examples. (Look forward to reading them.)

    The question you touch on about knowledge of production being a prereq for insight into what it is produced is one that pops up here and there and everywhere - like a little prairie dog. Can you create insightful critique of a painting without knowing how to mix colours or create paint sfx? Analogies: poetry, film, architecture, religion, cars... [your functional cultural artefact write-in candidate here].

    My perspective is that the experience of the user as a critic - and hence the quality of their output - is enhanced by an understanding of (a) the limitations of the production environment, whether it be working memory, in the case of syntax, or tools, in the case of the engraving of stone tablets, and (b) the (possible emergent, possibly violated) conventions of a form.

    @HAStark

    ReplyDelete