Monday, December 31, 2012

Understanding randomness in terms of mastery


Instead of categorizing games as either 'games of skill' or 'games of luck', I see games with randomness as being a subset of 'games of mastery'. This view helps the designer see randomness in games as the intersection between both the player skill set and the game mechanics. By understanding the underlying skills involved in mastering randomness, we can build more meaningful games.

Discerning cause and effect from noise

One of the fundamental elements of any game is how the player learns to distinguish useful patterns from environmental noise. Without a mental model of how a system works, most games appear random or at least arbitrary.  (Randomness is a concrete property of a rule set. However perception of randomness is a state of mind that can exist independent of the rule set.)

With time, experimentation and practice, some players build up a mental model with conceptual tools that let them manipulate the system to reach desired outcomes. They transform from unskilled players into skilled players.

The idea of noise is a broad one. A cluttered scene with hundreds of objects is said to be noisy. A combat scene rife with particle effects and crazed camera angles also is noisy. Noise is the extra stimuli that hides the next conceptual insight.

The perception of noise vary based off the player's skill in understanding and filtering various classes of noise. A chess board in the middle of a game is highly noisy to a new player trying to simply figure out how a knight moves. All the extra pieces and their subsequent movements are extraneous to what the player needs to learn next. However, that same chess board offers reams of insight to the advanced player. They are able to process the information and predict future outcomes based off their sophisticated cumulative models of chess cause and effect dynamics.

Categories of noise
Noise comes in a variety of categories that flow naturally from the basic skill atom we see in most game loops.


  • Action Noise:  Uncertainty, extraneous elements or unmastered complexity in the player action. 
  • Rules Noise: Uncertainty, extraneous elements or unmastered complexity in the processing of the blackbox rules. 
  • Feedback Noise: Uncertainty, extraneous elements or unmastered complexity in the stimuli that shows the effect of the player's action. 
  • Model noise: Uncertainty, extraneous elements or unmastered complexity in the player model of the situation. 
Each class of noise has its own category of skills associated with filtering the meaningful signal. In a hidden object game, the visual complexity of the scene creates noise. Advanced players cope with this by mastering silhouette detection, efficient visual search patterns and object association skills. A good hidden object game players is measurably better than a new player.

Randomness as a form of noise

From this viewpoint, randomness in the form of internal dice rolls can also be treated as a class of rules noise. There other forms of randomness that map onto Action Noise and Feedback Noise, but randomness as rules noise seems to cause people the most trouble.

Since randomness is just another form of nosie, we can expect it to have several key properties:
  • A model: There is often an underlying pattern or model that helps players deal with the randomness
  • Model ignorance: This model will not be readily apparent to new players. 
  • Learning curve: With time and education, players will learn how to appropriately deal with randomness. 
  • Learning variables:  There are also likely important variable for the system that make learning to deal with a system's randomness easier or more difficult. 

Skills for player modeling of randomness

Probability and statistics provides use with a set of mathematical skills for dealing with randomness.  Players instinctually use roughly equivalent concepts but modified by a set of well document unconscious biases.  Instead of summarizing all of probability theory, let's cover the symptomatic player behaviors you'll see in the field. 

Existing heuristics
When a player lacks a mental model for a phenomena, their immediate instinct is to adapt an existing model. They look for past experiences and skills that fit the current situation and then act accordingly. Players can pick from their personal experiences or they may use forms of social proof to follow what others are doing.

There is strong evidence that many of our default heuristics for dealing with randomness are instinctual and perhaps biological. As such, evolution selected for survival, not necessarily accuracy. This leads to a wide array of biases such as loss aversion or difficulty processing large odds.

In general, reliance on existing models is a poor method of dealing skillfully with random behavior. It is better than a purely random reaction in a pinch but is not well adapted to the engineered random systems players face in modern designs. The player can't know the properties of the random system beforehand and the wide range of different types of randomness mean that they will likely guess incorrectly.

Sampling
Perhaps the most confusing aspect of randomness it that it occurs as a result of an interaction loop. In a simple slot machine, you pull the handle once and get a single result. By its very nature, it is difficult or impossible to detect what that result might be.

So the first skill players acquire is the ability to take multiple samples of the event. For very rare events, you may need to take large numbers of samples. For common, more predictable events, you may need to sample it less often.

Sampling is a general skill that is useful for both complex, yet entirely deterministic systems and for systems with high amounts of pure randomness. Humans observe the vast universe through a tiny straw. Only by repeated and methodical exposure can we build up a more comprehensive image of what exists.

Cost of sampling
Sampling almost always has a cost. Here we see one of the more interesting economic decisions at the heart of random systems: Will the expense of sampling further result in enough improved understanding that I can then leverage in the future for outsize gains?

Averages and Variability
With large enough samples, most random systems become predictable. They tend towards an average with some variability around that average. Thus with enough sampling, the next skill that players learn is getting a feel for the 'typical result' and the likelihood of an 'atypical' result.

Advanced players of Triple Town see luck as a very minor component of the game. As you plan out 30 or 40 moves into the future, you learn that there is a very good chance that you'll get a bush or bear within your window of control. You don't know the order, but there are tools for mitigating out of sequence drops. The learned mental map of average drop rates becomes a tool to be applied skillfully.

Types of distributions
Often the player sees a variety of different types of distribution. The normal curve, multi-modal or exponential distributions are most common. Advanced players get a sense of the distribution. What will outcome is most common? What outcome is least common?

Payouts
All actions in games have payouts. Sometimes they are explicit such as a pawn capturing a rook and removing it from the board. Sometimes they are implicit such as a gift to a player that may in the future be reciprocated.

Through sampling, understanding averages, and understanding distributions, players gain a sense of the value of the payouts. In a sequence of player initiated causes and effects, how useful are the effects?

Expert players weigh these benefits against the costs reaching that average outcome.

New player mistakes due to model ignorance

There are numerous and well documented mistakes that the naive player makes when dealing with systems of randomness. With training, many such players can overcome these. Some will not. Placing an inexperienced driver in the middle of a professional NASCAR race will likely end in physical harm. Even with training, a certain population will never become competitive drivers.

Reliance on non-evidence based models
Players use existing models without considering the evidence. For example, it is common to assume that because 1D6 results in an even distribution of values, 2D6 will also result in an even distribution.

Not enough samples
Players don't sample enough instances of the game to understand the typical outcomes.

Low quality sampling
Players sample, but don't actively look for patterns. Without consciously making observations and testing those observations against future results, critical signals are often ignored. Many players will perform actions, faintly register the results but never ask 'why'.

Poor cost / benefit analysis
During the learning stages of a game, players typically over invest in learning activities, beyond what is strictly necessary to accomplish the desired result. This is seen as 'play' or 'practice' depending on how experimental the routine ends up being.

However, it is common for new players to invest huge amount of resources in activities with very little future pay off. They engage in 'play' behavior (not a consciously forward looking act) and find themselves never recouping. They misjudge when hold them, when to fold them or for that matter, when to walk away.

Balancing for skill in games of luck

Like any game of mastery, we have concepts of balance and progression in games of luck. Typical balancing techniques work

Dominant strategies
Is there an average outcome that is preferable? This is tricky to ascertain since you can still have a balanced random system where a single sampled event yield a rare outcome. When new players see this, they will scream at the top of their lungs that something is overpowered. With a reasonable understanding of combinatorics, you can guarantee that such events are interesting outliers. You can also gather metrics over a large population of games and verify that the 'game breaking outcomes' are in fact rare circumstance.

Is there any benefit to even having these outliers? I think so. They certainly add a strong emotional drama to the game that would otherwise be missing. Also players are kept on their toes and must plan for blackswan events as much as the average events. That's an interesting decision.

In Triple Town, the players that come back from a scenario with 5 ninja bears dominating their game end up being better players because of the experience. If that random outcome hadn't occurred, they would never have been pushed to take their tactical skills to the next level.

Does the game structure allow for multiple samples?
A single hand of poker is deeply imbalanced since it is prone to highly variable random outcomes. However, during a poker night or tournament, players churn through dozens of hands. This allows players to take multiple samples and use their knowledge of the game's random distributions to gain material advantages over weaker players. Thus, the right number of samples results in a more balanced game full of meaningful decisions.

Progression considerations in games of luck

You can use the following learning variables to create a progression system to help teach new players the subtleties of a random system. 

Scaffolding
Can new players learn foundational rules with a small number of samples? If you start players off with a random system that takes dozen or hundreds of sample to understand, they may quite before they accumulate enough experience. Instead, use system at are reasonably easy to figure out. In Triple Town, players get grass the vast majority of the time. This helps them learn how to build up more complex structures since they learn very quickly that there's a good chance that the next object is going to be grass.

Existing schema
Is there a known random system you can mimic in order to tie into existing heuristics? For example, many games use a 6-sided die since that is a model of randomness that many players have been using since childhood.

Use of random systems that reveal structure upon inspection
One of my favorite techniques is to pull random outcomes from a fixed pool. Thus the expert players learn what they are going to get, but not in the order they are going to get it. This is the basis of all card games that disallow reshuffling.

You've got two key variables you can tweak for progression escalation:
  1. When the pool is small, players tend to learn it quickly. By increasing the size of the pool, you require additional mastery.
  2. Randomness without replacement ends up being reasonably predictable when sampled across the size of the fixed pool. So if your sample count is higher than the pool size, players will learn the pool quickly. If the sample count is less than the pool size, they'll learn it slowly (or never)

Black hat techniques

There are also cynical techniques that will result in players never learning the system. There are entire gambling journals dedicated to these methods since the number of human randomness hacks are quite large.

  • Obscuring average results through high variability and high sample requirements.
  • Use of artificial close calls so new players see patterns were there are none. There is a measurable sub-segment of players that process near misses as wins. These games prey on people who are essentially dyscadentic, or the random equivalent of dyslexic.
  • Use of social signals so players approach the game with a costly mindset.
  • Obfuscated odds combined with a high cost of playing.
  • Use of high odds that players don't process well. At a certain point the brain says 'many' and doesn't quite grasp that there is a good chance the universe may expire first.

Conclusion

A well rounded designer does not remove randomness from their games. The world is a random place and learning to deal rationally with randomness is a critical life skill. Instead, they embrace the fact that players can learn to understand and master the game's random systems.

It is your responsibility as the designer of random systems to facilitate masterful play. Put new players through a progression where you teach them the system's average results, outliers and distributions. Give them tools for managing and mitigating randomness. Create expert game modes where players roll the dice enough to manipulate the big picture.

When you use randomness as an opportunity for mastery over noise, I think you'll find that games of luck become highly meaningful games of skill.

take care
Danc.

References

Psychology of near misses
  • http://www.sciencedaily.com/releases/2010/05/100504173817.htm
  • http://www.sciencedaily.com/releases/2012/10/121009111233.htm

Gambling addiction as a learning disability
  • http://www.sciencedaily.com/releases/2008/03/080326190802.htm

11 comments:

  1. Yes, you can certainly become skilled at luck! Although I am surprised you did not mention anything about some kind of negative feedback loop to counteract outliers. Or do you think the nature of the random distribution is more important to tweak? (Such as using "a deck of cards" rather than "dice")

    ReplyDelete
  2. >The world is a random place and learning to deal rationally with randomness is a critical life skill.

    Since when do games have to teach us critical life skills to be good, and who said they have to reflect "the world"? Was Reiner Knizia being a "not well-rounded designer" when he designed the luck-less Through the Desert?

    Randomness is noise. It doesn't matter if you have tools with which to understand odds; that doesn't change the fact that when you roll two dice and it comes up snake-eyes, that was meaningless noise from outside the system. You can understand that you have a 99.9% chance of getting a desired outcome, and still not get that desired outcome, because RANDOM NOISE - that's the only reason.

    The only situation in which it's justified to use randomness (output randomness) as a tool in game design is if you're building a game whose entire core mechanism is judging odds. So, in Blackjack or Poker or whatever, I understand it.

    But if you have a situation where it's a tactics game like X-Com or Summoner Wars but then you throw in random dice rolls, that's just injecting unrelated noise into an otherwise coherent system.

    ReplyDelete
  3. Actually, even a bit of noise fundamentally changes the way you play a game. In a fully predictable game, you *know* there is an optimal decision; every other choice is suboptimal. You only consider your target in game-space and the route that gets you there, and every other solution is irrelevant.

    With noise, you have to regularize your solution so you don't overfit -- you must aim for a general area in game-space where you probably win, considering that noise will cause slight deviations. The neighboring solutions of your target solution will matter a lot. This is much harder to consider.

    On another note, the comparison to poker is very relevant and makes it clear why some games are easier to master than others. To make the rules easier to learn, you need to reset most of the game's state very often, like the multiple rounds in poker. A game that keeps a large bag of state information for a long time will be harder to learn. The large number of small rounds makes such a random game much more tractable for a new player.

    In probability theory terms, the results of poker rounds are fairly IID (independent and identically distributed), which makes bayesian inference easier. In conclusion, to make a game easier you can segment it into smaller events, and carry over as little information between them as possible.

    ReplyDelete
  4. I enjoy randomness in games, to a point. As long as I can have consistently rotten luck and still win by using good strategy, then it adds flavor and depth. Otherwise, I feel it takes something away both from my wins and losses.

    ...Speaking of injecting unrelated noise into an otherwise coherent system, I noticed a number of typos in the article - dropped, added or substituted words.

    what that result might. -> what that result might be.
    the help teach -> to help teach
    There is These is uncertainty, -> These are uncertainty, (or possibly simply starting with "Uncertainty, ")


    Anyway, nice article :)

    ReplyDelete
  5. I will hold that randomness should not determine the outcome of a game. It can add interest and variety to the middle of the play, and when you are progressing close to the expected outcome, it is fine. But even then, you hit an outlier, and despite any skill you have, or how well you have mastered the randomness, through absolutely no fault of your own or skill of your opponent, you fail. In a competitive space, randomly doing well is also problematic, as that means the opponent is losing, again through no fault of their own. This invalidates your gameplay. Every good decision the player has made, every bit of skill they have applied, is rendered pointless.

    ReplyDelete
  6. Well informed, well written, useful. Thanks!

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. Really useful article Dan. It's been pretty helpful for me as i've been thinking about eSports and black box game design.

    Also, I found some typos as well.

    A good hidden object game players is measurably better than a new player. -> A good hidden object game player is measurably better than a new player.

    Probability and statistics provides use with a set of mathematical skills for dealing with randomness. -> Probability and statistics provides us with a set of mathematical skills for dealing with randomness.

    Instead, use system at are reasonably easy to figure out. -> Instead, use systems that are reasonably easy to figure out.

    Thanks for writing :)

    ReplyDelete
  9. I really enjoyed this post. I posted some reflections on it exploring the question of what exactly is a 'skill' and, in agreement with your ideas above, suggesting that noise is actually an integral part of skill rather than something that opposes or degrades skill in a game. thanks!
    http://synaptogaming.com/2013/01/16/is-randomness-antithetical-to-skill-based-games-or-a-crucial-element-of-mastery/

    ReplyDelete
  10. Having just found this web space I must say I'm pleasantly surprised not only by the quality of the article but by that of the comments.

    While I enjoyed the article I didn't share of your analysis on the application and desirability of randomness as a whole.

    In particular, the topics of outliers and your conclusion ("A well rounded designer does not remove randomness from their games. The world is a random place...") were the ones that I found particularly bothersome. Jotaf, Keith Burgun and Mistify have been more than eloquent already but I'll add a thing or two.

    On the topic of outliers, I believe the emphasis should be placed on what these do to the spectrum of effective player actions.

    One of the things that makes random events interesting is that they create a state in which optimum strategies must be selected and implemented dynamically; in other words, they help us move away from a complete separation of planning and implementation towards a state in which one informs the other in a loop during play.

    If an outlier destroys the action space in such a way that a big enough disadvantage/advantage is created, player action might be completely (or overly) devalued. Even in cases in which these can be overcome by experienced players it creates a separation of the difficulty level of play when the outliers are absent and when they appear that might impair a player's ability to produce an accurate model of the situation.

    Yes, drama shall abound, but not all drama is desirable emotional involvement.

    I applaude you taking a stand in your conclusion, but I couldn't disagree more. Randomness can be used to make good games, but arguing that there's causality between the quality of these games and the inclusion of randomness is simply bad logic. As Keith Burgun mentioned, there can definitely be great games in which judging odds is the core skill... in retrospect, however, randomness has been often used to obfuscate the underlying model in order to inflate gameplay or present something as more complex than actually is; in the worst cases it has been a tool to hide the inadequacies of a design (this is, of course, a personal observation.)

    ReplyDelete