Sunday, March 30, 2008

The Translation Game

The Rosetta Stone: I18N's early best friend

Online games have become an international business. In order to compete in the global marketplace, your game needs to appeal to players in countries ranging from America to Japan to China to Poland. None of these cultures speak the same languages, have the same cultural values or even celebrate the same holidays. If you are starting an online game company it is wise to starting contemplating the monumental task of localizing your service as early as possible.

There are two truths about localization for online games. One, it can dramatically multiply the number of customers that you reach. Two, it is more trouble than you might imagine. In this essay I look at how we might use we use techniques from game design to streamline this exciting, yet expensive opportunity.


The good and the bad The majority of players populating online games like Legend of Sherwood, Travian or World of Warcraft are from outside the United States. We have a solid 300 million potential customers, but the rest of the world has billions. After the core game is complete, localization can help you double or quadrupal your install base for a cost far lower than developing the game afresh. This is highly attractive to those folks with dollar signs for eyeballs.

Yet localization and globalization ends up being far more than a onetime cost for translating a few text strings. Popular games pump out an ongoing stream of new content that must be rolled out across multiple languages. The original team likely doesn’t even speak the language of the targeted countries so quality control is a huge issue. Even worse, most teams have little experience with the culture of the new country they are targeting. An expansion pack that celebrates Christmas as a family event may not translate well to your Japanese users who traditionally see Christmas as a holiday for lovers.

Larger companies end up creating comprehensive localization and globalization groups that can act as a giant drag on the software development process. Localization often ends up in its own silo with radically different organization and values than the main development team.

Help (for hire)
Whenever there is a difficult problem that falls outside the core competency of a game development team, outside companies will emerge to help them solve the problem. For a fee, of course.

At the most basic end of the spectrum are translation services. These take your table of text strings and translate them into a variety of languages. Quality varies dramatically and there is typically the need to re-edit the text afterwards. Many companies start by just translating their strings and then realize that bringing their games to other countries is far more complex than a simple data manipulation problem.

At the other end of the spectrum is the full service operator. An operator is a company that takes an existing game, typically from a traditionally inward looking market like Korea or China, and runs a localized version of that game in another country such as Japan or the United States. Depending on the company, the operator will handle everything from localization, to handling foreign currency, to running foreign language support. Many will run the local servers for your game. OutSpark is a good example of an operator.

Rolling your own?
Every time you deal with middlemen, even ones as innocuous as a translation service, you need to ask the question: What is the opportunity cost of rolling my own?
  • On one end of the equation are the operators that will take a percentage of your revenue for the lifetime of the game in return for expanding your market substantially. Woot, “free” money.
  • On the other end of the equation is a custom solution. If you can reduce costs, you might come out ahead. But what if it dilutes your focus as a company? What if you end up missing out on the economies of scale and experience that come from being a company focused solely on localization?
This decision is particularly tricky since middlemen make it their business to provide you with a very clear value proposition when you are examining their services. There is rarely anyone who can put hard figures on the benefits of rolling your own. The easy solution often becomes the outsourced solution promoted by the smiling salesman.

So this got me thinking: What would it take to roll your own localization service for an online game?

Constraints
There are some constraints to this little thought experiment:
  • Inexpensive: The solution needs to cheaper than going with an outsourced group.
  • Leverages game development skills: Ideally we can leverage our core skills as a game development company. This means using game design to solve our problems, not hiring a mass of translators.
  • Doesn’t rely on building up services that other people could do better: We need to be wary about spreading the company too thin by turning into an operator in our own right. Though there may be some benefits from vertical integration, internally replicating systems that are already run efficiently by third parties is something to be approached with great care and a skeptical eye.
Research clues
At GDC, I ran across two clues that point to a solution.
  • Leverage the community: I caught some offhand comments from the fine folks over at Three Rings about alternative ways of leveraging the community in order to avoid entering into partnerships with operators. What they’ll end up doing, I have no idea. Still the seed was planted in my little monkey brain.
  • Wikipedia as a game: Elonka Dunin gave a lovely talk on how Wikipedia can be viewed like a giant MMO. The mechanics happen to focus on user generated articles instead of killing monsters, but the fundamental rewards for writing articles is fundamentally game-like. The pertinent detail here is that Wikipedia users also translate articles on a regular basis. Wikipedia is one of the most comprehensively localized websites in the entire world and all of it is due to the user’s efforts.
User generated translation driven by game mechanics
Let’s build a game that rewards multi-lingual players for localizing the text in our product. We’ve got all the necessary ingredients in your typical game:
  • Passionate players who speak a variety of different languages
  • In game reward systems that have already proven attractive to the player.
  • A mediated environment that allows us to pose tasks and record the results.
Instead of hiring expensive middlemen, we harness the volunteer efforts of our passionate players. Instead of managing the process manually, we create an automated system of empowering tools and reward systems that encourage players to do the right thing. Above all, we make the process repeatable so that we can run it over and over again at almost zero incremental cost. We are building an engine whose mechanical structure is derived from the physics of human psychology and whose brightly burning fuel is a steady stream of fun seeking players.

The reviewer pattern
Here’s a basic structure of our translation game.
  1. Identify: Identify players with base level skills are capable of performing a desired task.
  2. Action: The players perform actions using in-game tools in the hope of receiving a reward.
  3. Review: The game independently verifies the actions using other players for tasks that require human computation.
  4. Reward: The game rewards the person performing the original action based off the verification process.
  5. Repeat: High scoring people get more tasks of that type and greater rewards. Low scoring people get fewer tasks of that type.
This reviewer pattern is quite flexible and can be used for wide variety of tasks that are much broader than mere translation. It is particularly useful when you need to judge the quality of your player’s efforts and “quality” is determined by strong aesthetic or cultural factors. Language is an obvious example of this, but art, fashion, moral judgment and other classic human endeavors fall under this umbrella as well.

Let’s apply the pattern to the process of translating text.

1. Identifying players with the right skills
Our imaginary game has persistent characters with extensive profiles. People can declare their real world skills including which languages they seek. Let’s say for a moment that we want to translate between English and Japanese. We would search the profiles of our thousands of users and find ones where players claimed proficiency in both languages. These users are tagged as potential translators.

2. Performing the task
The next time the player approaches a quest giver, the game substitutes a translation quest for their typical “Kill 5 rats” quest. There are numerous framing stories one could use ranging from the scholar seeking to translate a mysterious scroll to a warlord needing an intercepted spy message translated so they can prevent an attack.

Promise of a reward: The player is told that if they complete the task they’ll get a certain amount of gold. If they complete the task well, they’ll be inducted into the secret Guild of Translators.

Task specific tools: If the player accepts the task they are presented with a screen that shows them the original text and a space where they can translate it into Japanese. They type in a translation, hit submit and get the basic translation rate. They are informed that qualified reviewers will be looking at their translation and we’ll let them know if they make it into the secret Guild.

On the backend, every text string of every object can be pulled from a set of string tables that have slots for all supported languages. You need a system that supports multi-byte characters, various text orientations, IME input and all the rest of the glorious minutia that goes into localizing simple strings.

To generate the quest, we look at popular items that aren’t translated into languages for which we have translators available. A few strings, such as the name or the description are bundled up and given to the player as the source material they need to translate.

3. Review the results
The vast majority of translations will be of poor quality. This is the reality of free labor done as an idle hobby. We want to separate the truly talented reviewers from the masses so that we can ultimately put them on a pedestal as an example for everyone to strive towards. In the process of tagging our translators, we also tagged people who spoke Japanese.

Again, we give these tagged users quests. This time they need to read the text and rate it quality on a 7 point scale. There is also a small text field where they can type in comments. They are also told that they’ll get a bit of money for doing the task, but they’ll get even more money if they do it well.

Unbeknownst to the individual reviewer, we also give the same text to 10 others to review. We collect all the scores, lop off any outliers and calculate the average score. This is the rating for the translated text.

Translations that get ratings above a certain threshold (such a 6 out of 7), are automatically published to the world at large.

4. Reward the translators and reviewers
Translator rewards: Once the final score for the translation is determined, the original translator receives a message that contains a quality reward. If they scored 6’s and 7’s, they get huge rewards and are inducted into the Guild of Translators. They get a special cloak, and are promised future awards if they continue to do such a wonderful job translating. If they score lower, they get lesser rewards and may not be invested into the Guild. They are given a second chance as well as access to the comments that were left by the reviewers.

Reviewer rewards: We also reward reviewers based off the quality of their reviews. Those players scoring closest to the average score get mega points and an increase in their reviewer level. Those players that scored furthest from the average get no points and their reviewer level can even drop. Over time, competent reviewers whose opinions reflect the majority should rise to the top. When we calculate average scores for a particular bit of text, we can use weighted averages that take into account the level of the reviewers involved.

5. Repeat the process
Whenever text is added to the system, the translation system jumps to action. Translation quests are generated, translators translate them and reviewers review the results. Over time the text of the game is slowly but surely translated into other languages, driven by the demand and passion of the users.

The system is built to constantly improve the results. The highly rated translators and reviewers are showered with in game gifts and rewards. They gain levels, get new outfits and are given kudos by NPCs. We feed them stats on how many people use their translations and look for ways to promote their efforts to the larger community. Each of these rewards is automated and repeatable. In return for these social glories, the system gives highly rated translators more translation tasks. We want the best of the best handling most of the translation.
As time passes, many users will try translation and find that it isn’t for them. That’s alright as long as there is a core group of people that latch onto the job and make it into a fundamental part of their online identity. The system is built to support this natural winnowing process in order to build up an elite core. A dozen or so passionate users can translate hundreds, even thousands of pieces of text in relatively short order.

Popular pieces of text with lower scores are resubmitted to elite translators for another pass. As time goes on, the quality of the translations throughout the game continues to improve.

Benefits
This is a complex system, but it comes with some intriguing benefits.
  • Focus on core competency of game design: It is a reasonable amount of work building this localization system, but it leaves your company with a very solid reward system that can be leveraged for other areas of the game. You are investing in game design and player entertainment, both of which are core competencies for any game company.
  • Serve smaller cultures at low cost: There exist numerous cultures in the world that speak languages other than English, French, German, Spanish, Chinese or Japanese. Dozens of nations like Poland or Brazil can contribute hundreds of thousands of users to your game. By allowing players to translate the game into their own language, you can reach these niche cultures at a very low cost.
  • Adapt your text to each culture: Many concepts don’t translate well between cultures. By having the users perform the translation, they will often smooth over rough edges. If a particular joke or phrase doesn’t work, users will reword it to something that fits their culture. Often, users will take liberties and incorporate references to their own mythology or culture into the translations. It is an interesting trade off. You give up a small amount of authorial control in return for a translation that seems as if it was written by native speakers.
  • Highly scalable: Once a community of translators is in place you can release content in one language and watch it quickly and cost effectively be translated into other languages. The cost of localizing an additional piece of content goes to almost zero.
Limitations
This system is not without some pretty serious limitations.
  • Requires a large number of people: To get to those few dozen people who are interested in translating your game, you’ll need a population of thousands, perhaps hundreds of thousands. Only a very small number of people will be interested in playing your translation game.
  • High startup cost: You need to develop and balance this system. That is expensive and will likely require dozens of iterations on the design.
  • Questionable translation quality: There is a very good chance that the majority of your early translations are incorrect, confusing or insulting. You won’t be able to tell since you don’t speak the target language. This means that when the system isn’t balanced just right, your users may have bad experiences. Also, as mentioned above, you are giving up a certain amount of authorial control. Your epic bodice ripper about the Monks of Ra may turn into a joke gag about Swedish meatballs. C’est la vie.
  • The process moves at the pace of the players: If the players don’t find translation quests interesting or there aren’t many multi-lingual people on your service, translation will lag.
Other pieces
As stated earlier translating text strings is only a piece of the puzzle. To solve the whole puzzle, we can repurpose some of the systems that we created for the tranlation game.

Pieces to own: These are items that you should try to maintain control over.
  • Culture specific events: Combining the review system with tools that let users create their own events is one way to go. This moves you down the path of user generated content, which is a giant can of worms, but again has substantial benefits if you can pull it off. This would likely follow the same review pattern we saw with translation, except instead of translating text, you are asked to build something grand.
  • Culture specific support: Griefing, bugs, currency issues are all best dealt with by a support team that speaks the player’s native language. There has already been some solid work done with volunteer judges and moderators and it makes a lot of sense to invest in this further in the future. Giving player judges the ability to review past grievances, render judgment and then have their judgment reviewed by a jury of randomly selected peers is an obvious profession worth creating. It would likely appeal to a substantial minority of the player population since it involves direct power over others. Again, problem tickets are handled as quests and we slowly give people more power based on the quality of their previous efforts.
Pieces to outsource: Some things like payment systems and local servers, you’ll likely have to outsource to middlemen. Yet out of all the middlemen activities offered by operators, I’m most willing to let them take on these two. Why? Because they both becoming more commoditized by the year. Costs of running servers all around the world are going down, while reliability is going up. As cloud computing and virtualization become more dominant, the last place you want to be investing your valuable cash is in replicating the core business of the likes of Amazon, Microsoft, Google, Sun and a half dozen other future cloud computing behemoths.

There’s a rule of thumb here. Let the middlemen handle the commoditized plumbing, but don’t let them near your customer relationships. Your community and your addicting game design is your unique competitive advantage.

Conclusion
Ultimately, building something like a translation game for your service is a business decision. I believe that you should treat the decision to create a utilitarian social game system in a similar fashion to how you would treat the decision to make a capital investment in your company. It resembles a classic business problem: choose the proper mix of the following:
  1. High variable costs with low capital investment: You rely heavily on manual labor to make each widget. You don’t have much in the way of equipment so it costs you an arm and a leg to build each widget.
  2. Lower variable cost with high capital investment: Alternatively, you can invest in capital expenditures like beefy factory equipment. Capital investments cost a lot of money upfront. However, longer term, they can dramatically reduce your variable costs.
Building systems such as our translation game are remarkably similar to capital investments from the ancient age of manufacturing. They are expensive to develop and balance. They have a significant maintenance cost. Yet, they dramatically reduce the costs of servicing another customer in a foreign country. Instead of paying 30 to 60% of a customer’s revenue to a middleman, you instead end up paying a few pennies for bandwidth.

In the case of a game development company, the building of the equipment that reduces your costs also happens to be your core competency. You are good at manipulating players to volunteer their time and energy to complete obscure tasks. You are good at building enjoyable software and task oriented tools that facilitate the creation of these tasks. It is time to put these assets to work.

So what happens when you use game design to improve the way your run your business? The future online game is a complex digital factory filled with powerful social engines that chug and churn throughout the night. The workers funnel in one end, and out the other comes the high quality fruits of their carefully guided labor. This is no dystopia. The workers are volunteers and the factory is a playground. If they become bored, the players leave. But most do not, for the machine knows too well to their subtle human weaknesses. And, you, the builder of the great machines that measure and prod and coddle humanity in endless loop upon loop, sleep soundly as the money rolls on in.

Take care,
Danc.

References



Saturday, March 15, 2008

The Randomly Reinforced Lost Garden Prototyping Awards

What a whirlwind of a month it has been. GDC was crazy fun, the latest beta of Expression Design came out at MIX, and I decided to take a leap and start in a new position at work. In the meantime, some great games were created in the last prototyping challenge.


Let's reward awesome developers

You know what? I think it is time for an award ceremony. I've been meaning to do this for a while, but never got around to it with the previous prototyping challenges. Here are the awards and how they are handed out:
  • Bronze Medal: You built an interesting software toy. If you make an attempt at a design and it is interesting to futz about with, you get the Bronze Medal. Most people never get a Bronze medal due to the simple fact that they prefer to sit around and think rather than make something. Simply by doing (instead of not doing), you join an elite club.
  • Silver Medal: You found the fun. You've iterated on your design and have identified a few key elements that make the game enjoyable. There is at least 5 minutes of interesting play. It likely isn't polished and some of the higher order reward loops are broken, but the core is there. If past challenges are any indication, I'll give out only a handful of Silver Medals per challenge.
  • Gold Medal: You made the fun repeatable. The game that you've built is entertaining enough that I'm willing to play it for 15 to 20 minutes. This is a hard level to reach and it is only populated by the most elite cadre of weekend warriors. An entire production team could be seeded by your efforts. To reach this level, you've made some critical design steps beyond the initial concept and built unique and sustainable gameplay based off dozens of game play iterations.


You need to post a public, playable version in order to be eligible. I'll issue the rewards about one month after the initial challenge is posted. If something comes in after the original deadline has passed, I'll add it retroactively to the award post. If you win a Bronze or Silver, you can still come back later and make an attempt at the Gold. Anyone who gets a Gold medal is an automatic rock star in my book.

What do you get if you win? First off, you get the right to post a snazzy LostGarden medal on your website. Most importantly, you get that warm fuzzy feeling in your tippy-tip toes that stems from a job well done. This is a geeky challenge done for lurve of the game.

The Results
In my mind, the entire point of the Player with Your Peas exercise was to take a complex design with some potentially messy rat holes (user created levels, physics, path finding, oh my!) and see if you could quickly find the fun. Let's see how folks did:

Bronze Medals

All the prototypes were enjoyable for a couple of core reasons. First, building a world out of little blocks seems to be pleasant at a very simple level. Second, for those who implemented pathfinding, there is something inherently interesting about watching little creatures navigate your world on their own.


Silver Medals

Richard Sims' prototype came closest to capturing the fun. He focused on the falling peas portion of the design and managed to add in a solid pass at the combo scoring system. This moved his prototype beyond being merely an intriguing software toy to the point where one could imagine there being a real game. I played with it for more than five minutes.
Some lessons from Richard's prototype that are worth noting:
  • Focus on prototyping the important gameplay, not the tech: What was nice about Richard's prototype is that he skimped on much of the climbing and jumping portions of the design. Those weren't key to the 'fun' of the design so they could be done later. Knowing what to test first out of your usually exhaustive brainstorming session is a great skill to hone. Just because something is hard (like the pea pathfinding) doesn't mean it is the first thing to tackle.
  • Answer at least Three Why's: Richard's game links together several game atoms together in a sequence. Simple actions have meaning within the game. Most of the other prototypes only had isolated game atoms. There was very little reason for doing things. Here's a little description of the Three Why's that illustrate how they are used:
    • Action: First ask"What is your player's core action?" In this case, it is building blocks.
    • Why? Now ask "Why?" The answer is to give something for the peas to bounce against.
    • Why? Ask "Why?" again. The answer is because the peas generate combo points if they hits lots of blocks.
    • Why? Ask "Why should I care?" once more. Because combo points increase your main score and you need to get as big of a score as possible.
    • Good enough! At this point, the player who is clicking to add blocks probably has lost interest in asking why he should be building blocks. There are enough overlapping reasons that the justification circuitry in his brain is satiated. He'll keep building blocks until all the subsequent game atoms are exhausted.

Gold Medals

No one captured the Gold Medal on this challenge! Doh. This award remains wide open until someone submits an amazing prototype with 15 solid minutes of play. Hmm...I wonder who can make a Gold Medal version of Play with Your Peas...

There were several others folks that worked on the game, but I didn't have public links to all the playable versions. If you have some more, send them my way (danc [at] lostgarden [dot] com) and I'll add them to this post. Yes, Harold, I'm looking at you.

Woot! That was fun. Now I feel like Jon Stewart. Except less witty. Or famous.
Danc.