abusing anti-abuse systems

[quarrel]

The struggle between designers who make game mechanics and players who abuse them never ends in general and rarely ends well in specific cases. Here’s one that got rather convoluted.

I’ve mentioned A Tale in the Desert before. It’s a small indy MMORPG that offers its players difficult, long-term challenges called Tests. Approximately half a dozen Tests involve designing a puzzle for one of the game’s predefined puzzle frameworks, which include exercises similar to Rush Hour and tangrams. You pass the Test by earning a sufficiently high average rating from other players who’ve solved your puzzle. Naturally, some players pass these Tests by providing their friends and guildmates with the solution (or making their puzzle trivial to begin with) and having those people give their puzzle maximum ratings.

Oddly enough, the developers of this particular game don’t normally act to stop things like this. In fact, they deliberately design game systems where abuse like this is possible, since encountering and solving these conundrums is core to the game’s unique player experience. But the players couldn’t solve this particular problem, and they grew increasingly dissatisfied with the ongoing unfairness of a puzzle rating system that rewarded social connections more than puzzle design skill, so the developers stepped up.

The first thing they did was weight a player’s votes based on how much voting he’d done previously. This avoided the exploit of creating a wave of new characters to stuff ballot boxes, but it did nothing to address the ability of a guild to conspire within itself and scratch each others’ backs. So the devs went further. They attempted to create an objective measure of how honest and credible a puzzle judge was, called his Influence, and used it to further weight his rating votes. Influence was in turn a function of two other metrics: Worldliness and Quality. Your Worldliness correlates with the total number of distinct other players who’ve judged at least one puzzle that you also judged. It’s low if you rarely vote on puzzles (like, say, only when your friends ask you to), and it’s lower than normal if you habitually vote along with a static, organized bloc of colluding players rather than as an independent wanderer. Your Quality measures how close your votes tend to be to a puzzle’s average rating, the idea being that if the majority of other voters find a puzzle to be “Poor” (for example), then it probably really is “Poor” in some absolute sense, and you are a more accurate and honest judge if you rate it “Poor” than if you say it’s “Great”.

The developers then chose to reveal these metrics. The next step in the cycle began immediately. Dedicated unscrupulous players who still desired to boost their friends’ standings turned the system on its ear and used it to acquire Influence scores higher than honest judges had. They did this by building intentionally bad, easy puzzles in major population centers throughout the game world, then voting on those puzzles themselves, giving them the lowest rating. Since the puzzles are in high-traffic areas, they’re played by a broad cross-section of passers-by, and since they are trivially easy, those passers-by are guaranteed to solve them and earn the privilege of voting. Those factors combine to elevate the exploiter’s Worldliness. Also, because the puzzles are bad, they get low average ratings, thus inflating the exploiter’s Quality.

That sent the ball back to the devs’ side of the court, whereupon they complicated things further. They added a new item to the list of possible puzzle ratings: “Fraud”. A puzzle that receives too many Fraud votes is ignored when calculating Worldliness and Quality ratings. The idea was that the latest generation of abusers was easy to spot and this gave players a way to foil their cunning plans.

It’s about this point that the whole back-and-forth system of exploits and tweaks broke down. The purpose of the Fraud vote wasn’t explained in the game, and it certainly wasn’t intuitively obvious. Most players who issued Fraud votes were unfamiliar with the history behind them and instead believed the new rating was intended to identify puzzles that were only built to satisfy a leveling requirement. (All formal Tests had a list of preliminary steps, and that list was worth an immediate character level when completed regardless of whether the full Test was ever passed. Some players felt it was unethical to participate in a Test just to gain this level.)

Regardless, the system is still in place as far as I can tell. The game’s lead designer says, “Despite intense pressure to game it, and a really smart user base, it has stood up remarkably well, and I think it may be as game-proof as any system could be given the constraints.”