What with talk of killer heat waves, droughts, floods, etc. etc., this blog tends to get pretty serious. When it does, we don’t deal with happy prospects, but with the danger of worldwide catastrophe. But every now and then we need to “lighten up,” so let’s have a little fun.
Recently a reader comment pointed to a website reporting the results of testing dice for fairness. Specifically, it tested the “d20” or 20-sided die. It’s a die often used in tabletop games, especially D&D (Dungeons & Dragons). That site links to yet another site which tests dice (specifically, the d20). They make enough of their data available for us to take a close look.
The first website (by Mark Fickett) looks at many dice, but the one for which I retrieved data was a d20 from Chessex, which happens to be purple. I’ll call it “Chessex Purple.” The other site (by “Awesome Dice”) compares a d20 from Chessex (which I’ll just call “Chessex”) to a d20 from Game Science (which I’ll call “Game Science” — imaginative names, eh?).
The Chessex Purple d20 was rolled 8,300 times, the Chessex and Game Science d20s 10,000 times each. They conclude that none of the dice is “fair” in the sense of having equal probability for all results (from 1 to 20), and they’re right. I did a chi-square test of the counts, and for all three dice the p-value was less than 10-15; they’re not “fair.” But that doesn’t mean they’re lucky or unlucky — if their departure from uniform probabilty favors big numbers (which is what you usually want in D&D), they’re lucky.
The “nominal” probability of any give result would be 0.05 (5%, or 1/20) for a fair die. The estimated probabilities based on their tests look like this:
The error bars are 95% confidence intervals (based on a Bayesian analysis). Clearly, some of the results are more or less likely than they “should be” for a fair die. For example, the Chessex Purple (plotted in purple) has a higher-than-normal chance (6.4% rather than the nominal 5%) of rolling a “17” (a pretty good roll) and lower-than-normal chance (3.8%) of a “13”, the Chessex (plotted in black) a higher-than-normal chance (6.2%) of rolling a “6” and lower-than-normal (3.8%) for a “19”, and the Game Science has a higher-than-normal chance (6.0%) of an “18” and a much-lower-than-normal chance (a mere 2.95%) of a “14”.
I got the impression that they rated the dice lucky or unlucky by their average rolls. For the Chessex Purple that’s 10.41 (+/- 0.13, 95% CI), for the Chessex it’s 10.49 (+/- 0.11, 95% CI), and for Game Science 10.36 (+/- 0.12, 95% CI). The mean for a fair die is 10.5, and both Chessex Purple and Chessex include that value in the error range for their averages. By that measure, they’re at least “fair” in that your expected roll is — as near as we can tell — the same as for a truly “fair” die.
But the Game Science d20 looks unlucky; its average is below that of a fair die. However, the way I reckon things, if I were playing D&D that’s the die I would choose.
In D&D a d20 gets used a lot, and when it does, you almost always need to roll some fixed value or higher. When attacking a character or monster, you have an ability to hit and it has an armor class (its ability not to get hit), and your roll combined with your ability has to meet or exceed a required limit — the “to hit” value — for you to succeed. So, you want to roll the “to hit” or higher. If you need a 16 to hit that orc, you succeed on a 16, 17, 18, 19, or 20 but you miss on anything less. Likewise, for a “saving throw” you have to make your saving throw requirement or higher to succeed, anything less is failure.
There are also complications for extreme rolls. If you roll a “20” it’s a “critical hit” and you just might do extra damage. If you only roll a “1” you screwed up — you might drop your sword. Generally, 20’s are extra good, 1’s are extra bad.
I used the given data to estimate the probabiltiy, for each possibility from 1 to 20, of rolling that or higher. Actually I computed the “excess probability”, i.e. how much more-than-normal chance you get. The chance of a “1” or higher is 100%, so its excess probability is zero … rolling at least 1 is a sure thing, so you won’t do better or worse than a sure thing. But for other numbers, the excess probabilities depend on the die. I estimate them thus:
If you need a “3” or higher to hit, the Chessex is your best bet; your success probability is 2% higher than normal (92% instead of 90%). If you need a “12” or higher to hit, the Game Science d20 is your worst bet, with 2% less probability than normal (43% instead of 45%). So for low to medium “to hit” limits, the Chessex is better than the Game Science.
But those are the “easy” rolls. If all you need is a “3” or higher to hit, it’s probably some scurrying rat that can’t do you much damage even if it bites you, you’re going to win this one eventually. Even if you need a “12” to hit, you probably went into this knowing it needs an “18” to hit you, and you hit harder anyway, sooner or later it’s toast.
But if you need a big number to hit — if you need an “18” or above — that’s when you’re fighting the tough stuff. That’s when your chance to hit is low enough, that the battle could take some time, you could sustain some serious damage, you might even get killed. That’s when being “lucky” really counts.
And, that’s when the Game Science d20 is better. It has a better-than-normal chance of an “18” or above (16% instead of 15%, a full percentage point extra). And if you need a “19” or above, or — gods forbid — you need a “20” just to hit, it has at least the “normal” chance, while the Chessex Purple and Chessex give you a less-than-normal chance. Just when it really counts.
The Chessex die does have one virtue: it’s chance of rolling a “1” (not “1”-or-higher) is lower-than-normal, so you’re less likely to screw up.
Anyway, if I’m doing the D&D thing and have to choose one of those three dice, I’ll take the Game Science.
That’s just for these particular dice, mind you. Maybe other dice from Game Science aren’t so lucky, maybe some dice from Chessex are killer good. I can’t say without data … and unlike the aforementioned bloggers, I don’t have a machine to automatically roll them thousands of times and test them for me.
The upshot is, that if you want to know which one is the “lucky” choice, be sure to consider, carefully, just which results are lucky and which aren’t.
This blog is made possible by readers like you; join others by donating at Peaseblossom’s Closet.