Tag Archives: survey

Starting with D&D

Does starting with D&D, compared to starting with other tabletop roleplaying games, lead to playing a wider variety of roleplaying games?

Ben and I ran a survey during the winter to explore attitudes and behaviors related to gateway RPGs (you can find the questions we asked here, along with some descriptive statistics). All of this data is correlational, so interpret any trends below keeping in mind that many unmeasured third variables could provide better explanations. Spoiler alert: after controlling for time-related variables, starting with D&D (compared to non-D&D games) is associated with playing a narrower variety of roleplaying games.

Also, note: I made a programming error in the survey which caused the play variety question to show up for only approximately half the participants. That kind of sucks, but even so the sample is still large (> 1000). The problem was that I based this survey on a previous survey as a template which had used random assignment. The good news is that the presentation seems to have been random. (In any case, you can find a link to the numeric data at the bottom of this post, if you want to perform your own analyses.)

Next, I am going to step through some analysis. The statistics are there if you want them, but I tried to write everything in English as well.

Figure 1 (click to enlarge)

First, here is a histogram showing the distribution of play variety (figure 1). Response labels were 1 = Strongly Disagree up to 7 = Strongly Agree.

Overall, participants reported having played a wide variety of tabletop RPGs (N = 1370, M = 5.41, SD = 1.64), somewhat unsurprising for players with enough engagement to talk about games online. Based on the figure above, it looks like the distribution is censored, probably due to range restriction in the measurement. That is, the question we asked probably has trouble distinguishing between people at the high end of the distribution. Additionally, there may be some self-serving bias where participants overestimate the variety of games played. Some models (such as Tobit regression) can take censored data into account.

Figure 2 (click to enlarge)

What about the focal comparison of starting with D&D compared to starting with a non-D&D game? There is a marginal statistical difference, but the effect is quite small, which is clear if you look at a visualization superimposing the two distributions (figure 2). Using a t-test, the difference between means is 5.44 for non-D&D and 5.28 for any edition of D&D (t(947.326) = 1.67, p = .095). Tobit regression provides similar results for the effect of starting with D&D (b = -.249, SE = .136, t = -1.83, p = .067, 95% CI [-.516, .018]). The following two figures use kernel density estimation, which is (to oversimplify) similar to a histogram, but smoothed. (“Non-D&D” here includes “other” results, which was n around 500. I skimmed the free text response for those participants and there are only a handful which could be considered a D&D variant.)

What if we increase the granularity, distinguishing between TSR D&D and WotC D&D?

Figure 3 (click to enlarge)

These figures (3 and 4) might make it look like there is a clear split between the influence of TSR D&D compared to WotC D&D, however keep in mind that starting edition is confounded with date entering the hobby. WotC D&D came out more recently, so players that first land on WotC D&D tend to have started more recently (mean year = 2011) compared to players that started with TSR D&D (mean year = 1997). Dramatically, players also started much younger with TSR D&D (mean age = 12.37) compared to WotC D&D (mean age = 18.29). Both of these factors would give players starting with TSR D&D more time to explore other games, on average, so we should probably add both as covariates to control for that association.

Figure 4 (click to enlarge)

Indeed, when you control for the age/time related variables, you get an entirely different pattern (see figure 5), where starting with either generation of D&D (TSR or WotC) is associated with less variety playing other tabletop RPGs compared to starting with non-D&D games. Based on this data, I think this is the tentative conclusion. Whether that is a good or bad thing is a value judgment that I will leave to you. This could mean that people are more likely to be satisfied with D&D. It could also mean that people with more variety-seeking personalities are more likely to come across non-D&D games initially, and it is this trait, rather than initial exposure, which is responsible both for the exposure to the initial game and the decision to continue exploring other games. This result concerns individual behavior, not broader cultural influence; it is possible (and seems likely to me, though I lack supporting data) that broad brand awareness of D&D expands the overall player pool and so has a net positive effect on the absolute number of people exploring other games.

Figure 5 (click to enlarge)

The associations according to a Tobit regression including the covariates: TSR D&D (b = -.560, SE = .181, t = -3.09, p = 0.002, 95% CI [-.916, -.204]), WotC D&D (b = -.337, SE = .151, t = -2.23, p = 0.026, 95% CI [-.634, -.041]), year started playing covariate (b = -.066, SE = .008, t = -8.04, p < 0.001, 95% CI [-.082, -.050]), age started playing covariate (b = -.032, SE = .014, t = -2.37, p = 0.018, 95% CI [-.059, -.006]). (This seems to be a decent pass at an appropriate model, but notably the results are similar using standard OLS multiple regression.)

Here is the raw numeric data. Here is the Stata code I used for the analysis in this post. Please let me know if you have any questions, discover anything else, notice any errors, or have any suggestions.

Gateway Survey Items

Lighting a torch to explore (source)

We closed the gateway RPG survey today. Below is a brief description of the survey items, number of responses, and some questions we can ask based on the data. If you think of other questions that you are curious about, leave a comment below. I have yet to look at the data for the final set of responses beyond some descriptives.

Following are the items in presentation order. The bold text represents the concept, behavior, or whatever, that we were trying to measure. Italic text represents the exact item text participants saw. Parenthesized text describes the variable type (numerical scale with label, free-response text, and so forth).

I have also included descriptive stats for some of the items (N = number of responses for the particular item, M = mean value, SD = standard deviation). Keep in mind that for the seven point scales, 4 = Neutral, 5 = Somewhat Like/Agree, 6 = Like/Agree, and so forth. 2764 responses provided the correct answer to an attention check item near the end; the stats below only include responses that answered the attention check correctly.

So: ConceptItem text (description of variable type); maybe some descriptive stats.

  1. First RPGWhat was the first tabletop roleplaying game you played? (Selective list of options that we brainstormed, along with an “other” option permitting free-response text for anything we missed); N = 2549, top three: Dungeons & Dragons, any edition (n = 1658), Other (n = 508), Pathfinder (n = 120). 65.05% of respondents that answered this item started with an edition of D&D.
  2. Attitude toward first RPGThink back to the first tabletop RPG you played. How much do you like that game now? (1 = Strongly Dislike, 7 = Strongly Like)
  3. If first RPG was D&D: First D&D EditionWhich edition of D&D did you begin with? N = 1827, top three: 3/3.5E (n = 487), 5E (n = 382), B/X (n = 318).
  4. OwnershipDo you own any tabletop RPG materials? (Books, box sets, and so forth.) (Yes, No); N = 2764, Yes = 2688, No = 76 (97.25% Yes)
  5. Still play first RPGDo you still play the first RPG that you started with? (Yes, No); N = 2764, Yes = 1049, No = 1715 (37.95% Yes)
  6. Delay trying another RPGHow long (in years) after playing your first RPG did you try another RPG? (non-negative integer free-response)
  7. Attitude toward D&DHow much do you like Dungeons & Dragons? (For your favorite edition of D&D.) (1 = Strongly Dislike, 7 = Strongly Like); N = 2028, M = 5.27, SD = 1.63
  8. Belief about effect of D&D on the hobby as a whole (item 1)—The popularity of Dungeons & Dragons attracts new tabletop RPG players. (1 = Strongly Disagree, 7 = Strongly Agree); N = 2618, M = 6.25, SD = .94
  9. Belief about whether D&D crowds out other RPGsThe popularity of Dungeons & Dragons makes discovering other (non-D&D) tabletop RPGs HARDER. (1 = Strongly Disagree, 7 = Strongly Agree); N = 1613, M = 4.16, SD = 1.88
  10. Belief about the effect of D&D on the hobby as a whole (item 2)—The popularity of Dungeons & Dragons is good for the tabletop RPG hobby. (1 = Strongly Disagree, 7 = Strongly Agree); N = 2676, M = 5.64, SD = 1.34
  11. Favorite edition of D&DIf you have played D&D, which edition is your favorite? N = 2348, top three: 5E (n = 1121), B/X (n = 440), 3/3.5E (n = 270). 47.74% of respondents that answered this item said that 5E was their favorite edition of D&D.
  12. Preference for designer authority (item 1)—There are a lot of house rules (customization) in RPGs games that I run or play in. (1 = Strongly Disagree, 7 = Strongly Agree)
  13. Belief regarding whether system mattersWhen it comes to tabletop RPGs, a well-designed rules system is an important factor in enjoyable play. (1 = Strongly Disagree, 7 = Strongly Agree); N = 2217, M = 5.86, SD = 1.13
  14. Preference for designer authority (item 2)—I prefer to play RPGs “as written” rather than adjusting, customizing, or hacking the rules. (1 = Strongly Disagree, 7 = Strongly Agree)
  15. Preferences in play styleFor satisfying play, how important are the following aspects to you? (Aspects included: In-game shenanigans, Generating a satisfying story, Acting in character, Hanging out with friends, Challenge Exploration and discovery, Creative problem solving, Character optimization, Improving my character, Meticulous plotting; 1 = Very Unimportant, 7 = Very Important)
  16. Preferences in game materials—How important are the following elements to you in tabletop RPGs? (Elements included: Fictional setting, Hackability, Mechanical innovation, Ease of use, Art, Genre emulation; 1 = Very Unimportant, 7 = Very Important)
  17. Pleasure readingI read tabletop RPG materials for pleasure, apart from intention for direct use in play. (1 = Strongly Disagree, 7 = Strongly Agree); N = 1300, M = 5.81, SD = 1.37
  18. Play usageI have played most of the tabletop RPGs (systems, modules, adventures, etc.) that I own. (1 = Strongly Disagree, 7 = Strongly Agree); N = 1327, N = 4.16, SD = 2.08
  19. RPG play literacy/promiscuityI have played a wide variety of tabletop RPGs. (1 = Strongly Disagree, 7 = Strongly Agree); N = 1370, M = 5.41, SD = 1.64
  20. Attention check
  21. Age startedAt what age, approximately, did you start to play tabletop RPGs? (integer from the set [1, 200]; due to a typo, some respondents were only able to enter starting ages from the set [10, 200]); N = 2754, M = 15.93, SD = 6.79
  22. Year startedIn approximately which year did you start playing tabletop RPGs? (Drop-down menu with year options); N = 2721, M = 2002.04, SD = 12.25, min = 1971, max = 2019
  23. Other feedbackOptional: Is there anything else you would like to share with us? (free response text)

You may notice that we measure several concepts using more than one item. For example, presumably playing with lots of house rules and preference for playing rules as written should both tap into a preference for designer authority (and they correlate, as expected, at r(2437) = −.62, p < .001). Another: “D&D attracts new players” and “popularity of D&D is good for the hobby” (r(2550) = .46, p < .001).

(The number of observations for pleasure reading, play usage, and literacy/promiscuity seem systematically somewhat low: ~1300 compared to ~2700 for most items; I need to look into that.)

One (somewhat) surprising point that jumped out at me from the descriptives is that for people that started with an edition of D&D, 3E was the most common.

Some possible questions:

  • Do player preferences for game aspects cohere into conventional clusters? Are those preferences consistent with what you might expect? (For example, are people starting with 3E more likely to have a preference for character optimization?)
  • Is starting with an edition of D&D (compared to other RPGs), controlling for years played, associated with greater or lesser RPG play literacy overall?
  • Is starting with an edition of D&D (compared to other RPGs) associated with any of the preferences for aspects of play style or elements of game materials?
  • According to consensual belief, does system matter? (At least based on the mean, the answer here is yes: M = 5.86, roughly = Agree).
  • Is starting with an edition of D&D associated with preference for designer authority in rules?

What else would you like to know based on this survey?

Addendum: I just did a tally of referrer URL to see where respondents saw a link to the survey. To roughly summarize, about 30% came from Reddit, 30% came from Twitter, and 30% came from YouTube, with the remaining referrer URLs a smattering of other sites. In detail, N = 1167; the top five: YouTube (n = 363), Reddit (n = 358), Twitter (n = 311), survey.ascolais.com (n = 59), Facebook (n = 50). Referrer URL data only exists for some responses (1167 out of 2764), but preliminary checks indicate that referrer URL is missing at random (meaning it is probably reasonable to think of it as a random sample and representative of the respondent population at large).

Buying, Blogging, Creating

(This is part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

Particular tabletop roleplaying games are, in addition to being bundles of rules and subcultural traditions, commercial products and brands. Considering games as brands bridges the three broad aspects: commerce, rules, and social behavior. In this post, I will examine which group of players, by game, is most likely to have bought an OSR product, most likely to participate in long form discussion online (blogging), and most likely to publish their own game products for sale. The high level takeaways are:

  • Those most likely to buy OSR products, Lamentations of the Flame Princess players, were no more likely to blog or create OSR products themselves.
  • Swords & Wizardry players seem most likely to be both engaged in all three aspects, being the only group of players having positive associations with both buying and blogging. (I think it is reasonable to interpret blogging as both rules and social engagement.)
  • Just based on examination, there seems to be some clustering of B/X D&D, Swords & Wizardry, Labyrinth Lord, and B/X Essentials, probably due to the high similarity of these rulesets.

The wordings of the relevant survey questions, along with aggregate descriptive statistics, were:

  • Have you ever bought an OSR product? (Yes/No)
    • Total n = 1825, Yes = 1657 (91%), No = 168 (9%)
  • Do you have a blog where you post about tabletop roleplaying games? (Yes/No)
    • Total n = 1772, Yes = 503 (28%), No = 1269 (72%)
  • Have you created a tabletop roleplaying game product for purchase that you would consider OSR? (Yes/No)
    • Total n = 1793, Yes = 253 (14%), No = 1540 (86%)
Click to expand
Click to expand

These three questions represent, in some ways, a hierarchy of effort, or engagement with, tabletop roleplaying as a hobby. It is easier to buy than to blog and easier to blog than to create more finished products.

To examine the association between playing various games and the three outcome variables, I ran pairwise tetrachoric correlations. The following table shows associations between playing particular games and having bought an OSR product. Read the “rho” column as strength of association, ranging from -1 (for perfect negative correlation) to 1 (for perfect correlation). (Rho is the tetrachoric equivalent to a Pearson product-moment correlation.) You may notice, looking at the two mosaic plots, that substantively the difference seems small. This is likely partly due to artificially constrained variance—recall that 91% of respondents reported having bought an OSR product. The buying measure was yes/no, so was unable to distinguish between people who buy a little and people who buy a lot, so I suggest interpreting the following statistics mostly as directional trends. The same goes for the other two measures, though the pattern is starker for blogging, where the measure was able to capture more variation.

Who is most likely to buy OSR products?
Players of… N rho SE p
Lamentations of the Flame Princess 1825 .44 .08 < 0.001
Swords & Wizardry 1825 .44 .08 < 0.001
Labyrinth Lord 1825 .43 .08 < 0.001
B/X Essentials 1825 .41 .08 < 0.001
Runequest 1825 .32 .09 0.012
Home-brew (OSR) 1825 .31 .05 < 0.001
Dungeon World 1825 .16 .07 0.026
Vampire: The Masquerade 1825 -.16 .07 0.021

Playing Vampire was the only significant negative association. Playing other games had no association with having bought an OSR product. The upshot of all this is that Lamentations of the Flame Princess players were most likely to have bought OSR products. Curiously, Dungeon World surfaces here again, perhaps because OSR products can easily inform Dungeon World games in terms of themes and content even if other aspects of the games differ. The n here is constrained by people that answered the buy question.

Who is most likely to blog?
Players of… N rho SE p
Home-brew (OSR) 1772 .25 .04 < 0.001
Home-brew (other) 1772 .21 .04 < 0.001
Swords & Wizardry 1772 .11 .05 0.027
Dungeon World 1772 -.14 .05 0.005
Shadowrun 1772 -.17 .06 0.008
Who is most likely to create products?
Players of… N rho SE p
Home-brew (OSR) 1793 .34 .04 < 0.001
Home-brew (other) 1793 .29 .05 < 0.001
Swords & Wizardry 1793 .29 .05 < 0.001
Labyrinth Lord 1793 .23 .05 < 0.001
B/X D&D 1793 .23 .05 < 0.001
B/X Essentials 1793 .23 .06 < 0.001
D&D (1E) 1793 .23 .06 < 0.001
Pathfinder 1793 -.14 .06 0.038
Apocalypse World 1793 -.19 .07 0.013
Click to expand

Note that the blogging question was general rather than OSR-specific. You can find the full list of games that I asked about in the OSR Games post, along with the number or respondents that reported playing each game. There was also a free-response other option to capture any games I missed; there were no standouts in that item.

Click to expand

The number of pairwise hypothesis tests I ran for this post may seem high—21 games × 3 outcomes = 63—but the chance that most of these results came from chance alone is vanishingly small. One would expect approximately 3 spurious correlations at the .05 level, assuming no relationships exist in reality (3 ~= 5% of 63). As you can see, there were many more associations than 3 in this data, and with significance values that would be highly unlikely to see from chance alone, despite the relatively insensitive binary measures.


OSR survey report

Table of contents

  1. Purpose & Participation
  2. What About the R?
  3. Meaning Respondents Associate with OSR
  4. OSR games
  5. OSR Attributes
  6. OSR Play Behavior
  7. Buying, Blogging, Creating

OSR play behavior

(This is part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

Earlier, I reported the OSR ratings of various games (as in, level of agreement with the statement: I think the game ________ is OSR) and also which games respondents reported playing. As I noted in that post, these variables allowed me to score individual responses on degree of OSR play behavior. Respondents themselves, in aggregate, determined what OSR play behavior entails. In addition to being related to several other variables that you might expect (such as self-declared participation in the OSR, degree of identification with the OSR, and having bought an OSR product), degree of OSR play behavior also positively predicts blogging about tabletop roleplaying games generally and belief that the OSR welcomes diverse voices.

I came up with the idea of scoring respondents on OSR play behavior after designing the survey, so the set of games rated on OSR-ness differs slightly from the set of games respondents could indicate they were currently playing. However, there is a substantial overlap between the two sets. This set of overlapping games contributes to the OSR play behavior score. To create an OSR play score for each respondent, I calculated an average of play scores (playing = 1, not playing = missing value) weighted by game OSR rating, which ranges from 1 to 7. For example, a respondent that indicated playing Vampire: The Masquerade (OSR rating = 1.90) and B/X D&D (OSR rating = 6.14) would have an overall OSR play score of (1.90 + 6.14) / 2 = 4.02. One of the options for currently played games was Home-brew (OSR), which I decided should have an OSR game rating of 7. After all, what game could be more subjectively OSR than one the respondent categorized themselves as Home-brew (OSR)? I determined this scoring scheme prior to looking at any results.

1634 respondents reported currently playing some games for which I had OSR ratings, allowing me to score those responses on OSR play behavior. The mean rating of OSR play among these respondents was 4.41 (SD = 1.44, min = 1.75, max = 7). The distribution of this score was roughly uniform, though there were spikes around 2.87 (n = 198), which corresponds to respondents that reported only currently playing D&D 5E, around 4.93 (n = 64), which corresponds to respondents that reported playing D&D 5E + Home-brew (OSR), and around 7 (n = 96), which corresponds to respondents that reported only currently playing a Home-brew (OSR) game.

Each response gets an “OSR play behavior” score

This composite play score is obviously imperfect, as respondents indicated play dichotomously rather than by degree, the set of games available to select was incomplete, and the measurement of game OSR-ness was itself imprecise (though derived empirically from the beliefs of respondents). However, the relevant question is whether the rating captures some meaningful variation in degree of OSR play behavior rather than whether it perfectly describes each respondent’s play behavior.

Within sample, the OSR play behavior score has some predictive validity. For example, OSR participants have higher OSR play behavior scores (t(1630) = -18.13, p < .001, MOSR = 4.68, Mnon-OSR = 3.26, using a two-sample t test with unequal variances). Looking at the same relationship another way, I regressed the OSR play behavior score on effects-coded self-declared OSR participation (omnibus F(1, 1628) = 283.61, R2 = .15, b = 1.42, SE = .08, t = 16.84, p < .001, 95% CI [1.25, 1.58]). In English, people who said they participate in the OSR (yes/no) do actually report higher OSR play behavior (the R2 = .15 value means that self-declared participation explains about 15% of the variance in OSR play behavior). Additionally, people with higher OSR play scores were more likely to report having bought an OSR product, according to a logistic regression (b = .51, SE = .07, z = 7.21, p < .001, 95% CI [.37, .65]). This is somewhat exploratory, but strongly suggests the score is capturing some meaningful variation in reported behavior.

Now for some of the more intriguing relationships, such as with blogging (Do you have a blog where you post about tabletop roleplaying games? Yes/No). OSR play behavior positively predicts blogging about any tabletop roleplaying games (b = .32, SE = .06, z = 5.55, p < .001, 95% CI [.21, .43], using a logistic regression). General play behavior, however, measured as the number of different games played, has no relationship to blogging (b = .09, SE = .06, z = 1.55, p = .121). The logistic regression equivalent of R2 is pretty low for this relationship, around 2%, so the importance of the relationship may be small, but the point is comparative: something about OSR play inclines people toward sharing long-form thoughts about tabletop roleplaying compared to just playing a wide variety of games.

Finally, respondents with higher OSR play behavior scores reported the OSR to be more welcoming of diverse voices (omnibus F(1, 1628) = 129.69, R2 = .07, b = .46, SE = .04, t = 11.39, p < .001, 95% CI [.38, .54]). The text of this particular item was: The OSR welcomes diverse voices (1 = Strongly disagree, 7 = Strongly agree; M = 4.79, SD = 1.70, n = 1823). There are a few potential interpretations of this result. Optimistically, people walking the talk (that is, respondents that actually play consensually-defined OSR games) have a more positive view of the degree to which the OSR is welcoming. Pessimistically, there may be some selection bias, as people who feel the OSR is unwelcoming may have been less likely to participate in the survey at all. Anecdotally, I did observe this reaction from a handful of individuals. As a crude robustness check, I arbitrarily changed 50 of the “welcoming” responses (around 2% of the sample) to be 1 (Strongly disagree), simulating absent negative responses, and ran the same analysis again. The result held, suggesting that this sort of selection bias would have minimal effect on aggregate beliefs.


OSR survey report

Table of contents

  1. Purpose & Participation
  2. What About the R?
  3. Meaning Respondents Associate with OSR
  4. OSR games
  5. OSR Attributes
  6. OSR Play Behavior
  7. Buying, Blogging, Creating

OSR attributes

(This is part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

Along with beliefs about particular games, I also asked respondents about their attitudes toward several game elements and common approaches to rules. I chose things for rating that I thought might have a strong positive or negative association with OSR, such as random encounters (positive), 3d6 in order for stats (positive), and balanced encounters (negative). Some of these things are central enough that people use them to inspire the names of blogs. For example: save.vs.totalpartykill.ca, tenfootpole.org, and Ten Foot Polemic. Middenmurk writes: 3d6-in-order transports you into an entirely different world. Every single one of these attitudes differs significantly between self-declared OSR participants and self-declared OSR non-participants.

The predicted differences by group were more than a scale point for half of the items: 3d6 in order for stats (b = 1.48, 95% CI [1.29, 1.67]), Save or die (b = 1.31, 95% CI [1.11, 1.50]), Race as class (b = 1.26), Random encounters (b = 1.09, 95% CI [.95, 1.23]), Level drain (b = 1.03, 95% CI [.83, 1.22]), and Balanced encounters (b = -1.28, 95% CI [-1.45, -1.11]). In plain English, for example, this means that people who think of themselves as OSR participants on average Somewhat like 3d6 in order for stats, while non-participants are on the dislike side of Neutral. The average attitude across all attitude objects was higher for self-declared OSR participants than for self-declared OSR non-participants, but this is unsurprising given that I chose more attitude objects relevant to my associations with OSR play. I am unsure how to make sense of the result that OSR participants like Ascending AC more than non-participants. Maybe the non-participants have negative attitudes toward anything associated with Dungeons & Dragons of any stripe. I am open to alternative explanations though.

Somewhat surprisingly (at least to me), everyone, even OSR non-participants, seemed to be positive about Random encounters and Reaction rolls, which are mechanisms that weave juxtaposed outcomes into surprising sequence of fictional game events. This leads me to believe that positive attitude toward play designed to produce emergent (as opposed to planned) narrative is a broadly shared preference. It seems that Railroading, Alignment languages, and Dice fudging are universally disliked, with Railroading being solidly in full Dislike territory across all responses. Perhaps the title I gave to the second figure is an overstatement, as slightly on the Like side of Neutral for Balanced encounters is hardly a ringing endorsement, but relatively speaking self-declared OSR non-participants like Balanced encounters a whole lot more than self-declared participants.

Gygaxian fundamentalism

I included one quirky item which I thought might represent a form of early D&D orthodoxy: attitude toward Alignment languages. The general reaction I have seen to the idea of alignment languages is either confusion or bemused contempt, with the occasional attempt to use the idea as a prompt for some creative (but almost always ironically satirical) world building. I rarely (perhaps never) have seen alignment languages actually come up in play any time recently. I am sure there are exceptions, but the point is that it seemed like a good candidate for looking at the attitude respondents might have toward something cumbersome but authentically Gygaxian. However, even self-declared OSR participants on average dislike Alignment languages. I interpret this reported attitude as some evidence that Gygaxian fundamentalism, or the regard for something solely based on the fact that Gygax did it, is less central to the meaning of OSR than many other elements in the constellation of meaning making up OSR. Similarly, while respondents generally agreed that AD&D was OSR (and to a greater degree among self-declared OSR participants: MOSR = 5.85, Mnon-OSR = 5.08), AD&D was rated less OSR than the games closer to Basic D&D in the set of games I presented.

Based on these comparisons, it is clear that a set of distinct attitudinal preferences toward game elements at least distinguishes OSR participants from non-participants, and perhaps underlies the division. However, the attitudes differ in ways that are somewhat counter-stereotypical. It seems that few respondents like the idea of experiencing a planned narrative, contrary to some assumptions I have seen in OSR circles, and old rules pedigree (or that related concept, nostalgia) is insufficient alone to generate positive regard among OSR participants, contrary to assertions I have seen outside OSR circles.


OSR survey report

Table of contents

  1. Purpose & Participation
  2. What About the R?
  3. Meaning Respondents Associate with OSR
  4. OSR games
  5. OSR Attributes
  6. OSR Play Behavior
  7. Buying, Blogging, Creating

OSR games

(This is part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

Beliefs about the nature of OSR, according to previous exploration of the 2018 OSR survey responses, seem to cohere around three broad themes or aspects: OSR as rules tradition, OSR as commerce, and OSR as social scene. Of the three aspects, respondents overall agreed most strongly that OSR is a rules tradition. There was greater ambivalence around the other two aspects, with commerce rating lowest on average. This naturally leads to the question of what kinds of rules and which games seem most OSR. In this post, I will discuss which games survey respondents believed were most (and least) OSR.

According to respondents, the most OSR games, out of the games I asked about, are Labyrinth Lord, Swords & Wizardry, Lamentations of the Flame Princess, and B/X D&D, all scoring above 6 (Agree) on average, across all participants, with LL narrowly taking the top slot. The least OSR games are Fate, Pathfinder, and Vampire: The Masquerade, all scoring below 2 (Disagree) on average, across all participants. I think this set entails reasonable coverage, though there are a few unfortunate omissions in retrospect, such as the lack of OSRIC. Respondents seemed neutral about whether the classic non-D&D games counted as OSR, with Classic Traveller, Runequest, and Call of Cthulhu all hovering around the scale midpoint. I was moderately surprised to see how D&D 5E rated (less OSR than Dungeon World!), given the high regard many people in my circles seem to have for this edition. This is speculation, but I suppose this means that the dominant associations respondents have with D&D 5E remain mainstream trends such as adventure paths, character builds, and tactical set-piece combats. The games respondents rated as least OSR seem to have associations with narrative focus (Apocalypse World, Vampire, Fate) and D&D 3E (Pathfinder).

Near the end of the survey, I asked respondents which games they were currently playing. Respondents could select all that applied from the following list: B/X D&D, D&D 5E, Pathfinder, Home-brew (OSR), Home-brew (other), Vampire, Lamentations of the Flame Princess, Swords & Wizardry, Apocalypse World, Dungeon World, Other PbtA game, Other World of Darkness game, Labyrinth Lord, B/X Essentials, AD&D (1E), GURPS, Shadowrun, Call of Cthulhu, Runequest, Fate, and Savage Worlds. Combining these reports of play behavior with the aggregate OSR ratings of games described above allowed me to compute an OSR play behavior score for each respondent. More on this in a future post.

On average, respondents reported currently playing 3.26 different games (SD = 2.51). 34 respondents (1.86% of N = 1828) reported playing more than 10 different games currently, including one respondent that checked all 18 game options, which seems implausible, but I am going to chalk that up to careless responding/measurement error and expect sample size to wash out that messiness. Without those 34, the average number of different games played drops to 3.08 games (SD = 2.16), but I keep them for all following results, as I predetermined all exclusion criteria.

Keep in mind respondents could select more than one game. Apologies for the small fonts.

Unfortunately, the survey lacked an option to indicate playing DCC, which I suspect would have a substantial number of current players. That was an oversight on my part. I did include an open ended item for other games played, and 142 respondents indicated playing DCC there. Additionally, the two World of Darkness related items were Vampire (or other WoD) and Other World of Darkness game, which overlap. (Clearly I should have proofread this set of options more carefully.) The headline results here are the popularity of D&D 5E (which indicates the power of the market leader) and the popularity of games genealogically related to B/X D&D. In fact, in terms of the games people are actually playing, based on these results one could view OSR as a vehicle for propagating the B/X vision of D&D.

For those unaware, the B/X D&D rules tradition started with the Moldvay Basic rules (covering character levels 1 through 3) and the Cook/Marsh Expert rules (covering character levels 4-14). It continued with the Mentzer-initiated BECMI line (covering character levels 1-36 and collected in the 1991 Rules Cyclopedia), which TSR supported in parallel with the Advanced Dungeons & Dragons line. The B/X line lineage is notable for simple rules and treating character class as a more general abstraction, rather than something closer to profession. For example, “elf” is a class in B/X. Arguably, the modern “playbook” trend derives from the B/X notion of class.

As always, let me know in the comments if anything is unclear, if you have any questions, or if you have any suggestions.


OSR survey report

Table of contents

  1. Purpose & Participation
  2. What About the R?
  3. Meaning Respondents Associate with OSR
  4. OSR games
  5. OSR Attributes
  6. OSR Play Behavior
  7. Buying, Blogging, Creating

OSR survey: meaning

(This is part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

Over the past few years, I have been continually surprised by the degree of disagreement, confusion, and, occasionally, vitriol regarding the meaning of OSR. I think it is uncontroversial to state that the meaning is contested. It is unclear, however, whether there is any consensus and, more generally, what is the structure of underlying beliefs. Summarizing, the results of the survey support at least three main aspects of OSR meaning. Agreement about these three aspects varies by respondent age, with younger respondents seeing greater conflict between the aspects compared to older respondents. There are other variables to consider, but I will focus on age and self-declared participation in this post.

Age and participation might influence beliefs about OSR for many reasons. Considering rules, older participants might have played older games when they were originally released and so have a different frame of reference compared to younger players. Self-declared participants in the OSR may have different priorities, or more information, compared to self-declared non-participants. We asked respondents directly about participation. (Do you participate in the OSR? Yes/No.) We also asked five questions about identification with the OSR using a seven-point agreement scale (for example, I feel a bond with the OSR). Respondents who participate in the OSR also identify more with the OSR (by approximately 1.75 points of agreement, or the difference between Somewhat agree and Strongly agree).

Before I describe the associations in the survey between meaning beliefs and age or participation, it is worth first looking at the beliefs in aggregate, over all responses. Following are the means of the OSR is… questions, which I think of as the basic “meaning” questions. We also asked each of these questions in terms of respondent desires, so we will be able to look at the discrepancy between what people believe OSR is and what they want it to be.

OSR meaning across all responses

These averages tell a simple story, which is that respondents agree most that OSR refers to a play style and set of design principles. Respondents agree somewhat that OSR is a scene or movement and that OSR is a collection of aesthetics. Respondents are close to neutral, on average, about the commercial aspect of OSR. Looking at the text of each question, and the averages, three pairs seem to hang together conceptually, tapping into game design, social, and commercial aspects, with aesthetics and specific rules off in their own space. Formal statistical analysis1 leads to the same conclusion, so I averaged each of those three pairs into new scores for the game design, social, and commercial aspects. In the following figures, I color code aspects where possible, using black for rules, green for commerce, and blue for social interaction. I created similar scores for aspects as desired (that is, what respondents say they would like OSR to mean).

The following figure shows a box plot of the three aspects. Quartiles determine the look of box plots. The central line in each box shows the median. The diamonds represent outside values, which are values more than 1.5 quartiles away from the median.

OSR meaning and desired meaning based on quartile

One way to interpret the the greater number of outside values for the rules aspect compared to the other aspects is that there is more consensus around the rules aspect. The quartiles are tighter. Looking at a similar figure that compares OSR participants to non-participants yields similar conclusions, though participants seem to both perceive and want a greater social component. (The right pane below is OSR participants.) It is perhaps unsurprising that the discrepancy between perception and desire is greater for non-participants than participants on average, as that mismatch may be part why non-participants avoid participating to begin with.

Box plots comparing meaning for self-declared OSR participants with non-participants

Age provides a potential explanation, at least partially, for broader misunderstanding and disagreement, shown in the following figure. Younger respondents seem to have stronger beliefs about OSR meaning compared to older respondents (as in, positions more strongly held). Further, there is a stark divide between commerce and social/game design aspects for younger players. Older players seem to agree, relatively speaking, that OSR is more about rules than social interaction or commerce, but are basically neutral, on average, regarding OSR as commerce or social interaction. (The pattern looks similar if I plot a figure using years playing tabletop roleplaying games as the independent variable, rather than age.) I suggest only interpreting the linear predictions (the lines) for age < 60, as the number of responses decreases markedly for age > 60.

OSR meaning aspect trends over respondent age

This pattern is similar for non-participants compared to participants, though older non-participants have much stronger aversion to the social component of the OSR (see figure below). Though the left pane may look sparse compared to the right pane, 372 responses is substantial.

OSR meaning aspect trends over respondent age, separating non-participants (left) and participants (right)



1. A three-factor model fits the data best, with the three item pairs loading heavily on factors consistent with the three aspects explained above. For stats nerds, I used exploratory factor analysis with maximum likelihood estimation and oblique promax rotation. I chose oblique rotation because the latent variables are correlated above a conventional threshold, making orthogonal rotation less appropriate. I chose to score latent variables using averages rather than factor scores so that interpretation of the results in terms of the seven-point response scale is possible.


OSR survey report

Table of contents

  1. Purpose & Participation
  2. What About the R?
  3. Meaning Respondents Associate with OSR
  4. OSR games
  5. OSR Attributes
  6. OSR Play Behavior
  7. Buying, Blogging, Creating

OSR survey participation

(This is part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

This is the first post of several where I will summarize the results of the OSR survey Ben and I ran. The survey was open from December 3rd through December 8th (2018). Given the high level of participation, and a handful of common critiques, after I have a chance to post some results I plan to ask for general feedback about how we ran the survey and what people might like to see in the future. For those who want a short summary, the TLDR of this post is that there were 2000+ responses (that is a lot), mostly from North America/Europe, and some basic demographics (such as age and country of residence) are similar for OSR participants and non-participants (based on self-categorization).

Purpose

A few people asked why we were running the survey at all. The superficial (but true) answer is that I saw Ben post a question on Google Plus about what questions people might like to see on a hypothetical OSR survey. I was curious too, and I have some experience running surveys so I reached out, and here we are. The slightly more involved (and also true) answer is that I have been, over the last several years, continually surprised by seeming confusion, misunderstanding, and disagreement surrounding the meanings that people attach to the term OSR. Personally, I am primarily curious about the degree to which there is consensus about this meaning, whether the meaning differs substantially between people that consider themselves to be participating in (the?) OSR, and whether the meaning people currently see matches what they would like. Secondarily, I am also curious about what content (at the game level) people consider OSR, the geographical distribution of people that participate in (the?) OSR, and favorite OSR products. Other questions may occur to me as I proceed (and feel free to suggest questions if you have any).

Responses

2018 respondents completed the survey (that serendipitous number is a coincidence). Based on some preliminary checks, I see no sign of brigading, bots, or similar threats to data quality. 134 responses failed the attention check near the end. 46 responses provided age greater than 100 and one response gave age less than 10. 9 respondents specified that I should discard their response entirely. None of these responses will contribute to my summaries. After exclusion, there were 1,828 responses. To the question Do you participate in the OSR?, 1450 respondents said Yes, 372 respondents said No, and 6 skipped the question.

Generalizability

Several people seemed suspicious about why we asked any demographic questions. The point of such questions is have some basis from which to generalize the results. For example, if respondents skew older, it would be inappropriate to assume results represent the beliefs of younger players. And so forth. Additionally, many people indicated, both to me directly and in the final “share anything” question, that they found the two items about political orientation insufficient to accurately communicate beliefs. Due to this feedback, and to ease concerns that there is some prior political agenda motivating the survey, we decided to avoid looking at those measures. Most obviously, these results speak more to the beliefs about people who talk about games online. Only people who frequent RPG forums or follow people who talk about RPGs on social media are likely to have heard about the survey at all, so that is a property of the sampling process. Put another way, we oversampled (heavily, perhaps close to exclusively) people who spend large amounts of time on hobbies online. That is, however, appropriate to OSR, which originated and developed online.

Figures

The first two figures show scatter plots of responses over time. The first plot shows how long participants took to answer the survey, in minutes. For ease of visual interpretation, I avoided plotting a handful of observations where respondents took an extra-long amount of time to submit the survey, though these responses still contribute to other results. The second plot shows respondent age, which drifts very slightly up according to the trend over time, though it is difficult to see. Plots such as these often show potential confounds or technical errors. Nothing like that jumps out at me.

Response Duration Over Time (the X axis represents about one week)

Respondent Age Over Time (the X axis represents about one week)

Responses by day (survey dashboard)

The mean response age was 36.81 (SD = 9.88, n = 1828) and the stats are similar for self-declared OSR participants (Mage = 36.58, SD = 9.87, n = 1450) and self-declared OSR non-participants (Mage = 37.64, SD = 9.90, n = 372).

Respondent Age Distribution

There was a mix of people that have been playing tabletop roleplaying games for a while and people that started more recently. My guess is that the frequencies by year shown below approximately reflect general industry fortunes.

Year Started Playing Tabletop Roleplaying Games

I categorized responses by rough geographic area based on country of residence. Yes, I know that “Asia” is an oversimplification, but look at the number of responses before you object. Participants that skipped the country item, or responses that were ambiguous to me, show up in the “Uncertain” category (as in, I am uncertain). The majority of responses are, unsurprisingly, from North America (65%), and 89% are from North America plus Europe combined (being relatively expansive about the definition of Europe). So when you are thinking about what kinds of conclusions you can draw from these results, keep that in mind. This survey should give a decently representative picture of people from North America and Europe who talk about tabletop roleplaying games online, but speak less to players living in other cultures.

Geography or responses (overall)

Geography of responses (self-declared OSR participants)

Geography of respondents (self-declared OSR non-participants)

The social media context in which people reside may be even more important, for purposes of measuring beliefs about OSR, than physical geography. Respondents indicated where they heard about the survey, and that seems like a reasonable, if imperfect, proxy for social media environment. The immediate takeaway here is that Reddit is huge. The secondary takeaway is that the survey represents a much wider swath of RPG players than those that I hang around, as my crowd is mostly on Google Plus and blogs at the moment. We also asked people about how often they used various platforms for talking about games, but I think that is enough for now.

Social media context


OSR survey report

Table of contents

  1. Purpose & Participation
  2. What About the R?
  3. Meaning Respondents Associate with OSR
  4. OSR games
  5. OSR Attributes
  6. OSR Play Behavior
  7. Buying, Blogging, Creating