Tag Archives: survey

OSR attributes

(This is one part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

Along with beliefs about particular games, I also asked respondents about their attitudes toward several game elements and common approaches to rules. I chose things for rating that I thought might have a strong positive or negative association with OSR, such as random encounters (positive), 3d6 in order for stats (positive), and balanced encounters (negative). Some of these things are central enough that people use them to inspire the names of blogs. For example: save.vs.totalpartykill.ca, tenfootpole.org, and Ten Foot Polemic. Middenmurk writes: 3d6-in-order transports you into an entirely different world. Every single one of these attitudes differs significantly between self-declared OSR participants and self-declared OSR non-participants.

The predicted differences by group were more than a scale point for half of the items: 3d6 in order for stats (b = 1.48, 95% CI [1.29, 1.67]), Save or die (b = 1.31, 95% CI [1.11, 1.50]), Race as class (b = 1.26), Random encounters (b = 1.09, 95% CI [.95, 1.23]), Level drain (b = 1.03, 95% CI [.83, 1.22]), and Balanced encounters (b = -1.28, 95% CI [-1.45, -1.11]). In plain English, for example, this means that people who think of themselves as OSR participants on average Somewhat like 3d6 in order for stats, while non-participants are on the dislike side of Neutral. The average attitude across all attitude objects was higher for self-declared OSR participants than for self-declared OSR non-participants, but this is unsurprising given that I chose more attitude objects relevant to my associations with OSR play. I am unsure how to make sense of the result that OSR participants like Ascending AC more than non-participants. Maybe the non-participants have negative attitudes toward anything associated with Dungeons & Dragons of any stripe. I am open to alternative explanations though.

Somewhat surprisingly (at least to me), everyone, even OSR non-participants, seemed to be positive about Random encounters and Reaction rolls, which are mechanisms that weave juxtaposed outcomes into surprising sequence of fictional game events. This leads me to believe that positive attitude toward play designed to produce emergent (as opposed to planned) narrative is a broadly shared preference. It seems that Railroading, Alignment languages, and Dice fudging are universally disliked, with Railroading being solidly in full Dislike territory across all responses. Perhaps the title I gave to the second figure is an overstatement, as slightly on the Like side of Neutral for Balanced encounters is hardly a ringing endorsement, but relatively speaking self-declared OSR non-participants like Balanced encounters a whole lot more than self-declared participants.

Gygaxian fundamentalism

I included one quirky item which I thought might represent a form of early D&D orthodoxy: attitude toward Alignment languages. The general reaction I have seen to the idea of alignment languages is either confusion or bemused contempt, with the occasional attempt to use the idea as a prompt for some creative (but almost always ironically satirical) world building. I rarely (perhaps never) have seen alignment languages actually come up in play any time recently. I am sure there are exceptions, but the point is that it seemed like a good candidate for looking at the attitude respondents might have toward something cumbersome but authentically Gygaxian. However, even self-declared OSR participants on average dislike Alignment languages. I interpret this reported attitude as some evidence that Gygaxian fundamentalism, or the regard for something solely based on the fact that Gygax did it, is less central to the meaning of OSR than many other elements in the constellation of meaning making up OSR. Similarly, while respondents generally agreed that AD&D was OSR (and to a greater degree among self-declared OSR participants: MOSR = 5.85, Mnon-OSR = 5.08), AD&D was rated less OSR than the games closer to Basic D&D in the set of games I presented.

Based on these comparisons, it is clear that a set of distinct attitudinal preferences toward game elements at least distinguishes OSR participants from non-participants, and perhaps underlies the division. However, the attitudes differ in ways that are somewhat counter-stereotypical. It seems that few respondents like the idea of experiencing a planned narrative, contrary to some assumptions I have seen in OSR circles, and old rules pedigree (or that related concept, nostalgia) is insufficient alone to generate positive regard among OSR participants, contrary to assertions I have seen outside OSR circles.


OSR survey report

Table of contents

  1. Purpose & participation
  2. What about the R?
  3. Meaning respondents associate with OSR
  4. OSR games
  5. OSR attributes

OSR games

(This is one part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

Beliefs about the nature of OSR, according to previous exploration of the 2018 OSR survey responses, seem to cohere around three broad themes or aspects: OSR as rules tradition, OSR as commerce, and OSR as social scene. Of the three aspects, respondents overall agreed most strongly that OSR is a rules tradition. There was greater ambivalence around the other two aspects, with commerce rating lowest on average. This naturally leads to the question of what kinds of rules and which games seem most OSR. In this post, I will discuss which games survey respondents believed were most (and least) OSR.

According to respondents, the most OSR games, out of the games I asked about, are Labyrinth Lord, Swords & Wizardry, Lamentations of the Flame Princess, and B/X D&D, all scoring above 6 (Agree) on average, across all participants, with LL narrowly taking the top slot. The least OSR games are Fate, Pathfinder, and Vampire: The Masquerade, all scoring below 2 (Disagree) on average, across all participants. I think this set entails reasonable coverage, though there are a few unfortunate omissions in retrospect, such as the lack of OSRIC. Respondents seemed neutral about whether the classic non-D&D games counted as OSR, with Classic Traveller, Runequest, and Call of Cthulhu all hovering around the scale midpoint. I was moderately surprised to see how D&D 5E rated (less OSR than Dungeon World!), given the high regard many people in my circles seem to have for this edition. This is speculation, but I suppose this means that the dominant associations respondents have with D&D 5E remain mainstream trends such as adventure paths, character builds, and tactical set-piece combats. The games respondents rated as least OSR seem to have associations with narrative focus (Apocalypse World, Vampire, Fate) and D&D 3E (Pathfinder).

Near the end of the survey, I asked respondents which games they were currently playing. Respondents could select all that applied from the following list: B/X D&D, D&D 5E, Pathfinder, Home-brew (OSR), Home-brew (other), Vampire, Lamentations of the Flame Princess, Swords & Wizardry, Apocalypse World, Dungeon World, Other PbtA game, Other World of Darkness game, Labyrinth Lord, B/X Essentials, AD&D (1E), GURPS, Shadowrun, Call of Cthulhu, Runequest, Fate, and Savage Worlds. Combining these reports of play behavior with the aggregate OSR ratings of games described above allowed me to compute an OSR play behavior score for each respondent. More on this in a future post.

On average, respondents reported currently playing 3.26 different games (SD = 2.51). 34 respondents (1.86% of N = 1828) reported playing more than 10 different games currently, including one respondent that checked all 18 game options, which seems implausible, but I am going to chalk that up to careless responding/measurement error and expect sample size to wash out that messiness. Without those 34, the average number of different games played drops to 3.08 games (SD = 2.16), but I keep them for all following results, as I predetermined all exclusion criteria.

Keep in mind respondents could select more than one game. Apologies for the small fonts.

Unfortunately, the survey lacked an option to indicate playing DCC, which I suspect would have a substantial number of current players. That was an oversight on my part. I did include an open ended item for other games played, and 142 respondents indicated playing DCC there. Additionally, the two World of Darkness related items were Vampire (or other WoD) and Other World of Darkness game, which overlap. (Clearly I should have proofread this set of options more carefully.) The headline results here are the popularity of D&D 5E (which indicates the power of the market leader) and the popularity of games genealogically related to B/X D&D. In fact, in terms of the games people are actually playing, based on these results one could view OSR as a vehicle for propagating the B/X vision of D&D.

For those unaware, the B/X D&D rules tradition started with the Moldvay Basic rules (covering character levels 1 through 3) and the Cook/Marsh Expert rules (covering character levels 4-14). It continued with the Mentzer-initiated BECMI line (covering character levels 1-36 and collected in the 1991 Rules Cyclopedia), which TSR supported in parallel with the Advanced Dungeons & Dragons line. The B/X line lineage is notable for simple rules and treating character class as a more general abstraction, rather than something closer to profession. For example, “elf” is a class in B/X. Arguably, the modern “playbook” trend derives from the B/X notion of class.

As always, let me know in the comments if anything is unclear, if you have any questions, or if you have any suggestions.


OSR survey report

Table of contents

  1. Purpose & participation
  2. What about the R?
  3. Meaning respondents associate with OSR
  4. OSR games
  5. OSR attributes

OSR survey: meaning

(This is one part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

Over the past few years, I have been continually surprised by the degree of disagreement, confusion, and, occasionally, vitriol regarding the meaning of OSR. I think it is uncontroversial to state that the meaning is contested. It is unclear, however, whether there is any consensus and, more generally, what is the structure of underlying beliefs. Summarizing, the results of the survey support at least three main aspects of OSR meaning. Agreement about these three aspects varies by respondent age, with younger respondents seeing greater conflict between the aspects compared to older respondents. There are other variables to consider, but I will focus on age and self-declared participation in this post.

Age and participation might influence beliefs about OSR for many reasons. Considering rules, older participants might have played older games when they were originally released and so have a different frame of reference compared to younger players. Self-declared participants in the OSR may have different priorities, or more information, compared to self-declared non-participants. We asked respondents directly about participation. (Do you participate in the OSR? Yes/No.) We also asked five questions about identification with the OSR using a seven-point agreement scale (for example, I feel a bond with the OSR). Respondents who participate in the OSR also identify more with the OSR (by approximately 1.75 points of agreement, or the difference between Somewhat agree and Strongly agree).

Before I describe the associations in the survey between meaning beliefs and age or participation, it is worth first looking at the beliefs in aggregate, over all responses. Following are the means of the OSR is… questions, which I think of as the basic “meaning” questions. We also asked each of these questions in terms of respondent desires, so we will be able to look at the discrepancy between what people believe OSR is and what they want it to be.

OSR meaning across all responses

These averages tell a simple story, which is that respondents agree most that OSR refers to a play style and set of design principles. Respondents agree somewhat that OSR is a scene or movement and that OSR is a collection of aesthetics. Respondents are close to neutral, on average, about the commercial aspect of OSR. Looking at the text of each question, and the averages, three pairs seem to hang together conceptually, tapping into game design, social, and commercial aspects, with aesthetics and specific rules off in their own space. Formal statistical analysis1 leads to the same conclusion, so I averaged each of those three pairs into new scores for the game design, social, and commercial aspects. In the following figures, I color code aspects where possible, using black for rules, green for commerce, and blue for social interaction. I created similar scores for aspects as desired (that is, what respondents say they would like OSR to mean).

The following figure shows a box plot of the three aspects. Quartiles determine the look of box plots. The central line in each box shows the median. The diamonds represent outside values, which are values more than 1.5 quartiles away from the median.

OSR meaning and desired meaning based on quartile

One way to interpret the the greater number of outside values for the rules aspect compared to the other aspects is that there is more consensus around the rules aspect. The quartiles are tighter. Looking at a similar figure that compares OSR participants to non-participants yields similar conclusions, though participants seem to both perceive and want a greater social component. (The right pane below is OSR participants.) It is perhaps unsurprising that the discrepancy between perception and desire is greater for non-participants than participants on average, as that mismatch may be part why non-participants avoid participating to begin with.

Box plots comparing meaning for self-declared OSR participants with non-participants

Age provides a potential explanation, at least partially, for broader misunderstanding and disagreement, shown in the following figure. Younger respondents seem to have stronger beliefs about OSR meaning compared to older respondents (as in, positions more strongly held). Further, there is a stark divide between commerce and social/game design aspects for younger players. Older players seem to agree, relatively speaking, that OSR is more about rules than social interaction or commerce, but are basically neutral, on average, regarding OSR as commerce or social interaction. (The pattern looks similar if I plot a figure using years playing tabletop roleplaying games as the independent variable, rather than age.) I suggest only interpreting the linear predictions (the lines) for age < 60, as the number of responses decreases markedly for age > 60.

OSR meaning aspect trends over respondent age

This pattern is similar for non-participants compared to participants, though older non-participants have much stronger aversion to the social component of the OSR (see figure below). Though the left pane may look sparse compared to the right pane, 372 responses is substantial.

OSR meaning aspect trends over respondent age, separating non-participants (left) and participants (right)



1. A three-factor model fits the data best, with the three item pairs loading heavily on factors consistent with the three aspects explained above. For stats nerds, I used exploratory factor analysis with maximum likelihood estimation and oblique promax rotation. I chose oblique rotation because the latent variables are correlated above a conventional threshold, making orthogonal rotation less appropriate. I chose to score latent variables using averages rather than factor scores so that interpretation of the results in terms of the seven-point response scale is possible.


OSR survey report

Table of contents

  1. Purpose & participation
  2. What about the R?
  3. Meaning respondents associate with OSR
  4. OSR games
  5. OSR attributes

OSR survey participation

(This is one part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)

This is the first post of several where I will summarize the results of the OSR survey Ben and I ran. The survey was open from December 3rd through December 8th (2018). Given the high level of participation, and a handful of common critiques, after I have a chance to post some results I plan to ask for general feedback about how we ran the survey and what people might like to see in the future. For those who want a short summary, the TLDR of this post is that there were 2000+ responses (that is a lot), mostly from North America/Europe, and some basic demographics (such as age and country of residence) are similar for OSR participants and non-participants (based on self-categorization).

Purpose

A few people asked why we were running the survey at all. The superficial (but true) answer is that I saw Ben post a question on Google Plus about what questions people might like to see on a hypothetical OSR survey. I was curious too, and I have some experience running surveys so I reached out, and here we are. The slightly more involved (and also true) answer is that I have been, over the last several years, continually surprised by seeming confusion, misunderstanding, and disagreement surrounding the meanings that people attach to the term OSR. Personally, I am primarily curious about the degree to which there is consensus about this meaning, whether the meaning differs substantially between people that consider themselves to be participating in (the?) OSR, and whether the meaning people currently see matches what they would like. Secondarily, I am also curious about what content (at the game level) people consider OSR, the geographical distribution of people that participate in (the?) OSR, and favorite OSR products. Other questions may occur to me as I proceed (and feel free to suggest questions if you have any).

Responses

2018 respondents completed the survey (that serendipitous number is a coincidence). Based on some preliminary checks, I see no sign of brigading, bots, or similar threats to data quality. 134 responses failed the attention check near the end. 46 responses provided age greater than 100 and one response gave age less than 10. 9 respondents specified that I should discard their response entirely. None of these responses will contribute to my summaries. After exclusion, there were 1,828 responses. To the question Do you participate in the OSR?, 1450 respondents said Yes, 372 respondents said No, and 6 skipped the question.

Generalizability

Several people seemed suspicious about why we asked any demographic questions. The point of such questions is have some basis from which to generalize the results. For example, if respondents skew older, it would be inappropriate to assume results represent the beliefs of younger players. And so forth. Additionally, many people indicated, both to me directly and in the final “share anything” question, that they found the two items about political orientation insufficient to accurately communicate beliefs. Due to this feedback, and to ease concerns that there is some prior political agenda motivating the survey, we decided to avoid looking at those measures. Most obviously, these results speak more to the beliefs about people who talk about games online. Only people who frequent RPG forums or follow people who talk about RPGs on social media are likely to have heard about the survey at all, so that is a property of the sampling process. Put another way, we oversampled (heavily, perhaps close to exclusively) people who spend large amounts of time on hobbies online. That is, however, appropriate to OSR, which originated and developed online.

Figures

The first two figures show scatter plots of responses over time. The first plot shows how long participants took to answer the survey, in minutes. For ease of visual interpretation, I avoided plotting a handful of observations where respondents took an extra-long amount of time to submit the survey, though these responses still contribute to other results. The second plot shows respondent age, which drifts very slightly up according to the trend over time, though it is difficult to see. Plots such as these often show potential confounds or technical errors. Nothing like that jumps out at me.

Response Duration Over Time (the X axis represents about one week)

Respondent Age Over Time (the X axis represents about one week)

Responses by day (survey dashboard)

The mean response age was 36.81 (SD = 9.88, n = 1828) and the stats are similar for self-declared OSR participants (Mage = 36.58, SD = 9.87, n = 1450) and self-declared OSR non-participants (Mage = 37.64, SD = 9.90, n = 372).

Respondent Age Distribution

There was a mix of people that have been playing tabletop roleplaying games for a while and people that started more recently. My guess is that the frequencies by year shown below approximately reflect general industry fortunes.

Year Started Playing Tabletop Roleplaying Games

I categorized responses by rough geographic area based on country of residence. Yes, I know that “Asia” is an oversimplification, but look at the number of responses before you object. Participants that skipped the country item, or responses that were ambiguous to me, show up in the “Uncertain” category (as in, I am uncertain). The majority of responses are, unsurprisingly, from North America (65%), and 89% are from North America plus Europe combined (being relatively expansive about the definition of Europe). So when you are thinking about what kinds of conclusions you can draw from these results, keep that in mind. This survey should give a decently representative picture of people from North America and Europe who talk about tabletop roleplaying games online, but speak less to players living in other cultures.

Geography or responses (overall)

Geography of responses (self-declared OSR participants)

Geography of respondents (self-declared OSR non-participants)

The social media context in which people reside may be even more important, for purposes of measuring beliefs about OSR, than physical geography. Respondents indicated where they heard about the survey, and that seems like a reasonable, if imperfect, proxy for social media environment. The immediate takeaway here is that Reddit is huge. The secondary takeaway is that the survey represents a much wider swath of RPG players than those that I hang around, as my crowd is mostly on Google Plus and blogs at the moment. We also asked people about how often they used various platforms for talking about games, but I think that is enough for now.

Social media context


OSR survey report

Table of contents

  1. Purpose & participation
  2. What about the R?
  3. Meaning respondents associate with OSR
  4. OSR games
  5. OSR attributes