Gateway Survey Items

Lighting a torch to explore (source)

We closed the gateway RPG survey today. Below is a brief description of the survey items, number of responses, and some questions we can ask based on the data. If you think of other questions that you are curious about, leave a comment below. I have yet to look at the data for the final set of responses beyond some descriptives.

Following are the items in presentation order. The bold text represents the concept, behavior, or whatever, that we were trying to measure. Italic text represents the exact item text participants saw. Parenthesized text describes the variable type (numerical scale with label, free-response text, and so forth).

I have also included descriptive stats for some of the items (N = number of responses for the particular item, M = mean value, SD = standard deviation). Keep in mind that for the seven point scales, 4 = Neutral, 5 = Somewhat Like/Agree, 6 = Like/Agree, and so forth. 2764 responses provided the correct answer to an attention check item near the end; the stats below only include responses that answered the attention check correctly.

So: ConceptItem text (description of variable type); maybe some descriptive stats.

  1. First RPGWhat was the first tabletop roleplaying game you played? (Selective list of options that we brainstormed, along with an “other” option permitting free-response text for anything we missed); N = 2549, top three: Dungeons & Dragons, any edition (n = 1658), Other (n = 508), Pathfinder (n = 120). 65.05% of respondents that answered this item started with an edition of D&D.
  2. Attitude toward first RPGThink back to the first tabletop RPG you played. How much do you like that game now? (1 = Strongly Dislike, 7 = Strongly Like)
  3. If first RPG was D&D: First D&D EditionWhich edition of D&D did you begin with? N = 1827, top three: 3/3.5E (n = 487), 5E (n = 382), B/X (n = 318).
  4. OwnershipDo you own any tabletop RPG materials? (Books, box sets, and so forth.) (Yes, No); N = 2764, Yes = 2688, No = 76 (97.25% Yes)
  5. Still play first RPGDo you still play the first RPG that you started with? (Yes, No); N = 2764, Yes = 1049, No = 1715 (37.95% Yes)
  6. Delay trying another RPGHow long (in years) after playing your first RPG did you try another RPG? (non-negative integer free-response)
  7. Attitude toward D&DHow much do you like Dungeons & Dragons? (For your favorite edition of D&D.) (1 = Strongly Dislike, 7 = Strongly Like); N = 2028, M = 5.27, SD = 1.63
  8. Belief about effect of D&D on the hobby as a whole (item 1)—The popularity of Dungeons & Dragons attracts new tabletop RPG players. (1 = Strongly Disagree, 7 = Strongly Agree); N = 2618, M = 6.25, SD = .94
  9. Belief about whether D&D crowds out other RPGsThe popularity of Dungeons & Dragons makes discovering other (non-D&D) tabletop RPGs HARDER. (1 = Strongly Disagree, 7 = Strongly Agree); N = 1613, M = 4.16, SD = 1.88
  10. Belief about the effect of D&D on the hobby as a whole (item 2)—The popularity of Dungeons & Dragons is good for the tabletop RPG hobby. (1 = Strongly Disagree, 7 = Strongly Agree); N = 2676, M = 5.64, SD = 1.34
  11. Favorite edition of D&DIf you have played D&D, which edition is your favorite? N = 2348, top three: 5E (n = 1121), B/X (n = 440), 3/3.5E (n = 270). 47.74% of respondents that answered this item said that 5E was their favorite edition of D&D.
  12. Preference for designer authority (item 1)—There are a lot of house rules (customization) in RPGs games that I run or play in. (1 = Strongly Disagree, 7 = Strongly Agree)
  13. Belief regarding whether system mattersWhen it comes to tabletop RPGs, a well-designed rules system is an important factor in enjoyable play. (1 = Strongly Disagree, 7 = Strongly Agree); N = 2217, M = 5.86, SD = 1.13
  14. Preference for designer authority (item 2)—I prefer to play RPGs “as written” rather than adjusting, customizing, or hacking the rules. (1 = Strongly Disagree, 7 = Strongly Agree)
  15. Preferences in play styleFor satisfying play, how important are the following aspects to you? (Aspects included: In-game shenanigans, Generating a satisfying story, Acting in character, Hanging out with friends, Challenge Exploration and discovery, Creative problem solving, Character optimization, Improving my character, Meticulous plotting; 1 = Very Unimportant, 7 = Very Important)
  16. Preferences in game materials—How important are the following elements to you in tabletop RPGs? (Elements included: Fictional setting, Hackability, Mechanical innovation, Ease of use, Art, Genre emulation; 1 = Very Unimportant, 7 = Very Important)
  17. Pleasure readingI read tabletop RPG materials for pleasure, apart from intention for direct use in play. (1 = Strongly Disagree, 7 = Strongly Agree); N = 1300, M = 5.81, SD = 1.37
  18. Play usageI have played most of the tabletop RPGs (systems, modules, adventures, etc.) that I own. (1 = Strongly Disagree, 7 = Strongly Agree); N = 1327, N = 4.16, SD = 2.08
  19. RPG play literacy/promiscuityI have played a wide variety of tabletop RPGs. (1 = Strongly Disagree, 7 = Strongly Agree); N = 1370, M = 5.41, SD = 1.64
  20. Attention check
  21. Age startedAt what age, approximately, did you start to play tabletop RPGs? (integer from the set [1, 200]; due to a typo, some respondents were only able to enter starting ages from the set [10, 200]); N = 2754, M = 15.93, SD = 6.79
  22. Year startedIn approximately which year did you start playing tabletop RPGs? (Drop-down menu with year options); N = 2721, M = 2002.04, SD = 12.25, min = 1971, max = 2019
  23. Other feedbackOptional: Is there anything else you would like to share with us? (free response text)

You may notice that we measure several concepts using more than one item. For example, presumably playing with lots of house rules and preference for playing rules as written should both tap into a preference for designer authority (and they correlate, as expected, at r(2437) = −.62, p < .001). Another: “D&D attracts new players” and “popularity of D&D is good for the hobby” (r(2550) = .46, p < .001).

(The number of observations for pleasure reading, play usage, and literacy/promiscuity seem systematically somewhat low: ~1300 compared to ~2700 for most items; I need to look into that.)

One (somewhat) surprising point that jumped out at me from the descriptives is that for people that started with an edition of D&D, 3E was the most common.

Some possible questions:

  • Do player preferences for game aspects cohere into conventional clusters? Are those preferences consistent with what you might expect? (For example, are people starting with 3E more likely to have a preference for character optimization?)
  • Is starting with an edition of D&D (compared to other RPGs), controlling for years played, associated with greater or lesser RPG play literacy overall?
  • Is starting with an edition of D&D (compared to other RPGs) associated with any of the preferences for aspects of play style or elements of game materials?
  • According to consensual belief, does system matter? (At least based on the mean, the answer here is yes: M = 5.86, roughly = Agree).
  • Is starting with an edition of D&D associated with preference for designer authority in rules?

What else would you like to know based on this survey?

Addendum: I just did a tally of referrer URL to see where respondents saw a link to the survey. To roughly summarize, about 30% came from Reddit, 30% came from Twitter, and 30% came from YouTube, with the remaining referrer URLs a smattering of other sites. In detail, N = 1167; the top five: YouTube (n = 363), Reddit (n = 358), Twitter (n = 311), survey.ascolais.com (n = 59), Facebook (n = 50). Referrer URL data only exists for some responses (1167 out of 2764), but preliminary checks indicate that referrer URL is missing at random (meaning it is probably reasonable to think of it as a random sample and representative of the respondent population at large).

6 thoughts on “Gateway Survey Items

  1. allan grohe

    Some other Qs that come to mind:

    – What does the dataset look like if you remove 3e+ and other non-OSR games from the mix?: what picture of OSR gamers does that paint?
    – Curious to see permutations of 15, 16, and 17 by age, first edition, and OSR vs. non-OSR responses
    – 19 would be interesting to see a follow-up query on how many systems respondents have played and comparing to expressed “wide variety” of systems, then controlling for age/starting year to look at starting system: does starting with game X mean you’re more likely to play more different RPGs
    – Drilling down into 4 would be useful too, to compare OSR vs. non-OSR gamers, and perhaps to follow-up with ranges of spending $ per year; might also be interesting to compare what MSRP for an RPG book is considered “high” or “expensive”

    Allan.

    Reply
    1. Necropraxis Post author

      @Allan

      I definitely plan to dig more into items 15, 16, and 17 (I omitted descriptives here just because there are so many sub-items and it would end up being just a sea of numbers).

      For your first point, I think reasonable proxies might be comparing: starting with a TSR edition of D&D, starting with a WotC edition of D&D, and starting with a non-D&D RPG. (Doing this most accurately will require reading through the “other” responses and categorizing them appropriately, though my intuition is that doing this will probably make a minimal impact on resulting interpretation.)

      Unfortunately there’s almost no variance in responses to item 4 (almost everyone said they owned some RPG materials, and the response scale for this item is binary Yes/No), so exploring buying behavior in more detail would require some form of followup. The useful info from this item is more in terms of representativeness (we can assume that the respondents overall are invested enough in the hobby to own some RPG materials; unsurprising, but good to confirm).

      Reply
      1. Necropraxis Post author

        @Beoric

        I wasn’t really planning on using that frame at all myself. For this data, I think it is better to report the info about editions and so forth, along with any relationships that I discover. People can draw their own conclusions about what counts as OSR (or what have you).

  2. Slick

    I’d like to see what edition of D&D had the highest rate abandonment (i.e. choice for question 3 with most negative responses to question 5)

    I’ll second Allan’s interest in seeing the breakdown of 15/16/17.

    Reply
    1. Necropraxis Post author

      @Slick

      I just ran a few regressions, and the editions of D&D with highest rate of abandonment were 3E > 2E > 4E.

      Starting with 5E yields by far (comparatively) the greatest probability of still playing the initial gateway RPG (and this holds when controlling for starting year).

      My guess is that the 3E relationship is partly explained by a move to Pathfinder and/or 5E.

      (I will include formal stats for this in a future post.)

      Reply

Leave a Reply