Alignment Between Free Text and Close-ended Responses for Gender and Sexuality

In recent years, there has been increased interest in creating expansive measures for gender identity and sexual orientation in large-scale social surveys; many of these expansive measures include either a mixed-format question (e.g., with free-text write-in option combined with closed-ended responses) or a fully open-ended option. With the diversification in question formats, I seek to answer the following question: How well do open-ended text boxes for gender identity and sexual orientation correspond to traditional (closed-ended) categorical measures? To answer this question, I use original survey data collected in the US and UK in February 2022 (N=2518) to evaluate the within-respondent correspondence rates between free-text and multiple-choice measures of gender and sexuality. These data were collected using Lucid and had demographic quotas for race (in the US sample), age, region, education, and sex. First, using basic text-cleaning script (i.e., text sequence patterns) and hand-codes for places of ambiguity in my free-text responses, I evaluate the extent to which survey designers and pollsters should be concerned about harmonization and respondent confusion against the potential benefits of greater response diversity. Second, using the US National Center for Health Statistics’ AI tool called SANDS, I predict which respondents are more likely to provide invalid responses (or nonresponses). This paper adds more broadly to the study of social measurement of gender and sexual orientation measures used in large-scale surveys.