For my graduate-level survey research methods course, I devised a questionnaire and research proposal to investigate attitudes about personal disclosure and privacy on different social media platforms, especially among gender diverse people. I hypothesized that individuals with marginalized identities, including transgender people, would be more likely to either disguise or hide aspects of their identity or share less personal information on social media sites where those identities and/or perspectives put them at greater risk for harm, such as abuse, threats, or career problems.
As a solo project, I was responsible for developing the questionnaire, performing background research, and writing the proposal. Throughout the project, I consulted with my classmates, instructor, and external contacts to test and improve the survey instrument and overall research direction. To explore my research question, I knew I needed to collect survey data on two key areas: social media use and demographics. The specific question and answer sets for each were informed by background research, in particular several previous scholarly studies of social media and identity, as well as general knowledge of survey design best practices.
There were three sections of social media questions, one each for Twitter, Instagram, and Facebook, each with nearly identical question and answer sets (UI copy and functionality differences between the sites reflected) so they could be directly compared. Each section began by asking respondents how often they used a given social media platform and only those who selected “I use it once a week or more often” were shown the following question set. The questions for each platform were divided into two pages to reduce participant fatigue. The first page asked participants about the type of information they share about themselves and their interests and what kind of privacy settings they use. The second, shorter page asked users to rate their agreement with five statements on a 5-point agreement scale. These statements were particularly influenced by previous research about online privacy concerns and community.
For demographics, I wanted to include more aspects of identity than just my area of focus (gender) without seeming too invasive, as well as deliberating excluding answers that might identify respondents. Age was asked as a set of ranges; disability status was a simple yes/no; most questions allowed for multiple selections and every item could be skipped or marked “Prefer not to answer.” “Other” with a text field was an option for many questions, but I aimed to design the questionnaire so most respondents could easily answer without writing anything.
According to feedback from peers and survey testers, I mostly succeeded in my goal to write an easy-to-answer questionnaire that gave respondents options they felt described themselves and their point of view. Data collected with this instrument would have been readily analyzed and compared with (hopefully) minimal manual coding.
An important challenge I faced when designing this questionnaire involved writing questions and answers that were complete, accurate, AND readily answered without being time-consuming. Some of my questions asked users to recall information they might not have in mind, such as profile fields they completed. In an effort to eliminate open-ended questions entirely, I made a list of topics that people might post about, which started out very long (40 or so items) and ended up with 22 named topics, 14 of which combined two or more related topics. These labels might not have been a great fit for many respondents’ mental model of their interests, and I heard feedback from survey testers that they were confused about which boxes to check for more specific interests they had in mind. Additional user testing prior to launching the study would be ideal. Additionally, had I moved forward with this proposal, a key concern would be sampling–gender identity is not a screening question, nor can a gender diverse population be effectively quantified for the sake of random sampling techniques, so careful, targeted recruitment would be necessary to attract respondents with marginalized identities.
Research proposal and questionnaire (PDF) on Google Drive
Throughout this process, it was a pleasure to reconnect with survey research and quantitative analysis skills I first honed working in market research once upon a time. I learned quite a bit about the existing body of research about user behaviors and attitudes toward social media. We sometimes think of this as an emerging field, but relevant studies go back at least 20 years to online forums, newsgroups, and early social media like Friendster. It reinforced my desire to incorporate intersectionality and inclusion principles in my work as much as possible, which is why I did not opt to limit the study to participants of a particular gender identity and instead collect demographics that may reflect multiple oppressions to better contextualize my (proposed) findings.