Tag: Data collection

Workshop Recap: Introduction to Online Surveys

Anna Hohwü-Christensen

The use of online surveys in contemporary social science research has grown rapidly due to their many benefits such as cost-effectiveness and ability to yield insights into attitudes, experiences, and perceptions. Unlike more established methods such as pen-and-paper surveys, they enable complex setups like experimental designs and seamless integration of digital media content. But despite their user-friendliness, even seasoned researchers still face numerous challenges in creating online surveys. To showcase the versatility and common pitfalls of online surveying, Martin Emmer, Christian Strippel, and Roland Toth of the Methods Lab arranged the workshop Introduction to Online Surveys on February 22, 2024.

Martin gave a presentation on the design and logic of online surveys.

In the first segment, Martin Emmer provided a theoretical overview of the design and logic of online surveys. He started by outlining the common challenges and benefits associated with interviewing, with a particular emphasis on social-psychological dynamics. Compared to online surveys, face-to-face interviews offer a more personal, engaging, and interactive experience, enabling interviewers to adjust questions and seek clarification of answers in real time. However, they can be time-consuming and expensive and may introduce biases such as the interviewer effect. On the other hand, the process of conducting online surveys presents its own set of challenges, such as limited control over the interview environment, a low drop-out threshold, and particularities connected with self-administration such as the need for detailed text-based instructions for respondents. Nevertheless, self-administered and computer-administered surveys boast numerous advantages, including cost-effectiveness, rapid data collection, the easy application of visuals and other stimuli, and accessibility to large and geographically dispersed populations. When designing an online survey, Martin stressed the importance of clear question wording, ethical considerations, and robust procedures to ensure voluntary participation and data protection. 

Christian shared his insights on survey creation using online access panel providers.

In the second part of the workshop, Christian Strippel delved into the realm of online access panel providers, including the perks and pitfalls associated with utilizing them in survey creation. Panel providers serve as curated pools of potential survey participants managed by institutions, such as Bilendi/Respondi, YouGov, Cint, Civey, and the GESIS Panel. Panel providers oversee the recruitment and management processes, ensuring participants are matched with surveys relevant to their demographics and interests, while also handling survey distribution and data collection. While the use of online panels offers advantages such as accessing a broad participant pool, cost-efficiency, and streamlined sampling of specific sub-groups, they also have their limitations. Online panels are, for example, not entirely representative of the general population as they exclude non-internet users. Moreover, challenges arise from professional respondents such as so-called speeders who rush through surveys, and straight-liners who consistently choose the same response in matrix questions. Strategies to combat these issues include attention checks throughout the questionnaire, systematic exclusion of speeders and straight-liners, and quota-based screening. To conclude, Christian outlined what constitutes a good online panel provider, and shared valuable insights into how to plan a survey using one effectively.

Participants learned how to create their own survey using LimeSurvey during Roland’s live demo.

The third and final segment of the workshop featured a live demonstration by Roland Toth on how to set up an online survey using the open-source software LimeSurvey, which is hosted on the institute’s own servers. During this live demonstration, he created the very evaluation questionnaire administered to the workshop participants at the end of the workshop. Roland began by providing an overview of the general setup and relevant settings for survey creation. Subsequently, he demonstrated various methods of crafting questions with different scales, display conditions, and the incorporation of visual elements such as images. Throughout the demo, Roland addressed issues raised earlier in the first part of the workshop concerning language and phrasing, emphasizing rules for question-wording and why it is important to ask for one piece of information only per question. The live demonstration was wrapped up with a segment on viewing and exporting collected data. After letting the participants complete the evaluation form, the workshop concluded with a Q&A session.

The workshop was held in-person and online.

Recap: Digital Methods Colloquium (December 7, 2023)

Roland Toth

Digital and computational data collection and analysis methods such as mobile/internet tracking, experience sampling, web scraping, text mining, machine learning, and image recognition have become more relevant than ever in the social sciences. While these methods enable new avenues of inquiry, they also present many challenges. It is important to share and discuss research, experiences, and challenges surrounding these methods with other researchers to exchange ideas and to learn from experiences.

For this reason, Roland Toth from the Methods Lab and research fellow Douglas Parry organized the Digital Methods Colloquium that took place on December 7 at the Weizenbaum Institute. They invited researchers from all over Germany who had used such methods before. The focus lied on sharing not only successes, but – even more so – the challenges that they had experienced in the research process.

Fenne Große Deters (U of Potsdam) talking about the effects of smartphone use on sleep quality

In the first part of the colloquium, participants presented recent or past research projects for which they had used digital methods. The presentations covered various methods, including experience sampling, mobile logging/tracking, multimodal content classification, network analysis, and large language models. All presentations were received very well and led to high engagement with many questions and exchanges from the participants.

The second part of the colloquium was designed to facilitate interactive discussion and knowledge sharing among the participants. They were assigned to one of two discussion groups that focused on either data collection or data analysis in the context of digital methods. In each group, participants followed prompts and discussed urgent issues and possible solutions, which they then visualized using posters. Finally, both groups sat together and presented the posters to each other, leading to a final discussion. After a short wrap-up, some participants joined the hosts at the Christmas Market for a well-deserved hot beverage.

Patrick Zerrer (U of Bremen) talking about mobile usage patterns of young political activists

The hosts would like to thank all participants for attending and engaging in the Digital Methods Colloquium. Bringing together researchers from different fields demonstrated that there are more commonalities than differences when it comes to the challenging and exciting field of digital methods. We are looking forward to more exchange and, possibly, Part 2 of the Digital Methods Colloquium sometime in the future.

Introducing LimeSurvey at WI

Methods Lab

Surveys are an important method for data collection. Whether it is for conducting internal assessments, gathering feedback, or collecting valuable research data, a reliable survey tool is an integral piece in the methodological toolkit of any researcher. Using different survey tools for different projects leads to differences in the quality of data collection and unnecessary licensing costs. In order to find a more sustainable solution, the Methods Lab assessed some of the most popular survey tools with the aim of finding the ideal one to cater to the specific needs of the Weizenbaum Institute’s research groups and administrative departments. Important to us was to select a user-friendly, open-source survey tool suitable for research that can be hosted on our own servers. 

In this blog post, we introduce our choice: LimeSurvey. It is a free, open-source survey software with a strong commitment to data protection. It offers a versatile platform for data collection, making it ideal for researchers, academic institutions, and organizations of all sizes. In doing so, we hope that the insights from our survey tool comparison will prove useful to researchers and institutions beyond our own. 

Here are some of the distinctive advantages that we were able to identify, making LimeSurvey a compelling choice for research and data collection:

  • Cost-Effective and Open Source: LimeSurvey is open source, meaning, it is available for free when hosted on your own servers, thereby eliminating the need for costly licensing fees.
  • Data Protection: LimeSurvey prioritizes data privacy – a particular advantage appreciated by our IT department due to its compliance with the GDPR. Its servers are strategically located in Germany and Finland, ensuring adherence to stringent European data protection regulations.
  • User-Friendly Integration: LimeSurvey seamlessly integrates with existing user accounts, simplifying the onboarding process without requiring additional account setup.
  • Suitable for Research: LimeSurvey is designed with research needs in mind. It offers a wide range of features, including unlimited projects and administrators/accounts. This flexibility makes it suitable for both simple and complex research projects.
  • No Artificial Limits: LimeSurvey imposes no artificial limitations on user accounts, participants, or projects.

The WI LimeSurvey installation can be accessed by members of the institute here: https://limesurvey.weizenbaum-institut.de/index.php/admin/. Its use is documented in the internal Wiki.

Happy surveying!

Workshop Recap: Whose Data Is It Anyway? Ethical, Practical, and Methodological Challenges of Data Donation in Messenger Groups Research (August 30, 2023)

Methods Lab

On August 30th, 2023, the Methods Lab and Olga Pasitselska (U of Groningen) organized the workshop on data donation in messaging groups research. The workshop intended to tackle practical and ethical issues behind data collection, processing, and dissemination in the research of closed messaging groups. We asked four colleagues to share their experiences and struggles and provide their solutions for closed chat groups research. The invited speakers, Sérgio Barbosa (U Coimbra), Katharina Knop-Hülß (HMTMH Hannover), Connie Moon Sehat (Hacks/Hackers), and Julian Kohne (GESIS), paved the way for better conceptualization of messaging groups and application of tailor-made ethical and practical solutions. The workshop allowed for a cross-field discussion of ad-hoc developments in closed groups research and provided many insights for the audience, speakers, and organizers.

Sérgio Barbosa explained his approach of joining activist WhatsApp groups in Brazil. Sérgio suggested that informed consent cannot be assumed as a one-off solution: instead, one should go beyond the check-list of ethical guidelines and learn by doing and negotiating with the group members. When joining these types of groups, researchers should clearly state the purposes of the research and disclose their identity, and also share the outcome of the research and promote it in the local community as well. Different approaches should be taken, depending on the type of groups: for example, pro-democracy groups and extremist groups should be treated differently, independent of the group size.

Sérgio Barbosa shares his experience with digital ethnography in WhatsApp groups.

Dr. Katharina Knop-Hülß shared insights about studying non-professional secondary groups (e.g., choir, sport, volunteer groups) with her highly unobtrusive and highly invasive research approach of scraping chats’ content. Since these groups were representative of intimate environments of everyday communication, they can be considered as “safe spaces”, closed from the public eye. To account for the sensitive nature of the data collection, Katharina used an opt-in approach, provided pseudonymized chat logs to the participants before they consented to participate, and complied with the requirement not to share this data with anyone beyond the research team, even after the data was pseudonymized.

Dr. Katharina Knop-Hülß during her presentation, “The Permanently Connected Group (PeCoG)”.

Julian Kohne introduced his digital platform for WhatsApp data donation that automatically cleans and anonymizes the data, reducing researchers’ exposure to and intervention in the raw data. In his research, Julian takes a participant-centered approach: the data collection tool is designed to maximize usability and control of the data for research participants. They can pre-process the data in a way that allows them to review the chat logs and decide what exactly they want to donate, deleting undesirable pieces of data, up to the possibility of deleting time stamps and other meta-data. With that, the tool also allows researchers to track how much and what types of data was deleted.

Julian Kohne introducing his digital platform for transparent WhatsApp data donation.

Dr. Connie Moon Sehat presented the meta-review of closed messaging apps research that aimed to determine what are the conditions in terms of indexed invites, group size, discussion topics, or other aspects of closed groups that make them arguably public or private. Adding to the previous speakers’ examples of their research with activist/public and hobby and friends/private types of groups, the review summarized the discussed points and provided a framework for mapping chat groups according to the multiple parameters. Whether researchers scraped the groups without entering them, entered with invitation, disclosed or not their identity and research interest, depended on the nature of the groups and public interest that can justify researchers’ intervention into the closed communication spaces. Connie also stressed the possible differences in perceptions of groups’ “publicness” between users, researchers, and platforms, that also should be taken into account.

Dr. Connie Moon Sehat gave the presentation, “Ethical Approaches to Closed Messaging Research… And Data Collection?”.

After four presentations, we continued the discussion with the online and offline audience, addressing the issues of generalizability of messaging data (what slice of the “natural” social interaction are we looking at here?), the role of language, and the differences between long- and short-term groups. We also discussed what is the role of the researcher in the automated versus manual data collection process, and how participants can benefit from data donation.

The workshop provided theoretical and practical insights for messaging groups research and outlined future directions for collaboration in creating the guidelines for ethical closed messaging research and data donation.

Workshop Recap: Web Scraping and API-based Data Collection

Anna Hohwü-Christensen

On March 2nd, the Methods Lab hosted its first-ever workshop, Web Scraping and API-based Data Collection. The workshop explored various techniques for accessing and gathering data from platforms using APIs and web scraping. Speakers included Florian Primig (FU Berlin), Steffen Lepa (TU Berlin), Felix Gaisbauer (WI), and Leon Wendel (WI). The workshop received an overwhelmingly positive response, with many people attending both in person and remotely. It generated plenty of discussions and concluded with a Q&A session.

Lion Wedel gives an introduction to Web-Scraping (photo: Roland Toth).

Thanks to all our presenters and participants in helping us create such a successful first event. We look forward to organizing more workshops in the future on emerging methodologies in the realm of digital research!

Workshop: Web Scraping and API-based Data Collection (March 2, 2023)

Methods Lab

We hereby present the first workshop at the Institute to emerge from the methodological needs that were indicated in our institute-wide survey in December. It is titled Web Scraping and API-based Data Collection and takes place on March 2.

After an introduction to the topic by the Methods Lab team, Florian Primig (FU), Steffen Lepa (TU), Felix Gaisbauer (WI), and Lion Wedel (WI) will each present various use cases of these two data collection methods. You can find more information about the workshop on its program page.