The Methods Lab is excited to share a new publication in Journal of Quantitative Description: Digital Media, authored by researchers Jakob Ohme, Timothy Charlton, Roland Toth, Theo Araujo, and Claes H. de Vreese. This paper, titled Bursting Self-reports? Comparing Sampling Frequency Effects of Mobile Experience Sampling Method on Compliance, Attrition, and Sample Biases, explores the effects of different sampling frequencies in experience sampling studies on compliance, sample biases, and reactivity of measures in the context of digital media use.
The study presents an analysis of the impact of daily-intensive burst measures (seven surveys per day) versus hourly-intensive burst measures (12 surveys over two hours per day) on compliance, attrition, and sample biases. To do so, the authors used data collected through the Mobile Experience Sampling Method (MESM) with a mobile-only sample of Dutch Internet users. They reveal differences in compliance, attrition, and sample biases between the two sampling frequency designs.
The results indicate general problems with MESM studies, such as sample attrition during onboarding and sample biases due to age, education, tech savviness, and privacy literacy. They also show that hourly-intensive burst measures lead to lower MESM protocol compliance compared to a more spread-out sampling schedule. However, the study also finds that the average response rate across seven study days does not strongly decrease, suggesting that while the number of measurements may be lower using hourly-intensive burst measures, the day-to-day decrease is a minor issue.
The study highlights the importance of considering systematic biases in MESM studies, particularly during the recruitment phase, and suggests that researchers should be transparent and mindful about these. The findings have implications for future ESM studies with high-frequency sampling, suggesting that researchers should carefully consider the sampling frequency design to minimize biases and ensure high-quality data.