The Library
Data quality of platforms and panels for online behavioral research
Tools
Peer, Eyal, Rothschild, David, Gordon, Andrew, Evernden, Zak and Damer, Ekaterina (2022) Data quality of platforms and panels for online behavioral research. Behavior Research Methods, 54 . 1643-1662 . doi:10.3758/s13428-021-01694-3 ISSN 1554-351X.
Research output not available from this repository.
Request-a-Copy directly from author or use local Library Get it For Me service.
Official URL: http://dx.doi.org/10.3758/s13428-021-01694-3
Abstract
We examine key aspects of data quality for online behavioral research between selected platforms (Amazon Mechanical Turk, CloudResearch, and Prolific) and panels (Qualtrics and Dynata). To identify the key aspects of data quality, we first engaged with the behavioral research community to discover which aspects are most critical to researchers and found that these include attention, comprehension, honesty, and reliability. We then explored differences in these data quality aspects in two studies (Nā~ā4000), with or without data quality filters (approval ratings). We found considerable differences between the sites, especially in comprehension, attention, and dishonesty. In Study 1 (without filters), we found that only Prolific provided high data quality on all measures. In Study 2 (with filters), we found high data quality among CloudResearch and Prolific. MTurk showed alarmingly low data quality even with data quality filters. We also found that while reputation (approval rating) did not predict data quality, frequency and purpose of usage did, especially on MTurk: the lowest data quality came from MTurk participants who report using the site as their main source of income but spend few hours on it per week. We provide a framework for future investigation into the ever-changing nature of data quality in online research, and how the evolving set of platforms and panels performs on these key aspects.
Item Type: | Journal Article | ||||||||
---|---|---|---|---|---|---|---|---|---|
Divisions: | Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School > Health Sciences Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School |
||||||||
Journal or Publication Title: | Behavior Research Methods | ||||||||
Publisher: | Springer | ||||||||
ISSN: | 1554-351X | ||||||||
Official Date: | August 2022 | ||||||||
Dates: |
|
||||||||
Volume: | 54 | ||||||||
Page Range: | 1643-1662 | ||||||||
DOI: | 10.3758/s13428-021-01694-3 | ||||||||
Status: | Peer Reviewed | ||||||||
Publication Status: | Published | ||||||||
Access rights to Published version: | Open Access (Creative Commons) | ||||||||
Date of first compliant deposit: | 24 March 2022 |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |