Lim, Kenneth Teck Kiat (2019) Statistical practice and reproducibility in behavioural science. PhD thesis, University of Warwick.

Preview

PDF WRAP_Theses_Lim_2019.pdf - Submitted Version - Requires a PDF viewer. Download (9Mb) | Preview

Request Changes to record.

Abstract

Psychology and economics are undergoing a 'reproducibility crisis', with researchers attempting to replicate more published studies to verify existing findings. This thesis investigates the role of statistical practice in the reproducibility crisis. A 100-item checklist of recommended statistical practices was developed based on guidelines by the American Psychological Association. The checklist was used to evaluate a sample of psychology and economics studies that were already independently replicated. On average, the evaluated studies adhered to 30% of recommended statistical practice. Incomplete reporting hampered meaningful evaluation of the association between adherence to the checklist items and replication success.

Next, the thesis focusses on the sign effect, which is an established intertemporal choice anomaly. Verbal descriptions of the sign effect are formalised, a hypothesis testing framework is proposed and the concept of a discount rate is critically discussed. Then, the first systematic review and meta-analysis of the sign effect was attempted. Results suggested substantial heterogeneity within and between participants, which is not apparent when the convention of analysing aggregated data is used. There was a surprising amount of observations where no discounting occurred and where discount rates could not be estimated.

Then, individual participants' responses to questions are modelled to estimate the extent to which the outcome sign and other factors influence choices. Results suggested that the later amount was chosen more often for gains than losses. There was substantial heterogeneity within and between studies.

Good statistical practice is central to tackling the reproducibility crisis. Definitions need to be explicitly formalised, data need to be described sufficiently, assumptions need to be explored empirically, study designs need to be informative, and the different types of heterogeneity need to be documented and accounted for.

Item Type:	Thesis or Dissertation (PhD)
Subjects:	B Philosophy. Psychology. Religion > BF Psychology H Social Sciences > HA Statistics
Library of Congress Subject Headings (LCSH):	Reproducible research, Statistics
Official Date:	September 2019
Dates:	Date Event September 2019 UNSPECIFIED
Institution:	University of Warwick
Theses Department:	WBS Behavioural Science Group ; Department of Statistics
Thesis Type:	PhD
Publication Status:	Unpublished
Supervisor(s)/Advisor:	Read, Daniel ; Denrell, Jerker ; Hutton, Jane
Sponsors:	Leverhulme Trust
Format of File:	pdf
Extent:	xviii, 232 leaves : illustrations (some color)
Language:	eng

Request changes or add full text files to a record

Repository staff actions (login required)

View Item

Downloads