Reproducibility certification

Find out about the reproducibility certification scheme at the School of Psychology.

New reproducibility certification scheme

Computational reproducibility refers to the ability to rerun the same analysis on the same dataset and obtain the same results as reported in the original paper. Reproducibility is essential for trustworthy research, but studies suggest that published research is largely not reproducible (Baker, 2016; Stodden et al., 2018; Crüwell et al., 2023).

In March 2023, Professor Zoltan Dienes and Dr Reny Baykova set up a new reproducibility certification scheme open to researchers affiliated with the School of Psychology at Sussex. As part of the scheme, researchers can submit their studies to Dr Baykova who examines whether the shared analysis materials produce the results reported in the paper and provides suggestions for improving the study’s reproducibility. Finally, she completes a reproducibility report and uploads it on the scheme’s OSF page. To date, the scheme has had 15 submissions, and 8 of them have been certified as computationally reproducible.

You can read more about the scheme in this blog post written by Dr Baykova.

• Studies certified as reproducible

  Cabbai, G., Brown, C. R., Dance, C., Simner, J., & Forster, S. (2023). Mental imagery and visual attentional templates: A dissociation. Cortex, 169, 259-278. https://doi.org/10.1016/j.cortex.2023.09.014.

  Cabbai, G., Dance, C., Dienes, Z., Simner, J., Forster, S., & Lush, P. (2024). Investigating relationships between trait visual imagery and phenomenological control: the role of context effects. Collabra: Psychology, 10(1). https://doi.org/10.1525/collabra.92941.

  Gatley, D., Millar‐Sarahs, V., Brown, A., Matcham, F., & Brooks, C. P. (2024). Understanding Early Treatment Response in Brief CBT for Nonunderweight Eating Disorders: A Mixed Methods Study. International Journal of Eating Disorders. https://doi.org/10.1002/eat.24350.

  Lewis, H. M., Drury, J., Choudhury, S., Doyle, L., Eldarwish, H. A. I., Evans, D., Green, F., Mills, F., & Wen, Z. (2025). Understanding the ‘new’ disruptive behaviour at live music events: The role of group norms. PsyArXiv. https://doi.org/10.31234/osf.io/qf9pr_v3

  Souter, N. E., Bhagwat, N., Racey, C., Wilkinson, R., Duncan, N. W., Samuel, G., ... & Rae, C. L. (2024). Measuring and reducing the carbon footprint of fMRI preprocessing in fMRIPrep (Vol. 45, No. 12, p. e70003). Hoboken, USA: John Wiley & Sons, Inc. https://doi.org/10.1002/hbm.70003.

  Sheldrake, K., & Dienes, Z. (2023). Can Imagining Actions as Occurring Involuntarily Cause Intentional Behaviour to Feel Involuntary?. https://doi.org/10.31219/osf.io/74gcn.

  Souter, N. E., Racey, C., Bhagwat, N., Wilkinson, R., Duncan, N. W., Samuel, G., ... & Rae, C. L. (2024). Comparing the carbon footprint of fMRI data processing and analysis approaches.

  Zhag Q., Sun P., Zhao L., Zheng L., Ling X., & Dienes Z. (in prep). Facilitating Implicit Learning of Multiple Non-adjacent Dependencies Regardless of Language Experience: The Role of a Prior Exposure.