Publishing reform

Learn about alternatives to the standard publishing route of academic papers.

Improving the trustworthiness of research

We are eager to be involved in and promote actions that improve the accessibility, transparency, and trustworthiness of research. Below you can learn about two publishing reform initiatives that aim to increase transparency and reduce bias in research - Registered Reports (an alternative to the standard format of academic manuscripts), and Peer Community In (an alternative to the typical journal submission process). You can also watch Professor Dienes' 2023 Sussex Beacons Talk for an overview of these and other research reform activities we are involved in.

Registered Reports

A Registered Report is an article type where in-principle acceptance (or rejection) of a manuscript occurs before results are known. This ordering of acceptance relative to results is aimed at dealing with variousbiases that can corrupt the scientific process. When the chain of reasoning from theory to predictions and from data to predictions is constructed in light of the data, distortions can occur: for example, arbitrary predictions that do not actually follow from a theory can be stated (because they happen to match the data); or p-hacking or B-hacking (Bayes factor hacking) can be used to create pseudo-evidence for a preferred outcome. How many papers have you read where the “predictions” at the end of the introduction seemed somewhat arbitrary with respect to theory and yet were strangely confirmed? How many correlations with small N were just significant? A further bias is avoided by Registered Reports: that of editors or reviewers rejecting a paper because they did not like the results. As Chris Chambers often puts it: what part of the scientific process should we never have control over? Yet what part of the scientific process often determines publication – and hence can make or break careers? The results.

One advantage of the Registered Report is that reviewers and editors are collaboratively involved in designing the study – at a point in time when their expert advice can still be seriously considered and acted on if useful. Relatedly, once the procedure and analyses have been in principle accepted, a publication is virtually guaranteed. One does not have to chase various journals for months or years because of some aspect of the study that is regarded as weak by those asked to review. In my experience, the fact that advice from reviewers and editors can inform all aspects of design and analysis (in the absence of knowledge of results), really does change the dynamics of the relation between authors, editors and reviewers.

Do Registered Reports actually reduce bias? In a recent pre-registered study, Registered Reports confirmed the stated predictions less than 50% of the time – yet matched non-Registered Reports confirmed predictions over 90% of the time. Further, Registered Reports are associated with a higher reproducibility of main results from the original data than regular articles; and methodological rigour and quality of Registered Reports are rated substantially higher than non-Registered Reports.

Peer Community In Registered Reports

Peer Community In Registered Reports (PCI RR) is a peer-review community offering an alternative to the review process of academic journals and is co-managed by Prof Zoltan Dienes. Dr Maxine Sherman, Dr Warrick Roseboom, and Dr Pete Lush act as recommenders - the PCI equivalent of journal editors.

In PCI RR, researchers can submit Registered Reports in any discipline can be submitted by linking to a preprint. Manuscripts are refereed, editorially evaluated and (potentially) eventually accepted – “recommended” in PCI speak. The process and outcome are free to authors and free to readers. No institution needs to pay a fee for these freedoms; freedoms we take to reflect the openness that is the sine qua non of science.

Because PCI RR links to pre-prints, the recommended manuscript is not published by PCI RR (though you will have a citable PCI RR recommendation with a DOI). That means an author may (if they wish, though they may not) have the same manuscript published in a mainstream journal. Indeed, we have numerous PCI RR-friendly journals, which guarantee acceptance of our recommended manuscripts without further peer or editorial review (given certain technical conditions are met, such as being in the right discipline and payment of any APCs that the journal would normally charge). Notice it is the author that has the choice of journal, reversing typical power arrangements.

PCI RR overcomes some weaknesses in the current way Registered Reports are managed in journals. As previously mentioned, Registered Reports are completely open and free in PCI RR. Notice this addresses some concerns about equity: financial resources arising from social status (because of race, gender, or institution) are not necessary for having one’s research publicly recommended by the same Registered Reports editors who work for leading journals. (How easy is it for you to pay the APCs for prestigious journals?) PCI RR introduces some other unique key innovations.

With a programmatic Registered Report, a series of related studies can be accepted in principle before the results are in. Each of these studies can then be conducted without further ado, each potentially leading to its own paper. This considerably speeds the process by which a programme of research can be published as a series of Registered Reports.

With a scheduled review, the authors submit a short one-page template-based “snapshot” of their proposed study. The recommender then sends the snapshot to potential reviewers and organises the review process for a fixed future date nominated by the authors. During the intervening time the authors prepare the full manuscript. The aim is to substantially reduce the initial review time.

We operate a graded system of bias control. That means that while the full level of bias control applies to cases where the data does not yet exist, many of the benefits of the Registered Reports process may be still applicable even when the data exist – but for example, cannot be accessed yet; or can be accessed in principle but has not in fact yet been accessed by the researchers, and so on. In each case, we try to control bias as much as possible, while explicitly recognizing there are different risks in different situations.

People interested in becoming a PCI RR Recommender take a two-hour test after reading the extensive guidelines for Recommenders. As far as we know, formal training of Recommenders (editors) is a unique feature of PCI RR.