This has been a development of perhaps the last 20 years in which researchers publish a “protocol paper” as a public declaration of what they plan to do. A protocol is the word most used in the research world for a document specifying what data collection plan and data analyses are planned.
Details #
Prior to the emergence of protocol papers protocols were part of ethics committee application, or a grant funding application, but weren’t generally publicly accessible making it hard for people reviewing or just reading published research to know whether the authors followed their protocol or not.
This matters, particularly in the Null Hypothesis Significance Testing (NHST) statistical paradigm, as that entire paradigm rests on the idea that data analysis is completely confined to what was planned. Where that is the case, in that paradigm the findings have stronger likely replicability and trustworthiness, far more than if what is reported arises from cherrypicking and reporting only what looked interesting post hoc (essentially an extremely prevalent form of research fraud somewhat reduced by the replicability movement, pre-registration and protocol papers). Journals and researchers working in that dominant paradigm in the quantitative world now generally expect that the work should have a protocol paper or that a protocol should have been placed in an open repository making it possible for us to check the final reports/papers methods against those described in the protocol.
This is part of the “replicability” movement and a counter to the “replicability crisis” and it is one protection against some forms of research fraud. There is much that is clearly excellent about the idea if work is being done in the quantitative NHST paradigm. However it is unclear that the idea has the same merits outside that paradigm.
There have been some interesting explorations and suggestions about using protocols and protocol papers and more of the principles of the “replicability movement” in qualitative research. However, there are also important concerns that the movement can further entrench research in then NHST paradigm and amplify the problem of research treating the real world as if it were a laboratory or a factory production line. It can be hard to frame practice based evidence work and practice oriented research within the entirely pre hoc protocol model.
Try also #
Ethics committees
Null Hypothesis Significance Testing (NHST)
Practice Based Evidence (PBE)
Practice Oriented Research (POR)
(Pre)registration
Pre hoc hypotheses
Post hoc hypotheses
Replicability
Repositories
Research fraud
Chapters #
Not covered in the book.
Online resources #
My own examples are:
Grau Touriño, A., Feixas, G., Medina, J. C., Paz, C., & Evans, C. (2021). Effectiveness of integrated treatment for eating disorders in Spain: Protocol for a multicentre, naturalistic, observational study. BMJ Open, 11(3), e043152. https://doi.org/10.1136/bmjopen-2020-043152
and
Paz, C., Osejo-Taco, G., & Evans, C. (2021). Trajectories of success and/or distress: Protocol for an observational cohort study investigating changing psychological distress among emerging Ecuadorian adults over a year. BMJ Open, 11(12), e056361. https://doi.org/10.1136/bmjopen-2021-056361
They’re both open access so clicking through those doi links should get you to the full papers if want to read the papers!
Dates #
First created 15.ix.23, updated 15.xi.26.