2022 Preprint
Making peer review evidence-based: It’s time to open the "black box"
Abstract: Peer review serves an essential role in the cultivation, validation, and dissemination of social work knowledge and scholarship. Nevertheless, the current peer review system has many limitations. It is charged as being unreliable, biased, ineffective, and unaccountable, among numerous other issues. That said, peer review is still commonly viewed as the best possible system of knowledge governance, given the relevant alternatives. In this research note, I scrutinize this assumption. Although peer review can som…
Search citation statements
Paper Sections
Select...
5
0
0
0
Citation Types
0
5
0
0
Year Published
2022
2024
Publication Types
Select...
2
2
1
Relationship
0
5
Authors
Journals
Cited by 5 publications
(5 citation statements)
References 102 publications
0
5
0
0
“…Though recent developments in scholarly publishing have enabled the partial or even complete decoupling of peer review from this gatekeeping role (sometimes referred to as "journal-independent" or "journal agnostic" peer review; e.g., Eisen et al, 2022;Hamelin et al, 2022;Lumb 2023), social work journals still largely adhere to a traditional model of double-blind, pre-publication peer review (Caputo, 2019). While the published literature is in some ways validated by this model, the underlying processes are largely unstandardized and opaque, and its overall functioning is poorly understood (see Dunleavy, 2022b;Tennant & Ross-Hellauer, 2019). Blatant errors (e.g., misreported or incorrect statistical results; inaccurate or misleading citations), omissions (e.g., selective reporting of results), misrepresentations (inflation or deflation of findings), and even cases of fraud (e.g., fabrication of data, falsification) are inevitably published (see Dunleavy & Lacasse, 2023;Ioannidis, 2005Ioannidis, , 2008Nuijten et al, 2015;Srivastava, 2016, and references therein).…”
mentioning
confidence: 99%
“…Though recent developments in scholarly publishing have enabled the partial or even complete decoupling of peer review from this gatekeeping role (sometimes referred to as "journal-independent" or "journal agnostic" peer review; e.g., Eisen et al, 2022;Hamelin et al, 2022;Lumb 2023), social work journals still largely adhere to a traditional model of double-blind, pre-publication peer review (Caputo, 2019). While the published literature is in some ways validated by this model, the underlying processes are largely unstandardized and opaque, and its overall functioning is poorly understood (see Dunleavy, 2022b;Tennant & Ross-Hellauer, 2019). Blatant errors (e.g., misreported or incorrect statistical results; inaccurate or misleading citations), omissions (e.g., selective reporting of results), misrepresentations (inflation or deflation of findings), and even cases of fraud (e.g., fabrication of data, falsification) are inevitably published (see Dunleavy & Lacasse, 2023;Ioannidis, 2005Ioannidis, , 2008Nuijten et al, 2015;Srivastava, 2016, and references therein).…”
mentioning
confidence: 99%
“…Peer review remains understudied (Kassirer & Campion, 1994), with a bevy of areas worth pursuing (see generally Ioannidis et al, 2024). Going forward, I hope the field will take much more seriously (1) the need to rigorously investigate the past and current functioning of social work journals and their systems for peer review and (2) the need to develop and test alternative models for peer review, 5 publication practices, 6 and dissemination (see generally Bellantyne, 2022;Dunleavy, 2021aDunleavy, , 2024bEisen et al, 2022;Hamelin et al, 2022;Heeson & Bright, 2020;Priem, 2013;Sever, 2023;Tennant et al, 2017).…”
Section: Looking Aheadmentioning
confidence: 99%
“…For a novel concept (Verification Reports) which combines aspects of the Registered Report format with the concept of reproducibility and (possibly) replication, see Chambers (2020). Recommendations for strengthening the peer review system in social work are made in Dunleavy (2021Dunleavy ( , 2022a.…”
Section: Journal-level Data Auditsmentioning
confidence: 99%
“…For a novel concept (Verification Reports) which combines aspects of the Registered Report format with the concept of reproducibility and (possibly) replication, see Chambers (2020). Recommendations for strengthening the peer review system in social work are made in Dunleavy (2021, 2022a). Grassroots efforts (e.g., Morey et al, 2016) exemplifies how collective action by peer reviewers can support minimum standards for publication and the ideal of open data—while nudging authors and journals to engage with these norms and practices.…”
Section: Potential Remediesmentioning
confidence: 99%
