Refereeing a BEME review
This guidance paper is for colleagues asked to undertake peer review of a BEME review and review groups as they approach the writing of their review report.
On completion of a BEME review the report on the work is sent for peer review to at least three referees. Colleagues with topic and/or education review methodology experience and expertise are approached for this task. The following guidance is included in the request letter to referees in relation to the key issues considered important in a published BEME review. Referees are also encouraged to make other comments as necessary and to indicate a way forward for the review.
BEME operates a system of peer reviewer anonymity and only by exception do we attach a name to peer review comments. We do not normally ask reviewers to enter into any discussion with the review team or to undertake a second review after revision of the original submission. Colleagues wanting a different approach are asked to let us know.
- Each BEME review should be clear and logical to someone reading that review having not read previous BEME reviews.
- The title should accurately reflect the content of the review, be as short as possible and clearly convey the subject of the review.
- The abstract should be structured, e.g. using the IMRAD format, and be informative and clear.
- The introduction should provide a sufficient background to the review subject and the process of systematic review and clearly identify the research question, aims or hypothesis. There should be a description of any conceptual framework that shaped the review process and reference to relevant and contemporary literature.
- The processes of searching and selecting the primary studies, managing the data from these studies and analysing that data should be logical and clear. There should be comment on quality control procedures for the study selection processes, a full description of the process of critical appraisal and ranking of the primary studies and details of who carried out this work.
- There should a brief description of the source and content of the final set of primary papers used in the analysis: most usually this is best in table format.
- The results should be described in appropriate detail, be credible and consistent with the analysis. We encourage presentations of results in tables and figures and these should be consistent with, and referenced in, the text.
- The discussion should be clear and focused and the conclusions consistent with the findings. It should address items such as limitations of the study, anomalies in the findings, relationships to previous research, and theoretical implications.
- There should be a discussion about the practical applications of the review results for education practice and policy making and a set of key messages for either or both practitioners and policy makers as appropriate. We are particularly interested in what new knowledge is revealed by the review.
- Please also comment on whether the references should be appropriate, accurate and sufficient.
- Please also comment on whether the text shows consistency throughout in respect of referring to and discussing different aspects of the review subject.
- Please also comment on whether all acronyms are explained at first use and key terms included in a glossary.
With the above list in mind we ask that
- You make some general comments for the authors noting general strengths and limitations and then provide section-by-section comments.
- Where possible specify items you comment on by page, paragraph, and line.
- Note that we do not expect any line by line editing.
- Use the review as an opportunity to point out to the authors any important issues they may have missed.
Finally, please recommend one of the following options for the work you have reviewed:
- Publish as submitted
- Minor changes required
- Major changes required