Best Evidence Medical and Health Professional Education

The literature search

Devise a strategy. The RG should devise a robust search strategy appropriate to the review topic/question. The literature searches should be sensitive (broad and inclusive) in design so that no evidence is overlooked. Access to the search processes should be readily available and easy to verify. They should be briefly set out in the protocol and given in detail as appendices in the draft and final review report.

Use alternate search resources. Searches should be run across the various databases, specialist registers and the web; hand-searching should be carried out across selected journals, the grey literature (e.g., conference proceedings, newsletters) should be consulted and decisions taken on the methods the RG plans to use to contact experts in the area (see BEME Guide on Systematic searching for evidence in medical education, Part 1 and Part 2).

Narrow the period search. The RG should determine the years to be covered by the search, depending on the topic area to be reviewed. Whilst a 10-year period is suggested by the BEME Collaboration, this may vary according to the number of studies in the topic area, the relevance of papers published outside the 10-year limit, etc. The RG must justify its decision in relation to the years to be covered. It is strongly advised that a cut-off date for studies to be considered in the review should be set. The time period covered should be clearly stated in the protocol and report.

The BEME Collaboration recommends that as far as is reasonably possible studies should not be excluded on the grounds of either geography or language. Access to and translation of studies not published in English is the responsibility of the RG and will depend on its resources.

The search process will involve determining the final individual search syntax for each source, executing the searches, and collating the results (removing duplicates, etc.)

Depending on the time taken to carry out the review, searches may have to be rerun to locate any newly published studies that may be relevant to the review.

Initial appraisal of literature search. The RG should assess citations generated by the search and decide, on the basis of the title/abstract, whether the citation is relevant to their topic. An evidence-based search will tend to have high sensitivity to reduce the possibility of missing evidence, therefore it will produce clearly irrelevant items that can easily be identified and eliminated. If there is any doubt as to the relevance, items should not be eliminated to ensure comprehensiveness. The full text article for all potentially relevant citations should be retrieved by the RG. If it is unclear whether a citation is relevant (e.g., if no abstract is provided, or if relevance is unclear from the abstract), the full text article must be retrieved to be properly assessed. Note that RGs do report instances of poor quality abstracts which make it difficult to determine the relevance of the study to the review topic/question.

Decide on coding. The RG should decide on its policy for review of citations and coding of articles. BEME recommends that there should be two-person agreement on each aspect coded. In the event of difference of opinion, a third person should decide. A record of differences of opinion should be kept. For this reason it is unlikely that one person alone can conduct a BEME review.

Decide on criteria and state in protocol. The final inclusion and exclusion criteria should be agreed by the RG and stated in the protocol. Revisions to this and any other changes to the planned work should be noted and if necessary a revised protocol submitted to ensure up-to-date details are posted online.