Following the requirement for transparency of process, systematic reviews use a set of explicit selection criteria, to assess the relevance of each study found to see if it actually does address the review question(s).  Key issues to address:

Detailed decisions are recorded specifying precisely the basis on which information sources have been included and excluded.  The aim of being explicit about the selection criteria is to make the reviewer’s decisions available for scrutiny and evaluation.  Selection criteria also facilitate the updating of your review. 

In medical science the inclusion and exclusion criteria are usually specified in terms of the population, intervention, principal outcomes, and study design.  We have previously introduced the CIMO framework in the section on formulate review questions.  In management and organisation studies the criteria may be delimited by 

• context (e.g. only studies conducted in the public sector; only studies of women business leaders) 
• intervention (e.g. only studies that address the leadership development programmes)
• mechanisms (e.g.only studies that address the relationship between networking and innovation)
• outcomes (e.g. studies that measure organisational performance)
• methods (e.g. only case studies; only surveys)

As noted previously, the selection criteria will incorporate multiple concepts, constructs and perspectives.  It is therefore essential to define key terms during the question formulation phase.  In the example outlined above, it would be essential to define precisely what is meant by the term such as leadership development programme or organisational performance? 

Many researchers find difficulty in specifying selection criteria a piori. However, when asked whether or not a certain text is relevant to the study they can make that judgement based on implicit criteria.  The aim of this exercise is to make explicit your implicit criteria for selecting studies.

1. run a pilot search in a database as explained in the section protocol driven search 
2. read the title and abstract of ALL the articles of the first 10 articles in the results.
3. make a note of which ones appear relevant to addressing your review questions
4. try to articulate why these papers are relevant to your study
5. make a note of which papers are not relevant to your review
6. try to articulate why these papers are not relevant to your study 
7. continue onto this process until you exhaust all possible reasons for inclusion and exclusion
8. review all the reasons for inclusion and exclusion - look for commonalities
9. turn the reasons for inclusion and exclusion into selection criteria
10. add the form to your protocol / plan

Stop and reflect before moving on to the next 10 articles.  Are you still identifying new criteria for inclusion and exclusion? Only continue until you exhaust all possible reasons for inclusion and exclusion. 

 [To download and use this document - click on the word document here

The text on this page was created by Dr David Denyer, Management Practice Fellow, Advanced Institute of Management Research (AIM), and Senior Lecturer in Organization Studies, Cranfield School of Management, david.denyer@cranfield,ac.uk