Evaluation

  • Clinical audit and quality improvement projects can be used to evaluate healthcare initiatives.
  • Evaluation of quality improvement interventions often requires a variety of methods, ranging from quantitative methods such as randomised controlled trials (RCTs) to quasi-experimental (controlled before-and-after and interrupted time series) and uncontrolled before-and-after studies.
  • Qualitative methods are often also used to understand how or why an intervention was successful and which components of a complex or multifaceted intervention were most effective.
  • Mixed methods designs such as action research or case study methods are widely used to design and evaluate improvement interventions.

A range of methods have been used to evaluate quality improvement interventions in primary care. These can vary in terms of the rigour of the methods used and their ability to attribute improvement to the intervention being proposed. Clinical audit and quality improvement projects can be used to evaluate improvement initiatives and should be within the scope of practitioners.

More rigorous studies require support from academic teams with expertise in these methods. Studies can range in design from RCTs, where attribution is clearer, to other types of experimental methods, including quasi-experimental designs such as non-randomised control group (sometimes called controlled before-and-after) or interrupted time series methods to uncontrolled before-and-after studies (including clinical audits), where attribution is less certain.

Improvement interventions are often complex (i.e. multiple rather than single) and pragmatic, so that ‘real-world’ designs are called for. Improvement often involves a series of interventions including education (of professionals and/or patients), reminders (to professionals and/or patients), audit and feedback or other measures. These vary in content, intensity or timing between different intervention sites so that it is not always clear which components of the ‘black box’ of the intervention are effective.

In order to understand how or why an intervention works it is often necessary to use methods such as surveys or qualitative interviews, focus groups, documentary (textual) analysis, observational or ethnographic methods. It may also be necessary to combine quantitative and qualitative methods (e.g. with case study methods), or to work with participants to design the evaluation (e.g. using action research methods). Quality improvement methods themselves can also be used to evaluate improvement, which adds to the complexities of improvement evaluations.