Tips for measurement and evaluation

It is important to understand why you are undertaking any measurement and evaluation and develop your approach accordingly.

Not only can measurement and evaluation be undertaken at a number of different levels (individual patient; team or service; local population), but it can also have a number of objectives – to understand where improvements can be made (insight data), to understand how improvements can be made (improvement data) and to understand impact (outcome data).  

However, it is often true that measurement and evaluation has more than one objective, for example, if you are measuring how effective an existing self-management support programme is, the measures you use to assess its success should align with the outcomes you originally identified.  This can be used to demonstrate the programme’s success to commissioners and other stakeholders.  However, it may be that your thinking about what the programme should achieve has moved on – e.g. a previous focus on ‘improved clinical outcomes’ might be expanded to ‘improved clinical outcomes that enable people to undertake the activities that are important to them’.  The measurement and evaluation process is an opportunity to generate data that will help you refine what you are doing, so you should not always be constrained by only measuring the programme and its activities as they were originally conceived.

Some of the particular challenges of measuring and evaluating self-management support include:

  • As self-management support is about enabling people to live well with their condition(s), they have the best understanding of the changes to how services are provided that are most likely to support them.  This means that effective measures and processes for measurement and evaluation need to be co-designed with people with long term conditions.  Remember that co-design is different from user consultation or testing.
  • Effective self-management support involves implementing a number of activities that – such as providing training to patients, and adapting systems, for example so people receive their test results ahead of their appointment.  Each of these activities will contribute to people’s outcomes, making it hard to disaggregate the individual impact of each activity.
  • Some changes happen sequentially – for example, use of health services will not reduce without improved disease control, and this in turn is dependent on people improving their health-related behaviour.  The timeframe for measuring different outcomes therefore needs to match the timeframe within which it is realistic for those outcomes to be achieved. 
  • Longer term evaluation might be needed – for example, there is strong evidence that people’s skills and confidence to self-manage decline over time – so following up with them six to twelve months after they have completed a self-management training programme, may provide greater insight into the longer term impact of your work and how to maintain skills and motivation over time.
     

Learning from the Health Foundation’s Co-Creating Health programme


The Health Foundation’s Co-creating Health programme explored how self-management support could be effectively implemented in services.  The programme included both external evaluation and measurement and evaluation by the teams and services taking part.  Services involved highlighted the following tips for measuring and evaluating self-management support within a service:
 

  • Only collect the data that you need. There is often a temptation to gather additional data that the team think will be interesting or informative, but which do not relate directly to why you are undertaking the evaluation. This will make the data collection more laborious and can make it harder to analyse the findings.
  • Where possible, draw on existing evidence – including national and international evidence – and use your local evaluation to explore local factors.
  • Be realistic about what you can do and by when. It is better to focus on evaluating or gathering evidence around a few elements of your programme and do these well, than try to cover everything.
  • Ensure that part of your evaluation includes regularly seeking the views of people with long term conditions who are participating in different aspects of your work to develop self-management support, such as those taking part in the Patient Skills Programme.
  • Be clear about how you will use the results. This will help people to feel more confident about sharing information.
  • If your evaluation is designed to help the implementation of your self-management support programme, ensure you provide regular feedback on your findings and that these are used to shape and develop the programme.
  • Ensure you provide regular feedback to those involved, so that they see the value of collecting data, completing feedback questionnaires etc.
  • Different audiences are likely to be interested in different aspects of the evaluation and to prefer the findings in different formats – for some it may be a presentation, for others a brief summary, for others a detailed report.


All resource on self-management support