Measuring Impact

Given resource constraints within most IAPT services, it is important that services prioritise targeted interventions and outreach for those in the local population who are not accessing IAPT services, or who have lower recovery rates compared to other parts of the population.

Once services have identified where there are inequalities of access or recovery within their populations, they can then design interventions and services to address those inequalities.

It is also important for services to be able to identify whether the intervention has been effective or not, to then decide whether to continue with it, amend it, or stop it.

The Advancing Mental Health Equality (AMHE) resource recommends 4 practical steps (outlined below) to measure impact and improve mental health equality in local areas.

  1. Identify inequalities and groups requiring more targeted interventions from existing data and resources and work with communities.
  2. Design interventions and new ways of doing things to meet the needs of these groups through research, formulation of plans and setting out key priorities.
  3. Deliver the intervention by first creating a strategy and then implementing the strategy.
  4. Evaluate the intervention by collecting data & measures, providing opportunities for feedback and reviewing data and feedback

Although IAPT services have and are delivering valuable interventions to different groups in the community, there are however very few examples we have found where services are measuring the impact of those interventions. It’s our aspiration that services will adopt the The Advancing Mental Health Equality (AMHE) resource recommendations to demonstrate the impact of interventions going forward.

We have provided some advice and examples to support services to do this.

1. Identify

Firstly, services must understand their baseline position, to be able to measure any changes against.

Understanding Common Mental Disorder prevalence and overview of IAPT data for London

  • In 2019/20, 3% of London population accessed IAPT services versus the expected London access rate (~2.7%). Overall, around one in five (19%) of adults surveyed in London met the criteria for a common mental disorder (CMD) in 2014 (APMS).
  • More females accessed IAPT services than males with average access rates of 1% and ~1.5% respectively. This may be partially explained by the difference in the prevalence of common mental disorders (CMDs) in men versus women – 13% v 20% (APMS 2014).
  • 21-30 year olds have the highest IAPT access rate across London (4.7%). This is in line with the APMS 2014 finding that working-age people were around twice as likely to have symptoms of CMD as those aged 65 and over.
  • Unemployed or economically inactive people are more likely to have a CMD than those in part- or full-time employment (APMS)
  • Although for men there is not much variation between CMD prevalence across ethnicities (~13%), for women non-White ethnicities have a higher CMD prevalence than White ethnicities (for example Black ethnicities have a prevalence of 29% compared to 21% for White British). In 2019/20, the average IAPT access rate across London was 9% and 2.2% in people identifying as ethnic minorities and White respectively.
  • As highlighted above, the Adult Psychiatric Morbidity Survey (APMS) – last carried out in 2014 – provides a good summary of levels of prevalence and treatment for psychiatric disorders in England, to compare local data against.
  • PHE Fingertips has recently created some guides on inequalities within Mental Health, which can be accessed here.
  • Services can also look at inequalities data for their own service, using the ‘inequalities view’ provided on PHE Fingertips to explore several potential areas of of inequality, such as gender, ethnicity, age, sexuality and disability. This would be a good first place for services to start looking at their data to see if there is inequality of access or recovery.
  • NHS Digital monthly data submissions on access and recovery rates, which each IAPT service should have access to.
  • The London Mental Health Dashboard has been developed by NHS Benchmarking which brings together London Mental Health data linked to key strategic priorities which can be used by the NHS for planning and identifying areas for quality improvement.
  • Joint Strategic Needs Assessments produced by local health and care partners also provide information on health inequalities and the wider determinants of health at local, borough-based levels.

Analysing data alone will not provide all the answers. Each IAPT service knows and understands its population and will know if there are particular ethnic groups that have a significant presence within that borough, but do not access IAPT services. Through speaking with members of these communities, and other local CVS, social and health care providers, IAPT services can better understand the reasons behind inequalities and then begin addressing these.

Examples

Newham IAPT looks at the ethnic profile of those accessing IAPT services every month, and it generally matches the local population (two thirds come from an ethnic minority background).

Lewisham

The ethnicity data captured by IAPT services are such broad groups, that additional local knowledge of ethnicities (e.g. Tamil) is required to identify where additional interventions may be required. In Lewisham the service is looking at specific data on IAPTus to drill down into e.g. ethnicity data to monitor changes and improvements. The service has a number of dashboards set up and having ready access to this data is very helpful when working with community groups, as it leads to more informed discussions. The service employs an Assistant Psychologist to look at this data, and it forms a significant part of their role.

2. Design

Please read The Advancing Mental Health Equality (AMHE) resource for further information.

Designing interventions with service users and community groups from the population group is a good principle for services to adopt.

Example:

Many examples are set out within our ‘Involving service users and carers’ and ‘Specific population groups’ sections.

3. Deliver

Please read The Advancing Mental Health Equality (AMHE) resource for further information.

Example:

Hillingdon

As part of Hillingdon’s integration agenda, they are working with Care Connect teams and Social Prescriber teams to develop Patient Activation Measures scoring to understand how people are managing their LTC conditions. Using this information and work with Social Prescribers to connect people into the IAPT service

Hillingdon’s work with LTC as a wave 1 – early implementer measured the savings made in offering joint talking health sessions, as this led to a reduction in visits to A&E and GP appointments. Further details to be found here.

4. Evaluate

Collecting and reviewing both qualitative and quantitative data on the effectiveness of the intervention will help the service evaluate its success, and whether changes should be made.

Examples:

City & Hackney IAPT – Due to the provision of targeted interventions for ethnic minority groups, access and recovery rates were reportedly higher in these groups than in the general population – (case study)

Slough IAPT:

Referrals into a well-established IAPT service in Slough did not reflect the diverse population. A project was set up in 2014 to increase ethnic minorities access and the profile of the service in the local community. The percentage of ethnic minorities referrals increased from 2014/15 to 2017/18 and key relationships were developed with GPs, stakeholders and service users. Recovery rate increased from 45% to 56% during the project – Case study

Bexley and Lewisham Older Adults IAPT – The Health Innovation Network (HIN) did a review of two ‘behavioural insights interventions’ to increase referrals of older adults to IAPT. The report can be found here. The two behavioural interventions were:

  1. A letter to GPs – informing practices about the referral rates to IAPT services, prevalence of common mental health issues amongst older adults, the benefits of referring to IAPT and how to make a referral.
  2. Patient prescription leaflets – providing information on older adults and common mental health problems, how IAPT services helped and how to self- refer to IAPT services. The leaflets were given to patients by GPs, practice nurses or social prescribers.

The conclusion is that neither of these interventions provided enough evidence to demonstrate that their use increased older adult referrals to IAPT. Key to the success of the interventions is the engagement of CCG commissioners, GPs and practice nurses. The availability of these health care professionals to participate in the tasks allocated was challenging throughout the project. CCGs, although committed to improving access to IAPT services, were also challenged with their time and commitment to the project.

Some services feel the IAPT service model doesn’t make it easy to measure the outcome / impact of changes to the service. It’s hard to consistently change IAPT outcomes over the longer term and make a sustainable impact. Very hard to evidence the change. Outcomes that they are trying to achieve through community outreach work may not be easily measured by changes in PHQ/GAD scores.