Introduction
The following information sets out how the two options’ proposals were assessed and scored, and the process used to do so. This was for the evaluation of the options as part of the pre-consultation work.
A summary of some of the key detail of the proposals is shared, alongside panel thoughts on that information, the weighted score for each key area (‘domain’) which took into account the relative importance of each of the sub-criteria, and the overall scores, which took into account the relative importance of each of key areas.
Overview
The two options we are consulting on for the future Principal Treatment Centre are Evelina London Children’s Hospital in Lambeth and St George’s Hospital in Tooting. Under both options, all radiotherapy services (instead of some, as now) would be provided at University College Hospital (part of University College London Hospitals NHS Foundation Trust) in central London.
Guy’s and St Thomas’ NHS Foundation Trust, which runs Evelina London, and St George’s University Hospitals NHS Foundation Trust, which runs St George’s Hospital, have each put in proposals to be the site of the future Principal Treatment Centre.
In these proposals, they set out responses to questions about their expertise, experience, skills and plans for the future Principal Treatment Centre.
Their responses to these questions were assessed and scored by four expert panels, one for each of four key areas for the future service. The proposals were tested across all four key areas, because the Programme Board overseeing this work had been told by stakeholders that this was a complex decision and needed to incorporate a number of factors. The evaluation considered the proposals from a range of perspectives to help us assess the two options in more detail before going out to consultation. The four areas were:
- clinical services
- patient and carer experience
- enabling (non-clinical) factors
- research
To undertake the assessment and scoring, the panels used evaluation criteria which had been developed with involvement from many different experts. For more information about the criteria, see our information on how the evaluation criteria were developed, and view the evaluation criteria here (PDF).
Who assessed and scored the options?
The assessment and scoring of the options was carried out by four expert panels, one for each domain, in December 2022.
The 10 members of the clinical panel included leading children’s cancer specialists from across the country, and senior doctors and nurses who worked in Brighton, London, South East region and further afield.
The 10 members of the patient and carer engagement panel included parents from London, Surrey and Sussex whose children had been treated at The Royal Marsden, representatives from national children’s cancer charities, and senior nurses from London, South East region and further afield.
The 10 members of the enabling panel included senior doctors who worked in London, Kent and Medway, and South East region, an expert in emergency preparedness, a human resources expert, senior managers from NHS England London and South East region, and independent clinical representatives.
The nine members of the research panel were leading children’s cancer doctors from across the country and the Netherlands, research leads from The Royal Marsden and the Institute of Cancer Research, and a representative of Association of British Pharmaceutical Industry.
Seven people were on two of the panels. No one was on more than two. Thirty-two different people in total participated in the assessment and scoring exercise.
The panels were provided with training on the sub-criteria and how to use the definitions provided under each sub-criteria and the scoring templates to assess the proposals.
Even with this, panel members considered and judged the evidence in different ways. A large, but manageable, number of people were involved in the evaluation which meant that a broad range of perspectives were brought into the evaluation and the risk of bias minimised.
How the scoring process worked
The different sub-criteria required different types of evidence. Some were purely quantitative (numbers-based), such as how much bed capacity would be provided and whether it would meet the demand for the service, what scores an organisation gained in various workforce surveys, or how their site would impact travel times.
Others were fully qualitative, that is, they depended on an explanation and written evidence of how each Trust would deliver specific aspects of the future Principal Treatment Centre’s responsibilities, such as leadership of networks.
Some were a mix. Quantitative information was provided, but panel members were also required to make a judgment, assessing the proposal against the defined criteria.
Where the criteria were qualitative, a framework was drawn up to help guide the panels on what sort of evidence to look for in allocating different scores. It also allowed both Trusts to know how the evidence they submitted would be considered.
Where sub-criteria were qualitative or partially quantitative, a 10 point scale was provided with marks set at 0,2,4,6,8 and 10. Each point in the scale reflected a level of confidence from none to ‘exceptional’ or ‘very strong’, with a definition of the sort of evidence that should match that level of confidence.
An appendix to the scoring framework was produced setting out the type of evidence expected for each sub-criterion at different scoring levels, as a means of increasing objectivity as far as possible.
In the actual scoring process, the median was used to further maximise objectivity. The median score removes scores at either end (highest and lowest) and instead gives a mid-point score from each set of panel members.
Putting the process into practice
- Once the proposals were submitted by the two Trusts, we undertook checks, to verify that all necessary information was included. Where necessary, we asked for clarification or follow ups from the Trusts within a set timeframe.
- We assessed certain aspects of the proposals which required pre-analysis, for instance, travel time analysis and vacancy rates, and summarised these for the panels.
- Panel members allocated scores for each of the sub-criteria within their allocated domain. They scored on their own, using the pre-agreed evaluation criteria as the basis for their marking and explaining their rationale with reference to the criteria.
- We collated all the initial scores for each panel and held a ‘verification session’ for each panel. At these sessions, each panel’s members’ scores (while being kept anonymous) were shared. Panel members could describe their considerations in reviewing the information provided which resulted in them scoring the way they did. This allowed each of the members to hear the approach taken by others in applying the scoring criteria, and to identify any clarifications they wanted about the proposals.
- This produced a number of questions for the two shortlisted Trusts to answer before panel members submitted their final scores. There was no requirement in these sessions to reach a consensus.
- Following the meeting, panel members were given the opportunity to revise their initial score, if they wanted to (there was no requirement to do so), based on new information from the panel meeting or clarification information. Six people in all chose to change their initial scoring.
- The scoring process produced a ‘median score’ from panel members for each sub-criterion.
- We took the median value for each sub-criterion and used the pre-agreed weights to calculate the final score for each proposal.
The four key areas
This section gives a brief overview of what the proposals said about the four key areas they were asked to cover, the differences between them, and what the evaluating panels found.
Because the proposals were from the Trusts which run the hospitals, the scores were recorded for Guy’s and St Thomas’ NHS Foundation Trust on behalf of Evelina London Children’s Hospital and St George’s University Hospitals NHS Foundation Trust on behalf of St George’s Hospital. We also refer to the Trusts as “Guy’s and St Thomas’” and “St George’s.”
Clinical
The table below shows what was looked at, the Trusts’ responses, and what the clinical panel said. The Clinical Advisory Panel (not one of the panels which undertook the assessment and scoring but a working of group of clinical experts who advised the programme) defined “readily available” as “available on-site within 30 minutes”.
- Specialist children’s services which the national service specification says, if not on the same site as the Principal Treatment Centre, must be readily available 24/7 for its patients. To ensure children have access to as many specialties as possible on the same site
| Evelina London | St George’s Hospital | What the panel said |
|---|---|---|
| Provides all but two of the listed services (neurosurgery and radiotherapy) | Provides all but three of the listed services (inpatient heart and kidney care, and radiotherapy). | The expert clinical panel scored Evelina London higher because it has more of the listed specialist children’s services on site than St George’s Hospital. |
In addition to the listed services, both Trusts provided credible and robust plans for delivery of:
- bone marrow transplants including JACIE accreditation (Europe’s official accreditation body for haematopoietic stem cell transplantation (HSCT) and cellular therapy)
- imaging and diagnostics
- pathology and haematology.
Neither organisation proposed providing radiotherapy on site; under our proposals all radiotherapy services for children with cancer (rather than some, as now) would be provided by University College London Hospitals NHS Foundation Trust.
- Reducing avoidable transfers of patients to other hospitals for care, particularly where a transfer would have an impact on patient experience and safety
| Evelina London & St George’s Hospital | What the panel said |
|---|---|
| Both organisations provided their assessment of the degree to which transfers would be reduced. As both Evelina London and St George’s Hospital have a paediatric intensive care unit on site this would end transfers from the specialist centre of very sick children who need or might need intensive care*. *Patient transfers from shared care units in local hospitals to the specialist children’s cancer centre, including emergency patient transfers, would not be affected by our proposals. By their nature, most shared care units are a long way from the specialist centre. They are all on sites which have beds where children can be closely monitored and given support. Children are only transferred for care at a level 3 children’s intensive care unit when it is unavoidable. | The expert clinical panel was confident that both options would reduce treatment transfers. They did not assess either option as being able to eliminate transfers completely because London has other centres for specific cancers and also other relevant expertise, so both options scored the same and neither was awarded top marks. |
- Network effectiveness: experience of providing leadership for and working with a network of other hospitals.
The future Principal Treatment Centre will lead the Children’s Cancer Network, which has an extremely important role in coordinating and developing care provided in shared care units to deliver the best quality care for children across the catchment area.
| Evelina London | St George’s Hospital | What the panel said |
|---|---|---|
| Evelina London runs four separate children’s networks (cleft, congenital heart disease, neonatal, and paediatrics) two of which are delivered across the same wide catchment area as the Principal Treatment Centre. It also provides the emergency retrieval service which takes very sick children to expert hospitals, including The Royal Marsden, and trains staff at hospitals across the catchment area in the care of very unwell children. | St George’s runs 16 adult clinical networks in south west London and hosts the London Kidney Network and the South West London and Surrey neuroscience network. It is a member of the existing children’s cancer network run by The Royal Marsden both as the co-provider of the current Principal Treatment Centre and because it runs a children’s cancer shared care unit. | Evelina London demonstrated greater experience in managing complex paediatric networks across the catchment area of the Principal Treatment Centre. St George’s provided evidence which focused primarily on adult networks, in south west London and Surrey. The expert clinical panel scored Evelina London higher. |
- Transition: supporting children to make the move to teenage and young adult cancer services when they are ready.
| Evelina London & St George’s Hospital | What the panel said |
|---|---|
| Both organisations gave detailed evidence of their expertise and experience in supporting young patients to move on to teenage and adult services. Both take into account best practice, service specification requirements, and National Institute of Health and Care Excellence (NICE) Quality Standards. Both use the Ready, Steady, Go model. | Evelina London’s model was assessed by the expert clinical panel as very strong – better than the service specification. It showed how the move to teenage and young adult services would work for children with cancer by giving a detailed explanation of how it works for Evelina London’s specialist regional renal service. St George’s Hospital’s model was assessed as strong – meeting the requirements of the service specification. St George’s existing links with The Royal Marsden, including joint clinics for children moving on to teenage and young adults services were recognised. The panel scored Evelina London higher. |
Clinical scores
For this key area, Evelina London scored higher overall.
The weighted scores were:
Guy’s and St Thomas’ NHS Foundation Trust: 29.63%.
St George’s University Hospitals NHS Foundation Trust: 27.01%
Patient and carer experience
Both proposals described how they would create a child-friendly cancer centre, support parents to navigate care for their children and get support at periods of extreme difficulty, and how they engage and collaborate with children, young people and families. Travel times by public transport and private vehicles were also analysed.
- Quality of facilities, with age-appropriate environments, education, play specialists to support children, maintenance of privacy and dignity, space for parents to stay with their child
| Evelina London | St George’s Hospital | What the panel said |
|---|---|---|
| Evelina London scored lower on this, reflecting that privacy and dignity maintenance was evidenced for some ages of patient but not all. | St George’s proposal for the future Principal Treatment Centre showed good consideration of how privacy and dignity would be maintained for all ages of patients. | The expert patient and carer engagement panel gave the same maximum points to both organisations on four aspects of quality of facilities key area: age appropriateness, education, play specialists, and support for parents to stay with their child. They scored St George’s Hospital higher on maintenance of privacy and dignity. |
Both proposals provided good evidence of their approach to age-appropriate environments, education (including hospital schools rated outstanding by Ofsted), play specialists, and space for parents to stay with their child for the future Principal Treatment Centre.
- Support for patients’ cancer care including at other hospitals
- Family support during periods of extreme difficulty
- Approach to engagement and collaboration with patients and families to improve services
| Evelina London & St George’s Hospital | What the panel said |
|---|---|
| Both organisations provided extensive evidence of their experience and expertise in these areas, including: – how they engaged with children, young people and families to develop their proposals and accolades for their approach to engagement and collaboration with young patients and families from different organisations – how they provide access to support services that families need during times of extreme difficulty, including support from clinical staff, counselling, signposting to charities and, where necessary, end of life support and bereavement services – how named care coordinators for every child at the future Principal Treatment Centre, supported by digital solutions (shared electronic patient records and apps for patients and families) would ensure patients get ‘joined up care’. | The expert patient and carer engagement panel awarded both proposals the same high scores. |
- Service accessibility: minimising the impact of relocation on accessibility by car and public transport, with a focus on those who are less able to choose flexible arrangements
| Evelina London & St George’s Hospital | What the panel said |
|---|---|
| This was scored by looking at travel times to both the potential options for the future Principal Treatment Centre and comparing them to travel times to The Royal Marsden for actual patients (using anonymised data) who used the service in 2019/20. The scoring put a particular focus on the impact on patients living in the areas categorised as being in the 20% most deprived in the country. Both options were faster for journeys by public transport than to The Royal Marsden but slower for journeys by road. *See table below This score was sensitivity tested, using a 30/70 weighting to public rather than car transport, with the original weight being 50/50. This did not impact on the overall outcome of the options appraisal for the two proposals. We subsequently carried out a further sensitivity test, assuming no families used public transport and 100% came by road. This too did not impact on the overall scores for the two proposals. | St George’s Hospital scored higher because, while travel times by public transport were better than to The Royal Marsden for both options, fewer patients would have car journeys at least 15 minutes longer than to The Royal Marsden if they were travelling to St George’s rather than to Evelina London. (49.5% of car journeys to St George’s and 71.1% of car journeys to Evelina London would be at least 15 minutes longer than to The Royal Marsden.) The score for this was not given by the expert patient and carer engagement panel but was worked out using the independent travel times analysis. |
*Table showing journey times via public transport and by road
| Longer journey than to The Royal Marsden (by more than 15 minutes) | Longer journey than to The Royal Marsden (by more than 15 minutes) | Faster or very similar journey than to The Royal Marsden (no more than 15 minutes longer) | Faster or very similar journey than to The Royal Marsden (no more than 15 minutes longer) | |
|---|---|---|---|---|
| Method of travel | Evelina London Children’s Hospital | St George’s Hospital | Evelina London Children’s Hospital | St George’s Hospital |
| Public transport | 6.7% | 4.6% | 93.3% | 95.4% |
| Car | 71.1% | 49.5% | 28.9% | 50.5% |
Patient and carer experience scores
For this domain, St George’s Hospital scored higher overall.
The weighted scores were:
Guy’s and St Thomas’ NHS Foundation Trust: 20.59%.
St George’s University Hospitals NHS Foundation Trust: 21.84%
Enabling (non-clinical) factors
The enabling section of the proposals covered capacity, pathways and processes; resilience including in an emergency; organisational support for staff, and the impact of the relocation on staff.
- Resilience: good plans for keeping services running smoothly, including in emergencies.
| Evelina London & St George’s Hospital | What the panel said |
|---|---|
| Both proposals set out a clear set of operating policies for business continuity. | The expert enabling factors panel said both organisations are equally capable of providing resilient services in an emergency. They gave them both the same score. |
- Capacity: enough workforce, space and equipment to provide speedy access for children from across the catchment area and offer bone marrow transplants.
Evelina London & St George’s Hospital | What the panel said |
|---|---|
| Both Evelina London and St George’s Hospital shared details of how they would provide the future Principal Treatment Centre. This information came in a capacity spreadsheet and was pre-assessed against the activity data supplied by The Royal Marsden in the ‘data lake’ created for this programme. Both options have subsequently formally confirmed they have the flexibility to provide the number of beds and isolation cubicles that could be needed to meet the future service development needs, including surges in demand. Final capacity designs will be developed and agreed with key stakeholders, after a decision has been made on the future location of the Principal Treatment Centre. | The expert enabling factors panel deemed they both had the required capacity for the future service and scored them equally. |
- Organisational support for staff: The ongoing support both Trusts provide to staff (to assess how attractive they might be to staff who could move to the future centre, if they want to, because more staff are likely to move to an organisation that is highly rated by its workforce.)
| Evelina London & St George’s Hospital | What the panel said |
|---|---|
| To give a rounded picture of this, the Trusts shared: – the results of their annual staff survey – their children’s services’ rates of staff turnover, vacancies, and sickness for 2021/22. | This score was calculated using a pre-agreed method, rather than given by the expert enabling panel. Evelina London scored higher because the children’s services specific data from Guy’s and St Thomas’ staff survey results were higher and its staff turnover was lower than the equivalent for St George’s. The panel acknowledged that St George’s vacancy rate was lower than Evelina London’s. |
- Impact on the staff who provide the current service: their future staff benefits such as childcare, and health and wellbeing support for staff, training opportunities for staff, and travel times. These were compared to what is currently available at The Royal Marsden.
| Evelina London | St George’s Hospital | What the panel said |
|---|---|---|
| Significant detail on future benefits and training was provided. Travel times for staff were also assessed using a pre-agreed methodology. Travel by public transport at 7am Monday: 39.3% of staff who work at St George’s and The Royal Marsden, and who would qualify to move to the future Principal Treatment Centre if they want to, would travel more than 15 minutes longer if the service moved to Evelina London than they do currently if they come to work by public transport. (We recognise that many staff, especially at The Royal Marsden, drive to work.) | Significant detail on future benefits and training was provided. Travel times for staff were also assessed using a pre-agreed methodology. Travel by public transport at 7am Monday: 34.5% of staff who work at The Royal Marsden, and who would qualify to move to the future Principal Treatment Centre if they want to, would travel more than 15 minutes longer if the service moved to St George’s Hospital than they do currently if they come to work by public transport. (We recognise that many staff, especially at The Royal Marsden, drive to work.) | This sub-criterion looked at different aspects of the potential impact on staff who could move to the future Principal Treatment Centre. St George’s Hospital scored higher because fewer staff would have longer journeys by public transport than now, and because of its training offer. The expert enabling panel scored both organisations the same on continuous professional development for staff but St George’s was assessed by the panel to offer an enhanced package of professional training. Other benefits, including childcare and staff wellbeing support, scored equally highly for both options. |
Enabling scores
The scores were very close. Evelina London scored higher due to better staff survey results. The sub-criteria for this, as for every key area, were weighted by the expert panel who then went on to do the assessment and scoring.
The weighted scores were:
Guy’s and St Thomas’ NHS Foundation Trust: 15.42%.
St George’s University Hospitals NHS Foundation Trust: 15.27%.
Research
The Royal Marsden, parents, charities, NHS England and the Trusts are all really clear that research is a vital aspect of the Principal Treatment Centre’s work.
- The Royal Marsden working with the Institute of Cancer Research is one of the largest centres in Europe for the delivery of new innovative therapies to children aged 15 and under, and young people aged 16-24.
- Parents and families said accessing cutting-edge research and drug trials was a key part of what they wanted from Principal Treatment Centre care.
- Increasing access to clinical trials is one of the goals of the new service specification.
Sustaining the world-class collaboration between the Principal Treatment Centre and the Institute of Cancer Research is very important for this consultation. A very close working relationship between dedicated ‘wet’ (experimental) laboratories and clinical care is crucial to rapidly progress new treatments for patients.
The evaluation criteria for research were that:
- All patients have the same access as now to clinical trials and research is supported through:
- Trusts’ current research performance and future vision.
- Their research workforce.
- Their current capacity and excellence including physical space for research.
- People, research workforce; staff development programmes; income supporting research staffing; research networks and collaboration; previous impact on collaborating to advance international health policy.
| Evelina London | St George’s Hospital |
|---|---|
| Guy’s and St Thomas’ proposal on behalf of Evelina London Children’s Hospital listed 28 nurses, 38 allied health professionals and research coordinators, and 19 doctors in Evelina London’s children’s research team. | St George’s proposal on behalf of St George’s Hospital listed 7.6 whole time equivalent children’s research nurses, and 13 doctors working across paediatrics and infectious diseases in St George’s Hospital’s children’s research team. |
| Guy’s and St Thomas’ operates as a single research delivery partner with King’s College London. Academic support for research staff at Evelina London includes King’s Doctoral Training Centre, National Institute of Health and Care Research (NIHR) Integrated Academic Trainee programme and King’s Clinical Academic Training Office with full university support for highly research-active NHS staff. The Faculty of Life Sciences and Medicine at King’s College has 482 researchers with 93% of research deemed either “world-leading” or “internationally excellent” in the 2021 Research Excellence Framework. | St George’s Hospital is co-located and collaborates closely with St George’s, University of London (SGUL). They have jointly set up the Translational and Clinical Research Institute (TACRI) which brings together clinicians from different fields and has an allocated annual budget for research skills development. Training programmes include the George’s Academic Training programme, and a dedicated NIHR Clinical Research Facility-backed training programme. Reference was made to the Research Excellence Framework (REF)[1]. |
| Guy’s and St Thomas NHS Foundation Trust, which runs Evelina London, attracted more than £25 million of funding for research staff in 2019/20. | St George’s University Hospitals NHS Foundation Trust, which runs St George’s Hospital, attracted £8.2 million of funding for research staff in 2019/20. |
| Guy’s and St Thomas’ with King’s College London have broad experimental medicine research strengths in paediatric specialties and also in adult cancer. Guy’s and St Thomas’ has a wide range of partnerships in place that would benefit children’s cancer researchers. They are with: – King’s College London (KCL) School of Cancer and Pharmaceutical Sciences with the Francis Crick Institute (including partnerships with University College London (UCL), and Barts Health NHS Trust on the £25 million Cancer Research UK (CRUK) City of London Major Centre, and £14 million CRUK RadNet City of London Radiation Research Unit) – Stem cell transplant research led by the team at King’s College Hospital, including access to the IMPACT UK-wide clinical trials partnership – KCL Institute of Cancer Policy, a key policy partner for many major national and international programmes including paediatric oncology – KCL School of Biomedical Engineering and Imaging Sciences, which hosts the London Institute for Healthcare Engineering, a world-class centre for imaging and data analytics research – Industry partners, notably the GSK-KCL Translational Oncology Research Hub to accelerate development of anti-cancer drugs, and strategic collaboration with UCB Biopharma in immunology, oncology, neuroscience and bone health. | St George’s regional clinical genetics service collaborates closely with The Royal Marsden and the Institute of Cancer Research on cancer gene discovery and early detection. One example is the CanGene-CanVar programme which aims to translate cancer genetics research into clinical practice, to improve early detection and prevention of cancer. This is part of an ongoing collaboration between St George’s and The Royal Marsden and the Institute of Cancer Research. Consultant paediatrician at St George’s, Professor Paul Heath, leads the UK Paediatric Vaccine Group through which all UK paediatric vaccine trials are undertaken. A recent initiative linked the St George’s Vaccine Group with Southampton oncology teams in a trial of pneumococcal vaccines in children with cancer. The Trust is partnering with Barts Health NHS Trust and Queen Mary University of London in a new Biomedical Research Centre (BRC) focused on precision medicine, including in cancer. The Centre of Global Health in St George’s University London’s Institute for Infection and Immunity (part of the joint Infection Clinical Academic Group with St George’s) provides a platform for collaborating with major international networks, with leadership in tuberculosis and fungal infection clinical trials. Several of St George’s Centre for Neonatal and Paediatric Infection researchers are members of key international organisations (World Health Organisation (WHO) antimicrobial and vaccine committees, the London School of Hygiene and Tropical Medicine Global Centre for Maternal, Adolescent, Child and Reproductive Health). |
| Evelina London’s largest international collaboration research initiatives from the last five years are: – trials by the children’s allergy team which have changed public health advice on peanut allergy in the UK and US – studies by the Evelina neonatal imaging centre that have advanced understanding of brain and heart function, especially in foetuses and neonates – EU-funded Autism Innovative Medicines Study 2 trial, the largest autism biomarker discovery project in the world. | Examples of St George’s international collaborations are: – work that contributed to the introduction of the WHO AWaRE index which is helping optimise use of antibiotics in children – exploring sudden arrhythmic death syndrome in people under 35 in the UK – work with clinical teams in sub-Saharan Africa that found two promising treatments for HIV-associated cryptococcal meningitis and led to a change in WHO guidelines |
[1] The REF: World University Rankings 2022 by subject: clinical and health | Times Higher Education (THE)
- Place: current capacity and excellence – physical space for research, including infrastructure to support and enhance transferring research teams, capacity for (phase I, II, and III research) trials and tissues studies, ability to link with industry; plans to improve existing provision and capacity to scale
| Evelina London | St George’s Hospital |
|---|---|
| Research would be an essential part of the work by the teams treating children with cancer, including on the cancer ward and in intensive care. Has two dedicated children’s research wards – Wolf and Seal – which are part of the hospital’s National Institute for Health and Care Research Clinical Research Facility. Has a research-intensive paediatric intensive care unit and a dedicated children’s imaging research unit with the only 7 Tesla MRI scanner in London. Researchers would have access to all the existing on-site infrastructure, sample storage, services, and office space as part of the main Evelina service. And access to Guy’s Cancer Centre for adults including the Experimental Cancer Medicine Centre, advanced therapies manufacturing facility and state-of-the-art biobanking. Pathology and research pharmacy facilities are on site. An aseptic pharmacy is to be built on site. | Research would be an essential part of the work by the teams treating children with cancer, including on the cancer ward and in intensive care. Existing research is undertaken on the children’s intensive care unit. Research facilities in the children’s cancer centre would include a laboratory and offices for staff from the Institute of Cancer Research and a clinical research unit with six clinical/ consulting rooms, sample storage, a laboratory and offices. Researchers would benefit from and be supported by the hospital’s wider research infrastructure including its National Institute for Health and Care Research Clinical Research Facility and established biobank at St George’s University London. Would have access to the Vaccine Institute, pathology and research pharmacy facilities onsite. |
- Capability and performance: assessed current research performance and capability, providers’ ambition and future vision for research and innovation.
Proven research capability should help mitigate against the risks involved in moving to a new delivery model during the transition period, as well as setting aspirations for the future.
| Evelina London | St George’s Hospital |
|---|---|
| Evelina London’s research staff have a portfolio of 180+ research studies with particular strengths in neurology, cardiology and critical care including research that has directly changed children’s care. Vision for future research: To create exceptional capabilities for immunological and advanced cellular research for children’s cancer care; develop a comprehensive clinical trial programme for children with cancer focusing on early-phase molecularly-targeted anticancer drugs; use advanced imaging research to develop and bring into clinical trials new molecularly targeted drugs; and develop wider research such as how best to manage symptoms, mental health impacts of cancer, research by nurses and allied health professionals (such as pharmacists and physios). | The Trust has a five-year research strategy to increase the numbers of staff leading and delivering high-quality research. Vision for future research: To help deliver, promote and spread children’s cancer research; boost existing work on personalised medicine and drive forward training and commitments on early phase and other trials; further develop research in areas such as antifungal diagnostics, convection-enhanced delivery treatment for brain tumours, optimising nutrition in children with cancer; build on links to the national Medical Research Council Centre for Medical Mycology to provide national and international opportunities for antifungal research. |
| What the panel said (covers all three Research sub-criteria): |
|---|
| The evaluation process recognised the research expertise of both organisations. The research evaluation panel was particularly positive about the opportunities that Evelina London offered for continuation and further development of children’s cancer research in partnership with the Institute of Cancer Research. The panel recognised clear evidence of significant relevant cancer research experience, outputs and impact via Guy’s and St Thomas’ and King’s Health Partners for adult and, to some extent, teenage and young adult cancers. Adult basic and clinical cancer research was noted as very strong in leukaemia, stem cell transplantation and immunotherapy. The panel appreciated the financial and other support in place for education, training and career development for research active professionals across disciplines at Guy’s and St Thomas’. The proposed approach to implementation and co-design of the future research model with researchers from The Royal Marsden was also appreciated. The panel scored Evelina London’s proposal as very strong on each of the three sub-criteria, people, places and performance, and St George’s bid as strong on each of the areas. It noted there would be a need for both Evelina London Children’s Hospital and St George’s to work hard to ensure that research grant funding is sustained at current levels and for the Institute of Cancer Research staff to feel truly integrated into the relocated Principal Treatment Centre and wider Trust, whichever site it went to. |
Research scores
Evelina London scored higher.
The weighted scores were:
Guy’s and St Thomas’ NHS Foundation Trust: 14.88%.
St George’s University Hospitals NHS Foundation Trust: 11.16%
Overall scores
Overall scores and scores by area for the two proposals are shown in the table below.
| Weighted score | Evelina London Children’s Hospital | St George’s |
|---|---|---|
| Overall Score | 80.505% | 75.267% |
| 1. Clinical | 29.626% | 27.005% |
| 2. Patient and Carer Experience | 20.586% | 21.840% |
| 3. Enabling | 15.419% | 15.266% |
| 4. Research | 14.875% | 11.156% |
The evaluation panels scored the proposal on behalf of Evelina London Children’s Hospital higher overall (80.51% as compared to 75.27% for St George’s Hospital) and for three of the four domains: clinical, research and enabling factors.
Evelina London scored higher on three of the four sub-criteria for the clinical domain and on all three sub-criteria for the research domain. St George’s Hospital scored higher on two of the five sub-criteria for patient and carer experience. Other scores were the same or very similar as set out in section above.
Final domain and total scores have been presented to 3 decimal places, however the underlying calculation used full values of 12 decimal places. This means that when manually adding up the domain scores to 3 decimal places, the total score for Guys and St Thomas’ is 80.506% rather than 80.505%. This is due to the method of calculation only. The method of calculation does not affect St George’s total score.
Scoring patient travel times
Parents who were part of the panel evaluating the patient and carer experience key area did not agree with the approach we took to the assessment and scoring of travel times for the options appraisal.
This approach was:
- an independent analyst mapped the journeys from the areas where actual children treated as inpatients at the current Principal Treatment Centre in 2019/20 lived (using anonymised data) by public transport and road to The Royal Marsden and to the two options, Evelina London and St George’s Hospital. This is a common way of identifying the impact of proposed moves of NHS services.
- the scoring assumed half the journeys were by public transport and half by road. This was to take account of the fact that not all families have cars so driving is not an option for everyone, that other Principal Treatment Centres work very well with no on-site parking, which could be expected to reduce the proportion of families who drive, and that it is our duty to work to reduce carbon emissions as part of the NHS’ journey to net zero.
- it also took particular account of the journey times for those patients who lived in areas categorised as being in the 20% most deprived in the country
- a calculation was then used to work out scores for the two options which reflected the potential changes in patient journey times, including for families who are likely to have fewer choices about how and when they travel.
However, parents and carers felt this was not the right approach, firstly because it used statistical data rather than drawing on their knowledge and experience of travelling with children who have cancer and, secondly, because they said we had assumed more people would travel by public transport than would in practice.
Seven parents (including two couples) from the Stakeholder Group, which advises this programme, wrote to us to raise their concerns about this, and about the impact that the move of the service to either option could have on access for families to the future service.
NHS England London took these concerns very seriously. We ran checks (called a sensitivity analysis) which assumed that more children came by car and fewer by public transport. First, we assumed 30% of families came by public transport and 70% drove (this reflects the level of car ownership in the population that the Principal Treatment Centre supports); subsequently we also looked at a more extreme scenario in which no families came by public transport.
When we calculated the new scores, this reduced the score for both options. This is because travelling by car to both Evelina London and St George’s Hospital would take longer for a lot of people than travelling to The Royal Marsden, unlike public transport. Journeys to either option by public transport are similar or faster than travelling to The Royal Marsden for almost everyone.
However, our sensitivity analysis showed that, even if we assumed all children came by car and none by public transport, although this changed the scores, it did not impact the overall outcome: Guy’s and St Thomas’ NHS Foundation Trust proposal on behalf of Evelina London Children’s Hospital still scored higher than St George’s University Hospital NHS Foundation Trust proposal on behalf of St George’s Hospital.
NHS England has also undertaken a range of work, including through its initial equality and health inequalities impact assessment (EHIA) to understand the impact of our proposals and also develop recommendations to help ensure there are good plans in place to help ensure that travel and access to the future Principal Treatment Centre is as easy as possible. Further information on this is included in our consultation document and extra information on travel.
Sensitivity analysis
A sensitivity analysis is a way of testing whether using different measures would impact the overall scores. In addition to the sensitivity test for transport, the other sensitivity tests that were applied were:
- each of the four key areas was weighted equally at 25% each instead of clinical services and patient and carer experience getting more of the marks and enabling factors and research getting less of the marks.
- the mean instead of the median was used to work out the scores (a different definition of the average which did not remove the impact of very high or very low scores as the median does).
In each case, Guy’s and St Thomas’ proposal on behalf of Evelina London Children’s Hospital still scored more than St George’s proposal on behalf of St George’s Hospital overall.
Preferred option
Based on the assessment by panels of experts, we have a better understanding of the strengths of both options. Both options scored highly in the pre-consultation evaluation, but Evelina London Children’s Hospital scored higher. On this basis, at this stage in the process, Evelina London is the site we prefer for the future Principal Treatment Centre.
However, we are open-minded about both options and open to any other evidence that the public may share. There may also be other solutions for our case for change which we haven’t identified. We will consider any viable alternatives that are suggested.
We will only make our decision on the location of the future centre after hearing the views that come forward during the public consultation and taking account of all relevant data, evidence, and other factors.