Value for money: 5 things to know about our approach
Assessing value for money is about how well resources are being used and to what extent the investment was worth the money. To put it more simply: are we getting bang for our buck?
Ensuring and demonstrating the value for money of public investments such as the Newton Fund and GCRF is extremely important. It’s also quite difficult when it comes to the complexities of research and innovation.
The Department for Business, Energy and Industrial Strategy, its funding partners and other stakeholders have been working together to find the most appropriate way to conceptualise and assess the value for money of the funds. Our approach essentially takes a sample of projects and programmes (multiple projects under a given theme) through a peer/expert assessment based on specific criteria.
Here are 5 things to know about it:
- Assessing value for money is hard
It’s hard to tell how much ‘benefit’ can be attributed to one project and this problem is compounded by the timeframes involved. Most research takes time. Some research takes a very long time. So impact, such that it is, may not be seen for years after the project has completed. Indeed, GCRF’s Theory of Change posits long-term impact at 10 to 15 years.
This situation is not unique to the Newton Fund and GCRF. Many other funds within research and innovation for development (and elsewhere) are grappling with the same challenges and have been for years.
Our approach recognises these challenges by bringing together both quantitative and qualitative evidence and to look at different facets of a project through a rubric – such as the impact already achieved and the likelihood of impact after the project has finished.
- We need to embrace risk
Researchers must be able to learn and build on the unsuccessful attempts of others. Supporting transformational research and innovation means taking risks, knowing that some attempts will fail and others may be highly successful. We see that today in the work underway across the world to develop and trial a vaccine for COVID-19.
Large research funds such as the Newton Fund and GCRF use a ‘portfolio approach’ to manage this risk. This means funding many projects to tackle one problem. So our work to assess value for money aggregates results accordingly. Instead of looking at one project and making a judgement, we look at ten or twenty. And in this way, we stand a better chance of making a proper assessment of what has been achieved overall.
- Every project is different
Every development project is different from others. Projects have different aims, methods, stakeholders, scale, budget and context. We must allow for comparability while accounting for such diversity.
Our value for money approach has a characterisation step. Projects are characterised based on their own aims to enable the most appropriate application of the rubric. For example, a health project that has capacity-strengthening as its aim would be assessed on that basis. A health project without capacity-strengthening as its aim wouldn’t be. It would be assessed on something else instead.
- It’s not all about outcomes
In international development, building equitable partnerships may help research to have greater impact but it’s also extremely valuable in its own right.
Say there are two research projects that achieve similar outcomes. Project A costs more because it uses resources to further equitable partnerships. Project B costs less but does nothing to further equal partnerships. Clearly project A is better in the sphere of international development, but a typical cost-benefit analysis would favour project B based on the numbers alone.
Therefore our approach accounts for ways of working (process) as well as outcomes – with equitable partnerships forming part of our assessment.
- Our approach is learning-oriented
Value for money assessments are crucial for ensuring the good use of public money. They generate important evidence on whether programmes are delivering what they set out to deliver. But they can also tell us things about how we measure value for money.
We are learning by doing through conducting four pilots of the approach throughout 2020. The richness of the data in the sampled project coupled with analysis from the peer/expert panels allow us to look for strengths and weakness in our assessments.
Our first pilot looked at 24 Newton Fund programmes and projects from 2018 worth £27 million. The external panel found that 88% were providing value for money – 2 of which scored as ‘excellent’ value for money. Projects scored highest on equitable partnerships, showing progress of delivering expected activities and outputs, and being relevant to partner country ODA priorities and wider ODA goals.
Does this suggest 3 projects in the pilot weren’t delivering value for money? Yes, and this is consistent with other international research funders looking at value for money at the portfolio level. It’s an unavoidable part of embracing risk. The important thing is to learn lessons from what goes well and what goes badly.
The panel provided really useful feedback on our approach, such as grouping projects for greater comparability, which we will take forward in our next pilot iteration.
Value for money assessments are never about adding up all the benefits and comparing that to the sum of the costs (if only!). We’ve had to take the time to consult and understand the complexities and challenges of demonstrating the value of research and innovation for development projects. In doing so we hope to yield more meaningful assessments and ultimately result in better information, transparency and decision-making for these types of projects in future.
By Heidi Peterson, senior monitoring, evaluation and learning manager for the Department for Business, Energy and Industrial Strategy’s ODA Research Management Team.