Impact Evaluation
Clients may be tempted to go for short-term solutions and then wonder why longer-term goals aren’t being met even though immediate delivery has been successful.
Sometimes change, especially of attitudes, practices, or ways of working, takes years to fully manifest itself.
We specialise in designing long-term measurement plans that define both the intended impact and the intermediate steps needed to get there. We use both qualitative and quantitative methods to determine whether impact has been achieved and whether the programme is likely to have contributed.
- We specialise in impact evaluations of programmes, measuring long-term changes in programme outcomes and impact. We use indicators and methodologies informed by industry best practice to best measure or proxy the desired change.
- We use experimental methods including randomised controlled trials (RCTs) and quasi-experimental methods using comparison groups to show not only whether impact was achieved but whether it was likely that the programme was responsible for the change.
- We use theory of change diagrams and narratives to lay out the intermediate steps needed to achieve impact, so that we can test success or failure at every step and also test assumptions and external dependencies that are needed to obtain goals.
Research design and scoping studies
Programmes are often designed without taking into account the existing available evidence and lessons learned previously or seeing whether existing approaches can be properly adapted to new contexts.
Clients may also jump in without seeing whether a new programme is likely to generate a positive return on investment, either in profit or in other benefits.
We draw on years of experience designing programmes, including programming in education, business support, and conflict and governance, to inform the design of new programmes and test whether they are appropriate.
- We research and develop evidence for Expressions of Interest, Requests for Proposals, and business cases for new programming to unlock funding and justify the amount being spent.
- We use specialist methods to appraise the value of programmes, including economic appraisals of programmes, value for money propositions and cost-benefit analysis of new programmes comparing different options.
- We also produce bespoke research, including lessons learned pieces, answers to specific questions, or full studies on research topics of interest at all phases of programming.
Evaluation of Training and capacity building
Too often, capacity building components do not comprehensively assess the effectiveness of training and the impact of learning on reaching objectives.
It results in inadequate learning tools and channels being selected, theoretical knowledge not getting translated into actual competencies, poor prioritisation of skills to be acquired depending on timing of programme delivery.
It can leave the staff helpless and discouraged, and impact workplan and transformative change.
Through our work in M&E and building on previous experience in HR, we have realised how important embedding a stringent assessment of capacity building progress into the overall evaluation plan is, and how much of a different approach is needed.
Cynosure has experience using quantitative and qualitative methods to assess the extent to which training and capacity building components have reached their targets in building competencies in targeted staff, which they can use to reach towards completing programme objectives.
Through case studies, interviews, data analysis, Organisational Design (OD) research and design of skills measurement plans, and using the principles of cognitive psychology and social science research, we offer clients tailored ways of analysing their performance in capacity building.
- We design and perform training needs analysis (TNA) and provide recommendations for learning programmes and optimal organisational design, in order to maximise the person/job fit within programme teams.
- We support clients from conceptualising the TNA framework all the way through selecting training providers and monitoring progress post-training, depending on how little or how much expertise they need.
- We design training material and manuals in M&E tailored to the audience, to make sure beneficiaries understand how to properly monitor their activities up to their outcomes, and understand the importance of collecting useful and accurate data to demonstrate their progress and keep their level of funding.