SYSTEMIC EVALUATION
Systemic evaluation recognised the importance of emergence – we cannot truly know what impact our actions have on the world. There are too many unintended consequences and costs about the choices we make about what or who, we include or exclude in our measurements. Systemic evaluation is developmental. It shifts the ground from thinking that we can measure the important things in a linear way, for example, assuming A only causes B, leading to outcome C and which means we can then claim D as our impact. We learn our way forward instead and make sense as we go in a thoughtful and considered way. Systemic evaluation approaches do this by collecting and analysing real time data that can inform decision making throughout the journey of a project. It combines accountability and rigor and supports projects to pivot where necessary. It is particularly suited to new innovations where the context is evolving in real time.
How viable this structure is, depends on the business it is in. Photo by Mark König on Unsplash
Example of Systemic Evaluation:
Systemic evaluation of capacity of a specialist advisory group in large public service organisation to provide strategic advise led to a two phased intervention. An assessment of current capacity indicated the need to reconfigure the governance structures supporting the function as well as raise capability of all stakeholders inputing at a strategic level. The second phase was to raise internal specialist knowledge internally and strenghten communication and coordination between internal and external specialists.
Developmental evaluation of a team within a multi-national organisation that was seen as not functioning effectively and whose role was increasingly regarded as peripheral to main function of the organisation. Historical relationship issues had led to a breakdown in communications and coping strategies that meant they worked as siloed individuals, which caused much confusion in roles. By working with the team to explore the multiple perspectives that led to this point, and revisiting their shared purpose, they were able to design a better functioning team and negotiate role boundaries. The result was a psychologically safer working environment and a shared purpose and respect that laide the ground for the development of an operational plan, that secured their place in the wider organisation.
Reports:
https://www.eurofound.europa.eu/publications/report/2021/disability-and-labour-market-integration-policy-trends-and-support-in-eu-member-states#tab-03
O’Donnell, J., Desmond, D., MacLachlan, M. (2022) Shaking up Services: Reframing Service Delivery as a Self-Organising Complex Adaptive System to Sustain Innovation, in Open Access Compendium: Assistive Technology, Accessibility and (e)Inclusion, AAATE-ICHHP 2022