Feb 25 2015 Evaluating systems change through complexity theories

“Systems change” and “systemic change” are terms that are now used ubiquitously across philanthropic, government, and non-profit organizations when discussing the ideal level of impact partners and funders want to make. These terms reflect an understanding that social sector work is based on a complex set of relationships, interactions, and power asymmetries that influence how information is shared and services are delivered in all sectors.

But how do we evaluate initiatives that aim to reshape the social and economic institutions that structure our everyday lives? How do we measure success in high-impact, complex collaborations? FSG, in a recently published brief titled Evaluating Complexity: Propositions for Improving Practice acknowledges that as management practices evolve, so to should the practices of evaluation and suggests methodologies for evaluators to consider.

There is increasing recognition that implementation research, which assesses practitioners’ fidelity to a program model and attempts to understand how the model affects outcomes, is suitable for some settings but not all. Some methodologies are perceived as mechanistic, deterministic, and linear and fail to capture the social dynamics that make impact evaluation possible at the systems level.

In accordance with the idea that systemic change is non-linear but rather adaptive and interactive, the authors attempt to bridge the theories of complexity with the practice of evaluating long-term, multi-stakeholder, multi-site initiatives. They distill nine essential principles of complex systems to guide evaluation design and implementation that emphasize adaptability and iteration throughout the data gathering process to reflect the true dynamism of the systems in which a program and its stakeholders are embedded. Some of the principles they highlight are:

• Complex systems are never static.
• Events in one part of the system have ripple effects in others.
• Context matters.
• Understanding relationships between entities is equally if not more important than understanding the entities themselves.

Each of the principles listed is accompanied by a proposition for evaluation and examples of research methodologies to follow. For understanding context and relationships, the authors suggest partners conduct comprehensive systems mapping exercises and social network analyses, updating them every few months to track change in relationships. They recommend evaluators strengthen feedback loops throughout the evaluation process to help improve effectiveness and capacity. Most importantly, they assert that evaluators who study complex systems change are “actively engaged in the process of change and learning, along with those who design, implement, and fund the systems change efforts.”