New Evaluation Practices to Foster Innovation for Inclusive Growth

Abstraksi

The Problem For economic growth to be inclusive, it must address the multidimensional nature of inequalities and their impact on different groups. Besides improving income and consumption, Indonesia needs policies that support the non-monetary wellbeing of social groups. These policies and their implementation should be rigorously evaluated using advanced methodologies to assess their impact on prosperity and its distribution. However, inclusive growth is the product of a complex system where the interdependence of economic and social policy, living standards, and structural transformation are not well understood. Traditional evaluations are poorly equipped to deal with these dynamics because they gather, analyze, and use data to formulate solutions using linear problem solving. By the time a summative evaluation has rendered a judgment, the economic system has changed, and the judgement is moot. The Solution & Practice Instead, the Government of Indonesia (GOI) and United States Agency for International Development (USAID) with its partner Social Impact, Inc. are investing in developmental evaluation (DE). DE is a relatively new approach to evaluation that differs from traditional evaluation by supporting innovation and learning, rather making a judgment of merit or worth (see table). In DE, an evaluator is embedded within a program or institution to contribute to modifications in its design. This enables real-time, evidence-based reflection and decision-making consistent with collaborating, learning, and adapting. The evaluator promotes advanced learning, as opposed to everyday learning and linear problem solving, by establishing learning loops (graphic below). Learning loops provide continuous feedback to partners to enable adaptive management with rapid course corrections that achieve the greatest impact for outcomes. From its embedded position, DE supports innovative programming by testing for intended and unintended effects and monitoring changes in context that challenge assumptions. DE is more suitable for supporting inclusive growth than traditional evaluation because it is designed for situations of high complexity like an evolving economic system with technological and demographic changes. Here creative thinking is necessary in addition to critical thinking. Indonesia needs both to test and nurture its strategies for job creation and increasing economic opportunities for vulnerable groups. DE can do this by testing these strategies’ assumptions, monitoring for unintended outcomes, and rapidly collecting feedback to advise on strategic course corrections. The Evidence GOI and USAID have successfully employed DE to assist a maternal and newborn health program to learn by building a feedback loop between it and thought leaders and policymakers. The DE has also introduced innovation to the program by redefining who it considers a stakeholder and how it engages them in partnerships. Similar in complexity to inclusive economic growth, reducing maternal mortality requires the commitment of actors in the government, private sector, academia, and civil society. With its focus on collaboration and learning, the DE worked with the program to establish and strengthen its advisory body of 30 thought leaders and policy makers from these sectors. Through novel approaches like social network analysis and classic methods like key informant interviews, the DE introduced measures to better resource the advisory body, build a consensus on how success will be measured, and connect thought leaders with local solutions developed in Indonesia’s provinces and districts. The advisory body is now seen as a potential model for a national commission on maternal health. In addition, the DE redefined what it means to be a stakeholder in reducing preventable maternal and newborn deaths to support broad stakeholder engagement. It took advantage of its embedded position within USAID’s flagship maternal and newborn health program to create a network map of 1,910 potential stakeholders. The DE then provided the program with innovative tools to analyze and engage these stakeholders, including online GIS maps and network maps using advanced software. By helping the program to engage the right mix of stakeholders in co-creation, the DE encouraged creative evidence-based solutions for maternal survival. The Lessons Learned & Implications Too often development practitioners and policymakers apply traditional formative or summative evaluations in situations of high complexity and unstable parameters. As a result, these evaluations reconfirm what experts and stakeholders already know, existing solutions are not working, and fail to propose meaningful alternatives. The GOI and its partners should consider DE to spur new thinking and approaches that can address complex issues like creating equitable employment opportunities. However, policymakers and practitioners should note that DE is not appropriate if stakeholders lack time or willingness to collaborate with evaluators, have limited capacity to adapt, or are averse to failure or negative findings. DE can also be expensive depending on the duration and scope of the evaluation. A DE may seem more complicated than a traditional evaluation, but the benefits reflect this intricacy. This new evaluation practice is more responsive and hence more effective at adapting programs to maximize intended outcomes in complex initiatives like those necessary to drive inclusive growth.