Increasing Effectiveness in Sustainability Research: Comparative Analysis


Scientific research is done to generate new knowledge. Research projects and programs also strive for excellence by means of impact. New transdisciplinary approaches are emerging that place research around the interests of stakeholders to produce more applicable and relevant knowledge. However, as these approaches are new and because they engage in complex systems, it has been difficult to know what works, how, why, and what opportunities exist to enhance the effectiveness of the research. Comparative analyses of applied research project case studies will generate empirical evidence and theoretical insights about whether and how transdisciplinary research contributes to learning, change and, ultimately, impact.

Through comparative analysis, we will synthesize what elements of project design and implementation are critical for enhancing research effectiveness and achieving outcomes.

Purpose: To contribute to improved and more effective inter- and transdisciplinary research.

Guiding question: How can we improve research effectiveness?

Method: This project examines and compares the components of project design and implementation with outcomes and score them for their effectiveness based on the Transdisciplinary Research Quality Assessment Framework, in order to analyze and compare the design, implementation, and outcomes of a series of applied research projects.

Phase 1. Evaluating policy-relevant research with a Theory of Change 

Cases analyzed: Sustainable Wetlands Adaptation and Mitigation Program, Furniture Value Chains, Global Comparative Study REDD+, and Sustainable Forest Management in the Congo Basin

Results: This theory-based approach for research evaluation provides a valuable means of assessing research effectiveness (summative value) and supporting learning and adaptation (formative value) at the project or program scale. The approach is well suited to research-for-development projects represented by the case studies, but is also applicable to any research that aspires to have a social impact. However, the retrospective theory of change development proved challenging and resulted in simplistic theories of change. More work is needed to draw on theories of knowledge translation and policy processes to develop and further test more sophisticated and complex-aware theories of change.

Read the full peer-review publication: Belcher, B., Suryadarma, D., and Halimanjaya, A. (2017) Evaluating Policy-Relevant Research: Lessons from a Series of Theory-Based Outcomes Assessments. Palgrave Communications 3: 1-16.

Phase 2. Assessing the effectiveness of research projects with theory of change and the Transdisciplinary Quality Assessment Framework (in progress).

Cases analyzed: Sustainability of Timber Harvesting in Brazil Nut Concessions, Political Economy of Fire and Haze, Poverty and Environment Network, Integrated Management of Non-Timber Forest Products for the Improvement of Smallholder Livelihoods, and Collaborative Research Initiatives to Create Networking and Social Learning Strategies in Biosphere Reserves.