Forests and Sustainable Forest Management - OECD.org

Number 11 · April 2016

Evaluation Insights are informal working papers issued by the Network on Development Evaluation of the OECD Development Assistance Committee (DAC). The views and opinions expressed here are those of the author and do not necessarily reflect the official policy or position of the OECD DAC or its member countries. This paper is published under the responsibility of the Director of the Development Cooperation Directorate.

Forests and Sustainable Forest Management Evaluation evidence on addressing deforestation to reduce CO2 emissions

Learn more about the DAC Evaluation Network at:

Susanna Morrison-Métois, Hans Lundgren OECD DAC Network on Development Evaluation Secretariat

www.oecd.org/dac/evaluation

Comments and feedback on a previous draft of this paper were received from Lauren Kelly (IEG), Jeneen Rayes Garcia (GEF-IEO), and Balbir Singh (Norad).

Why forests? Consensus on the need for international co-operation to combat climate change has resulted in increased attention to the role of forests in storing carbon and the large quantity of C02 emissions that could be avoided if deforestation was halted. Deforestation and forest degradation are the second leading human cause of CO2 emissions contributing to global warming 1 according to the Intergovernmental Panel on Climate Change. It is estimated that deforestation and forest degradation account 2 for approximately 17 percent of global greenhouse gas (GHG) emissions. Furthermore, tropical forests capture and store carbon – since the turn of the century tropical forests are estimated to have removed 22-26% of all human caused carbon emissions. 3 Forests are also important storehouses of biodiversity and provide livelihoods for over a billion people worldwide including many living in extreme poverty.

Forests at the climate change development nexus Developing countries have an excellent opportunity to pursue low-carbon development strategies going forward. Many low and middle income countries are seeking to pursue the twin goals of development (poverty reduction and economic growth) and combatting climate change. Deforestation and degradation represent over one third of total emissions in developing countries, where many large tropical forests are found. The important role that forest-rich developing countries can play in combatting climate change by reducing emissions from deforestation and forest degradation has become central to international dialogues on preventing global temperature increases as a global public good. There are currently many new initiatives and programmes working at the forefront of the development/climate change nexus. As the number of policy and programme evaluations in this area rises, there is an opportunity to learn from existing evidence and emerging findings. Given the importance of the sector, it is surprising that there have been relatively few attempts to synthesise evidence from evaluations to learn lessons about the use of development assistance to combat deforestation.

The Sustainable Development Goals The Sustainable Development Goals underline the need to balance objectives and potential trade-offs between poverty reduction, growth and sustainability. Goal 15: “Sustainably manage forests, combat desertification, halt and reverse land degradation, halt biodiversity loss” and Goal 13: “Take urgent action to combat climate change and its impact” place forest management and sustainability into the international development framework and underscore the importance of these objectives in both developing and developed countries.

1

The UNFCCC & COP21 At the 2015 United Nations Climate Change Conference in Paris, 195 nations reached a climate agreement with the ambitious goal of pursuing efforts to limit global temperature increases to 1.5° C above pre-industrial levels. At this meeting the Parties to the United Nations Framework Convention on Climate Change (UNFCCC) negotiated the text of the agreement which explicitly acknowledges the key role of forests in combating climate change. At the same time, the governments of Germany, Norway and the United Kingdom pledged to provide US $ 5 billion (by 2020) in financial support for countries implementing Reducing Emissions from Deforestation and Degradation (REDD+) programmes and to scale up support for technical assistance and capacity building. To maintain the momentum from COP21, it is appropriate to take a closer look at the existing evidence on forest sector programmes in developing countries.

EMERGING EVALUATION EVIDENCE There is a growing body of evidence from recent evaluations conducted by the World Bank Group, the United Nations and OECD DAC countries’ development ministries and agencies on using ODA to incentivise reform. This paper aims to give insights into forest management and deforestation programming – it highlights findings from a number of recent evaluations and discusses some of the 4 various approaches and programmes. It aims to attract attention to the existing evidence base and to highlight areas that merit further analysis. The paper concludes with brief policy implications based on emerging evaluative evidence. Source: Susanna Morrison-Métois

COMMON EVALUATION FINDINGS This section highlights some common findings from recent evaluations of interventions in the forest sector, with respect to: 1) synergies and trade-offs between different goals; 2) co-ordination, alignment and leadership from partners and donors; 3) inclusive engagement of stakeholders and local ownership; and 4) specific findings on common programmatic approaches.

1. Trade-offs between climate change objectives and other goals International efforts to help developing countries decrease deforestation rates must balance carbon reduction and development goals and strive to formulate clear, coherent models of change. Evaluations of UN, World Bank Group and bilateral projects on deforestation and sustainable forest management often highlight the need to clearly articulate a vision of long-term progress (or theory of change) and to better define and measure the delicate balance between environmental, poverty reduction and 5 other social goals or objectives. Furthermore, programme managers, policy makers and evaluators must do more to capture and take into consideration ‘co-benefits’, which should be more explicitly defined in programme planning and policy. Several recent evaluations have underscored the need to better understand the potential trade-offs between climate objectives and 6 broader development benefits. A few evaluations have gone a step further and have questioned whether programmes which may have proven successful in preventing deforestation include and provide positive benefits for the poorest households, and whether these goals are fully 7 compatible. Several mention the growing agreement on the need to further encourage and promote sustainable livelihoods for those living in or near forest areas and to address land-use issues as a necessary step in reducing deforestation rates and CO2 8 emissions. Additionally, the drivers of deforestation are often factors outside of the forest sector and therefore programmes designed to halt or reduce deforestation must address a broader range of related issues, including: land tenure, agricultural policies, the potential for climate smart agricultural practices, alternative livelihoods, livestock and gazing practices, urban 9 expansion, mining policies, and other social and economic drivers of deforestation and change in land use. Encouragingly, there is evidence that programme managers and policy makers have been responsive to recommendations concerning the need to better clarify programme objectives and rationales. For example, the UN REDD Programme made 10 changes to its official strategy following the 2014 programme evaluation. The new REDD Programme strategic framework includes a clearly articulated theory of change, demonstrating that international programmes have been able to adapt and 11 incorporate learning from evaluative findings. The World Bank Group’s strategy in the forest sector has also evolved over 12 time, in part as a result of the Independent Evaluation Group’s (IEG) findings. Additionally, the Forest Carbon Partnership

2

Facility (FCPF) Readiness Fund first established an M&E framework following a recommendation made in the Facility‘s first 13 programme evaluation. Overall, policy and strategy has moved in the direction of more explicitly recognising the need to balance environmental, social and economic objectives and more clearly articulating theories of change. Greater attention is being paid to measuring intermediate outcomes and objectives that are expected to lead to longer term impacts. Policies and programmes in the sector have become more holistic in addressing a broader range of issues rather than maintaining a narrow focus on forest carbon. Notwithstanding, there have been some concerns raised in evaluations that despite increased recognition of various tensions and trade-offs, more still needs to be done to increase synergies between development and environmental goals.

2. Need for co-ordination, alignment and leadership from partners and donors The complexity of multilateral frameworks working to halt deforestation and the intricate international architecture of aid 14 delivery in the forest sector has featured in a number of evaluations and reviews. While some question the necessity of the complicated aid architecture in this sector, the more general finding is that the multiplicity of institutions and financing mechanisms requires greater levels of co-ordination among donors and partners. Several evaluations underscore the need for 15 co-operation among donors working at the country level and concerning decisions on financing for multilaterals. Overall, recent evaluations suggest that multilateral aid in the forest sector has shown mitigation effectiveness and is particularly important in helping to ensure that donors respect their environmental commitments. Pooled funding mechanisms are considered essential to gather momentum and ensure harmonisation of approaches between donors. There is also a need for collaboration between multilateral initiatives. The 2014 evaluation of the UN REDD programme, for example, recommended that UN agencies further their collaboration with the World Bank’s FCPF in order to harmonise approaches and to reduce the 16 duplication of effort. There have been some challenges with multilateral programmes and pooled funding, such as those seen with the UN REDD programme, which has had a relatively slow rate of implementation. These challenges seem to stem from unanticipated obstacles and capacity gaps that need to be addressed prior to full programme implementation. Recent evaluations highlight the complexity of initiatives in the sector, the slow pace of progress and the need for programmes to adopt a cross-sectorial approach. A global programme review of the FCPF conducted by IEG in 2012 summarised this view, stating that “the REDD+ readiness process is a more expensive, complex, and timeconsuming process than originally envisaged” and suggested that a cross17 sectorial approach would help increase effectiveness. Similarly the 2014 UN REDD Programme evaluation found that “UN partner agencies and participating countries should place greater emphasis on integrated cross18 sectorial approaches to REDD+…”. Not only is a cross-sectorial approach needed, but bilateral development providers also need to focus on environmental policy integration and the overall coherence of their international development co-operation portfolios. A recent evaluation by Belgium found that while many of their programmes had positive results in stabilising or increasing the land area covered by forests, greater attention could be paid to negative environmental externalities, such as the impact of other programmes on 19 forest coverage. Furthermore, a 2014 SDC evaluation, ‘Swiss International Co-operation in Climate Change 2000-2012’, noted that climate change mitigation requires long-term commitment and suggested that it be “mainstreamed as an aspect of all development projects… This approach would ensure that good progress in climate change mitigation or adaptation 20 21 is not undermined or ‘undone’ as a result of other interventions.”

3

Source: Susanna Morrison-Métois

How Brazil reduced deforestation – A case study Brazil is often considered the success story in efforts to halt deforestation. From 2000 to 2012 GHG emissions declined in Brazil by over 40%. This impressive reduction in GHG emissions happened at a time of economic growth [per capita income in Brazil increased by approximately 30% between 2000 and 2013 (OECD 2015)]. How did this happen? In 2004 Brazil’s government launched the Action Plan for Prevention and Control of Deforestation in the Amazonia Legal and has passed legislation such as the Forest Code, a key legal document, which requires landholders to set aside a share of their land for forest and soil conservation and restoration. In addition, the business sector has been brought on board and a number of large companies have made voluntary agreements to stop using soya beans that have been grown on illegally cleared land. What practices can other countries seek to replicate? What is unique about Brazil? Brazil’s protected areas system is one of the largest in the world and is organised under the National System of Protected Areas (SNUC). The SNUC requires public consultations with local communities, stakeholders and relevant sectorial institutions prior to the establishment of new protected areas (OECD, 2015).

Brazil has also implemented PES programmes and income support schemes designed to compensate communities for environmental conservation. Furthermore, Brazil is considered the world’s leader in monitoring deforestation via satellite imaging. New technology and almost real time information monitoring has helped improve targeting and enforcement. Civil society organisations in Brazil have also been actively engaged and influential in this sector. Finally, Brazil has benefited from international development co-operation funds from the World Bank Group, Norway, Germany, the UK, and other providers. The Amazon Fund, for example, was launched in 2008 and as of early 2015 had accumulated more than 970 million USD and supported over 70 projects (OECD, 2015). Significantly, Brazil has received around half of the total approved international finance from REDD+ (through the Amazon Fund) and Brazil was the first country to submit its forest reference emissions levels for payments under REDD+. A 2014 real-time evaluation of Norway’s International Climate and Forest Initiative emphasised how: “In Brazil, Norway’s US $ 1 billion provided validation of national reforms already underway to tackle deforestation, and probably gave strength to continue this route… the funds were a motivator which had a much deeper and potentially more sustainable change rather than an economic incentive, which was likely to have only a short-term impact and be potentially less sustainable.” (Norad, 2014, P. XXV) Despite Brazil’s impressive reduction over the last 10 years, there are concerns that these achievements may not be sustained, 21 suggesting that efforts to maintain low rates of deforestation must continue.

[Information in this section partially adapted from OECD Environmental Performance Reviews: Brazil 2015]

A few evaluations have expressed concern regarding the large number of partner countries now engaging in multilateral 22 programmes. In particular, there is concern that some partners interested in REDD+ readiness programmes may lack capacity or have unrealistic expectations; some evaluations question the appropriateness of engaging such a large number of new partner countries when many have faced challenges moving successfully into the results based payment phase. For instance, as of early 2016 there are 64 partner countries engaged in the UN-REDD Programme. Hence there are apprehensions, such as those highlighted in a 2014 ‘Real-time evaluation of Norway’s International Climate and Forest Initiative (NICFI)’, about raising the expectations of partners unrealistically when an international co-operation agreement on financing for REDD+ has so far not been put in place. The 2012 Global Review of the FCPF mentions the large number of interested client countries as a case of 23 ‘over-demand’. Serious discussions among donors, multilaterals and partner countries need to be had to assess if the current 4

approach of multilateral engagement with a large number of partner countries is advisable and realistic, including how these 24 programmes can best align with country generated strategies. Norway’s 2014 real-time evaluation of its forest sector programme found that strong bilateral relationships, such as between Norway and Brazil, can lead to successful outcomes; however there are concerns that this success may not be replicable with other partners. While evaluations have repeatedly highlighted Brazil as a key success story, it has often been noted that this 25 success has been possible due largely to the strong leadership and political will in Brazil to halt deforestation. Norway played a key role in supporting Brazil and the Amazon Fund with financing and strong partnership; although the most recent real-time evaluation of Norway’s large-scale commitments in the sector suggest that Norway’s bilateral relationships with other key partners have advanced less quickly. The uneven progress has been attributed to differences in local governance and varying levels of commitment and political decision-making within partner countries. Donors such as Norway have been successful in using large financing commitments to gather support for and to influence the international dialogue on approaches in the sector. Evaluations of Norway’s and Switzerland’s development co-operation in the forest sector both conclude that their countries’ aid has been crucial in advancing the dialogue and has influenced decisions and contributed to progress made on REDD+ and in the UNFCCC climate change negotiations. Generally, there is strong recognition of the need for co-ordination among donors, particularly in regards to financing for multilaterals and on thinking about ways to improve their transaction costs and to address the challenges caused by the slow implementation process, which has often led 26 to a lower level of disbursements than anticipated.

3. Local ownership and inclusive engagement of national and local stakeholders Recent evaluations suggest that more attention should be given to local ownership and ensuring inclusive dialogue with a wider range of stakeholders. Some evaluations, such as the 2014 UN REDD Programme evaluation, have recommended “a more bottom-up 27 approach”. Within this evaluation were some concrete recommendations on how to ensure greater partner country ownership including: 1) prioritising the use of national and regional experts 2) using existing structures and co-ordination mechanisms 3) strengthening national capacity in regards to financial management and accountability mechanisms, and 4) delegating decision making authority to the level closest to the field “while applying basic principles for 28 robust resource governance…”.

Challenges for putting safeguards ‘in practice’: Lessons from the UN-REDD Programme? “The UN-REDD Programme has been a strong voice in the prioritisation of free, prior and informed consent (FPIC) principles and the recognition of Indigenous Peoples’ rights in the implementation of UNFCCC safeguards, especially in decision-making fora. Programme contributions to social and environmental safeguards are helping to raise awareness at national and global levels, but it remains a challenge to put such principles into practice, notably due to the high number of safeguards and the lack of clear guidance on how to implement, monitor, and enforce these. “ – (UN-REDD Programme, 2014c)

Evaluations also conclude that local stakeholders, particularly forest-dwelling communities and those that depend on forest resources for their livelihoods could be better incorporated into local forest governance structures. While there has been growing recognition of the rights of indigenous groups, recent evaluations have highlighted the need to consider the 29 participation of non-indigenous local groups and the specific role of women as forest users and stakeholders. Programmes must seek to understand the ways that forest dwellers and local people traditionally use and manage forest resources and better understand the existing governance structures, local access issues, land tenure and tribal or local authority systems. These findings have led to a greater acknowledgment of the fact that success in halting deforestation may depend on the extent to which programmes incorporate understanding of these systems and practices into forest management programming. This requires more careful consideration and analysis of local contexts, including more extensive political economy analysis and the deliberate inclusion or design of mechanisms to ensure that local peoples’ needs and voices are fully incorporated into programming. While programmes in the sector have developed official policies of inclusiveness and increasingly work with more local stakeholders, some evaluations such as the 2014 UN REDD Programme evaluation have noted that substantial ownership and decision making powers regarding programming may be lagging. There is a need to move from the ‘recognition of local peoples rights’ to ensuring that in practice, local communities are fully involved in all stages of programming, starting from conception. Well-organised civil society groups have been relatively effective in influencing the dialogue and practice in many countries, however, it is likely that the needs and views of groups lacking more structured representation will need continued attention. At the same time, there is acknowledgement that the process of involving more stakeholders and the move towards greater inclusiveness comes with its own set of trade-offs and potential short-term efficiency losses. Repeatedly, success in reducing deforestation rates and progress within the REDD+ framework has been attributed to strong local political will in partner countries. Many stakeholders have clearly understood that future payments and distributions of benefits must be equitable and that there must be assurances that the benefits will accrue to local people, to avoid the risk of elite capture, if programmes are not well-managed. Overall, the trend has been towards recognition of the need to focus more 5

extensively on capacity building, readiness and ensuring inclusive local governance. Significant attention and funding must continue in these areas as potential achievements in halting deforestation and cutting CO2 emissions appear, unsurprisingly, to rely to a large degree on strong local political will and good governance.

4. Emerging specific findings on programme approaches This section looks briefly at what some recent evaluations have to tell us about the effectiveness of various approaches to programming in the forest sector, specifically looking at the available evidence on: 1) payments for environmental services; 2) protected areas; and 3) community / participatory forest management. Payments for Environmental Services A common approach in forest sector programming has been the use of Payments for Environmental Services (PES) which have frequently been implemented under the REDD+ umbrella. A 2014 systematic review of PES schemes drew evidence from 11 30 quantitative evaluations of PES programmes in four countries (Costa Rica, China, Mexico and Mozambique). It concluded that 31 PES schemes likely have a modest, positive effect on deforestation. However, there appears to be a lack of evidence that PES has beneficial effects on poverty, with evidence suggesting that PES schemes have been “less effective in poor areas and are less 32 likely to attract the participation of poor households than wealthier ones.” A 2013 IEG evaluation, for example, found that 33 “…running PES in the most cost-effective manner may conflict with other objectives such as poverty alleviation”. The evaluation found that there are “possible trade-offs between the effectiveness of particular payment arrangements and the 34 transaction costs of implementing them”. Furthermore, an IEG evaluation entitled ‘Climate Change Phase II: The Challenge of Low Carbon Development, Climate Change and the World Bank Group’ found that while PES schemes have been popular, “a substantial proportion of payments has gone to areas that are not at high risk for deforestation, diluting carbon and 35 environmental benefits and prompting attention to targeting.” While attention to targeting and poverty has increased in PES schemes, to date there does not appear to be a strong evidence base demonstrating that conservation and poverty-reduction 36 goals can effectively be complementary in PES programmes. That said, the existing evidence base in this area is weak and more 37 efforts should be made to assess the effects of PES programmes in order to potentially refute these initial findings. Protected Areas An approach which has been widely considered successful for the conservation 38 of forest resources is the creation of protected areas (PAs). Evaluations of protected areas agree that they are at least modestly effective in reducing deforestation and suggest that when local people and forest dwellers are part of the forest management process, protected areas are even more successful. For example, a global analysis conducted in 2010 by IEG found that “…[protected areas] are on average effective in reducing deforestation. Areas that allow sustainable use are more effective than strictly protected areas and 39 indigenous areas are most effective of all.” This finding was based on a study by Nelson and Chomitz (2009) that used spatial data on protected areas and 40 the location of forest fires, as a proxy indicator of deforestation. A 2013 IEG evaluation, ‘Managing Forest Resources for Sustainable Development’ confirmed that “the level of community participation in the management of 41 protected areas mattered both in terms of conservation and sustainability”. The evaluation warned against ‘’the siloed nature of many protected areas’’ as too often livelihoods are considered in the context of alternative income 42 schemes without addressing the root causes of deforestation. There have been some concerns that there is not enough evidence to date that protected areas specifically designed to address biodiversity are, in fact, effective at 43 achieving this objective. IEG has also pointed out that few projects integrate Source: Susanna Morrison-Métois 44 sensitivity to climate change into the monitoring and design of PA projects. A major recent (2015) impact evaluation by the Global Environment Facility which included the use of remote sensing to measure forest coverage, found that GEF supported protected areas have had positive impacts on biodiversity, yet recommended that more should be done to improve targeting, to address socio-economic conditions in local communities, to improve results 45 monitoring and to address large-scale drivers and governance issues. The evaluation recommended more investment in understanding what works and why, including looking further at how to address local livelihood needs. Participatory Forest Management / Community Forest Management Participatory or Community forest management (PFM or CFM), as it is often called, has been a frequent approach in the sector. The FAO defines participatory forestry as “the processes and mechanisms which enable people with a direct stake in forest 46 resources to be part of decision-making in all aspects of forest management, including policy formulation processes”. While PFM/CFM is a common approach, the evidence base on its results and impacts is mixed and highly context specific. A 2015 evaluative study conducted by theBank Natural Environment Research Council entitled ‘Effectiveness of Community Forest Curt Carnemark, 1993 / World Management at reducing deforestation in Madagascar’ found that Community Forest Management, as broadly defined, 6

47

encompasses various practices and approaches and “is not a guarantee of forest conservation”. Significantly the study found that community forest management which allowed commercial use did not reduce deforestation, while the areas that prohibited commercial forest use did manage to reduce deforestation. It also concluded that “differentiating among types of 48 community forestry is important in impact evaluation”. A 2010 review of the evidence base for community forest management by The Scientific and Technical Advisory Panel (GEF) found that “there is insufficient evidence to form any conclusions about the 49 effect of CFM on local livelihoods, which is in part due to the absence of common indicators of CFM effects on livelihoods”. Furthermore the study was unable to clearly determine the impacts of CFM as there was often no clear pattern. This study highlighted the challenge of comparing and synthesising findings between programmes due to differences in indicators and methodologies used, they noted that “of great concern for the studies reviewed is the inability of most studies to credibly 50 eliminate rival explanations of any changes, or absence of changes, in the outcome variables.” Comparably, IEG’s 2013 evaluation found examples of success and highlighted best practices in countries such as Mexico. The World Bank supported Mexico’s Community Forestry Program; the initial pilot programme has been institutionalized and is now implemented nationwide. The evaluation also contains examples from World Bank supported PFM programmes in India, Lao 51 PDR and Liberia. Moreover, the evaluation outlined some of the successes of Participatory Forest Management, finding evidence of positive outcomes: “Participatory Forestry Management projects exhibit the most balanced goals as compared to other interventions in the sector. World Bank support for participatory forestry management has yielded positive livelihood benefits, such as the generation of employment, increased incomes, and diversification of revenue streams. Where specifically targeted, these projects have also achieved positive environmental outcomes such as reduced deforestation rates, regeneration of degraded forests, reduced incidence of fires, and protection of biodiversity.” (IEG, 2013b p. XIII) This evaluation highlighted that while the World Bank successfully worked with the formal sector through programmes such as concession regimes and the creation of formal conservation areas, it ought to do more to work with actors in the informal sector 52 upon which many forest-dwelling communities are dependent. Hence, while there is still limited evaluation evidence on the overall effectiveness of community forest management (likely due to the lack of common indicators and the context specific nature of CFM/PFM), there is evidence from the World Bank Group’s portfolio that participatory forest management can be successful in achieving the triple goals of poverty reduction, growth, and sustainability.

WHAT WE NEED TO KNOW MORE ABOUT While there is a growing body of evidence on what approaches work best in the forest sector, there are a large number of remaining unknowns and some obvious outstanding evidence gaps. Four areas where strong evaluative evidence and/or international consensus is clearly lacking are: 1) the appropriateness and effectiveness of results based payments; 2) challenges and issues around measuring progress, intermediate outcomes, and longer term results; 3) the cost effectiveness of multilateral initiatives and country specific cost effectiveness for engagement; and 4) the role and potential of the private sector and forest concessions in conservation and sustainable forest management. The most obvious evidence gap is related to the lack of consensus around the use of results-based payments. There are a number of questions that have not been answered related to results-based payments, questions such as: How well do they work? Do they provide the right incentives to the right people? What quantity of CO2 emissions is actually avoided? There have been a number of debates about how to ensure accurate baseline data and whether measurement and verification systems can be robust and precise enough to ensure that payments are made based on accurately measured results. Furthermore, there are still many who question the payment for results approach, who argue that forests and other natural resources have intrinsic value, hence paying local people or developing countries to protect forests risks not only undermining the intrinsic value of these resources, but also shifts the responsibility for protection from a local responsibility to a potentially less sustainable programme approach dependent on financing from developed nations. Another area that emerges as a continuous challenge is issues relating to the measurement of progress. Climate change and environmental degradation are long-term processes that often follow non-linear trajectories which pose a number of problems for monitoring and evaluation. For one, it has often proven challenging to define significant outcome indicators, as change in this sector is often measured by proxy indicators and over short time horizons. Secondly, programmes and approaches such as REDD+ readiness programmes have invested considerable time and money on intermediate steps (capacity building, partnerships, local governance issues, etc.). At times, the theories of change relating these intermediate steps to long-term outcomes have not been clearly articulated and appropriate indicators to measure intermediate outcomes have hence been neglected. As more time has been spent on preparedness and intermediate steps than originally anticipated at the onset, it has proven both more important and more challenging to evaluate the effectiveness of these intermediate steps. While there are many sources of hard data or scientific evidence that allow us to measure CO2 emissions or to visualise the area of forest coverage, there are fewer internationally agreed upon indicators and standards for measuring the extent of relevant 7

53

international partnerships and capacity building processes. The challenges of monitoring progress underscore the importance of considering evaluability in the initial stages of programme design. Likewise, more attention ought to be given to incorporating 54 rigorous impact evaluation methods and/or quasi experimental design approaches into programming. This would help better ensure that programmes are achieving their stated objectives and are fully accountable. The need to pay more attention to measuring and evaluating the cost effectiveness of programmes is related in many ways to 55 the slower than anticipated implementation of multilateral initiatives, such as REDD+. In order to ensure the cost effectiveness of REDD+, it is important to continue to improve cost analyses and models both for multilaterals and for partner countries. Country-driven strategies and increased political economy analysis may help to better anticipate and address countries' specific advantages and challenges earlier in the process. As the drivers of deforestation are often country specific, donors and multilateral will need to work with partners to support context specific approaches, while also working to ensure cost effectiveness and achievability. Another large evidence gap is related to the role and impact of the private sector in sustainable forest management. More research and evaluation work ought to be done to better determine the scale, positive and negative consequences/outcomes, and potential future benefits of increased private sector engagement. There are concerns, expressed in some evaluations, that some forest or biodiversity conservation approaches may unintentionally curtail private sector development. Better country level political economy analysis of forest use should be undertaken to gain a more complete understanding of the utilisation of forest resources in the informal sector, the extent of illegal forest activities, and the current level of private sector interest and opportunities. While illegal logging has rightly been recognised as a crucial problem Source: Yosef Hadar/ World Bank and is frequently addressed through programmes focused on improving governance, the full scale and potential of forest resources deserves further attention from evaluators. Initiatives led by the private sector and driven by consumer demand (such as commitments to deforestation-free supply chains and commodities) have also shown high potential. Increased involvement and engagement with the private sector is inevitable for those seeking to achieve long-term forest conservation goals, hence more evaluation studies should be commissioned in this area. In particular, the potential to leverage additional financing from private sector actors and the repartition of benefits at the local level deserve further attention from evaluators. That said, efforts by development actors to better understand and promote private sector approaches are likely to be contingent upon the development of appropriate legal frameworks (and institutions for their enforcement) which are capable of adequately protecting local peoples’ rights and interests.

IMPLICATIONS FOR POLICY MAKERS Evaluation findings in the forest sector to date suggest that: 1) Deforestation and forest management programmes must work from the conception stage with a wide range of local stakeholders to design programmes and approaches ‘from the bottom up’. Evidence suggests that inclusive processes are essential to address and overcome local obstacles and barriers to behaviour change and to bridge capacity gaps while working to improve local governance. Inclusive processes must be ensured from the onset in order to better anticipate challenges and to build consensus at an early stage. While funding and co-ordination between donors needs to take place at a global level, there is growing evidence to suggest that inclusive, country-driven approaches are needed.   

Programmes need to ensure the inclusive participation of civil society groups, local peoples, local government authorities, women, the rural poor and traditionally marginalised groups in decision making processes. Financial partners should focus on designing flexible programmes, promoting decentralised decision making and supporting greater country ownership so that programmes are fit to context. Programme managers and policy makers should ensure that M&E systems are designed from the onset that include indicators on inclusiveness, local ownership and substantive stakeholder involvement.

2) Bilateral development co-operation providers should continue to invest in pooled financing mechanisms and multilateral programmes (such as the UN REDD Programme, GEF and World Bank initiatives), but should remain engaged in order to ensure that these mechanisms are inclusive, pro-poor and efficient. Financial partners have a role to play in influencing the future development of these initiatives. Focus should be given to leveraging the comparative advantages of each partner/programme while pursuing a harmonised policy approach in relevant sectors. Multilateral programmes working on REDD+ readiness programmes have proven to be adaptable and have led to successes in assisting countries achieve emission 8

reductions through the provision of technical services and knowledge sharing. However, to date these programmes have not 56 made as much progress as originally expected in piloting performance-based systems of payments.  

Policy makers should ensure that reporting on the application of safeguards is thorough and consistent. As evidence seems to suggest that political will is a necessary precondition for successful programming, policy makers and bilateral development providers may need to consider that countries which currently lack strong political will to halt deforestation may require different modalities of engagement over longer timeframes in order to gradually increase awareness and local political buy-in, to ensure local ownership of programmes and to foster the emergence of country specific approaches. At the same time, financial partners need to work with multilaterals to ensure that programmes with partner countries remain focused and based on realistic expectations.

3) Bilateral providers and multilaterals have room to improve programme monitoring and results frameworks and can also work to improve communication on programme achievements. These frameworks must be designed to measure both climate change (CO2, biodiversity) and socio-economic goals. More efforts and resources need to be dedicated to monitoring and communicating results, including communications targeted towards general audiences. There is also room for increased learning which could come from more systematic reviews and more evaluations looking at the coherence of bi-lateral providers’ overall portfolios for achieving climate change goals. Policy makers should increase the level of focus on evidence and on the communication of evaluation findings. 4) There is a strong opportunity for innovation and pilot programmes. Policy makers should continue or increase financing for innovative pilot programmes and projects and should accept high risk tolerance levels, generating evidence on effective programmes. Innovative pilot programmes such as those using new technologies and real-time monitoring will likely need increased public sector funding, to develop the tools and methods needed to help developing countries implement their Nationally Determined Contributions.

CONCLUSION

Deforestation and climate change are global challenges and the international community must develop more inclusive partnerships, institutions and financing mechanisms to work with partner countries, local communities and local peoples to address them. There has been growing pressure for visible results and rapid successes, while progress on programmes such as REDD+ and readiness initiatives have proven to take time and persistence to build strong coalitions, change attitudes and behaviours and gain political and local buy-in. Evidence from a range of evaluations in the forest sector suggest that success can only be achieved when local ownership is at the heart of programme design and implementation, when countries and local communities are empowered and motivated to find workable solutions and alternatives to deforestation. Therefore, the international community, including bilateral donors, multilateral funds and institutions, and partners in developing countries, must work together where there is political will to find country level solutions. There is reason to be optimistic as current programmes are well positioned and capable of adapting while moving forward on climate change goals and commitments.

Source: Susanna Morrison-Métois 9

SOURCES Baastel and ECO Consult (on behalf of FCPF) (2013), Readiness Fund: Monitoring and Evaluation Framework for the FCPF, Forest Carbon Partnership Facility. Baastel and Nordic Agency for Development and Ecology (commissioned by FCPF) (2011), First Program Evaluation for the Forest Carbon Partnership Facility. Nelson, A., Chomitz, K. (2009), Protected Area Effectiveness in Reducing Tropical Deforestation A Global Analysis of the Impact of Protection Status, Evaluation Brief 7, Independent Evaluation Group, Washington, D.C. FAO (2015a), Community Based Forestry, www.fao.org/forestry/participatory/en/ (accessed 8 November 2015). FAO (2015b), Strengthening Forest Resources Management and Enhancing its Contribution to Sustainable Development, Land use and Livelihoods Project-GCP/GLO/194/MUL: Final Evaluation Report, Food and Agriculture Organization of the United Nations, Rome. FAO and JRC (2012), Global forest land-use change 1990–2005, by E.J. Lindquist, R. D’Annunzio, A. Gerrand, K. MacDicken, F. Achard, R. Beuchle, A. Brink, H.D. Eva, P. Mayaux, J. San-Miguel-Ayanz & H-J. Stibig, FAO Forestry Paper No. 169, Food and Agriculture Organization of the United Nations and European Commission Joint Research Centre, Rome. GEF (2015), Impact Evaluation of GEF Support to Protected Areas and Protected Area Systems, The Global Environmental Facility, Washington, D.C. GEF (2012), Evaluation of the GEF Focal Area Strategies, Technical Paper 6: SFM/REDD+, Global Environmental Facility, Washington, D.C. GEF (2006), The Role of Local Benefits in Global Enviornmental Programs, Global Envionrnmental Facility, Washington, D.C. Gelo D., Koch S. and Muchapondwa E. (2013), “Do the poor benefit from devolution policies? Evidence from quantile treatment effect evaluation of joint forest management,” Economic Research Southern Africa (ESRA) Working paper 400. Goodman Rosa C. and Herold Martin (2014), "Why Maintaining Tropical Forests Is Essential and Urgent for a Stable Climate," CGD Working Paper 385, Washington, DC: Center for Global Development. http://www.cgdev.org/publication/why-maintainingtropical-forests-essential-and-urgent-stable-climate-working-paper-385 Hardcastle P, Davenport D, Cowling P, Watson C (2011), Discussion of Effectiveness of Multilateral REDD+ Initiatives, IDL Group for REDD+ Partnership. http://www.fao.org/partnerships/redd-plus-partnership/291490db58e819221ae34c06b8182087d74669.pdf Hardcastle P, Forbes A, Karani I, Tuominen K, Sandom J, Murtland R, Müller-Plantenberg V & Davenport D (2010), Evaluation of Finnish Support to Forestry and Biological Resources, Evaluation report 2010:5/I Main report, Ministry for Foreign Affairs of Finland, Kopijyvä Oy, Jyväskylä, 174 p. ISBN 978-951-724-876-1 (printed). ICF International (2014), Independent Evaluation of the Climate Investment Funds, World Bank, Washington, D.C. IEG (2015), Project Performance Assessment Report, Brazil: First Programmatic Reform Loan for Environmental Sustainability (LOAN NO. 7256-BR), Independent Evaluation Group, Washington, DC. IEG (2014), Project Performance Assessment Report, Brazil: Ecosystem Restoration of Riparian Forests in Sao Paulo, Independent Evaluation Group, Washington, D.C. IEG (2013a), Brazil Country Program Evaluation FY 2004–11, Independent Evaluation Group, Washington, DC: World Bank. License: Creative Commons Attribution CC BY 3.0. IEG (2013b), Managing Forest Resources for Sustainable Development: An Evaluation of World Bank Group Experience, Independent Evaluation Group, Washington, D.C. IEG (2012), The Forest Carbon Partnership Facility, Global Program Review Vol. 6, Issue 3, Independent Evaluation Group, Washington, D.C.

10

IEG (2010), Phase II: The Challenge of low-carbon development, Climate Change and the World Bank Group, Independent Evaluation Group, Washington, D.C. IPCC (2007), Climate Change 2007: Synthesis Report. Contribution of Working Groups I, II and III to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change [Core Writing Team, Pachauri, R.K and Reisinger, A. (eds.)]. IPCC, Geneva, Switzerland, 104 pp. JICA (2015), JICA Annual Evaluation Report 2014, Evaluation Department Japan International Cooperation Agency, Tokyo. Joppa, L. Pfaff, A. (2011) Global Protected area impacts, Proceeding of the Royal Society of London: Biological Sciences, vol. 278(1712), pp 1633-1638. Karousakis, K. (2009), "Promoting Biodiversity Co-Benefits in REDD", OECD Environment Working Papers, No. 11, OECD Publishing, © OECD. doi:10.1787/220188577008 Kissinger, G., M. Herold, V. De Sy (2012), Drivers of Deforestation and Forest Degradation: A Synthesis Report for REDD Policymakers, Lexeme Consulting, Vancouver. Kronenberg, J., and K. Hubacek. (2013), Could payments for ecosystem services create an "ecosystem service curse"?, Ecology and Society 18(1): 10. http://dx.doi.org/10.5751/ES-05240-180110 Lele, Uma, Zazueta, Aaron and Singer, Benjamin (2011), "The Environment and Global Governance: Can the Global Community Rise to the Challenge?," Climate Change and Land Policies, eds. Ingram, Gregory K. and Yu-Hung Hong., Lincoln Institute of Land Policy, Cambridge, MA. Lund JF, Balooni K, Casse T, (2009), Change We can Believe in? Reviewing Studies on the Conservation Impact of Popular Participation in Forest Management, Conservat Soc [serial online] vol. 7 pp. 71-82. Miteva, D. A., et al. (2012), "Evaluation of biodiversity policy instruments: what works and what doesn’t?," Oxford Review of Economic Policy 28(1): 69-92. Norad (2014), Real-Time Evaluation of Norway’s International Climate and Forest Initiative Synthesising Report 2007-2013, Norwegian Agency for Development Cooperation, Oslo. Norad (2012), Real-Time Evaluation of Norway’s International Climate and Forest Initiative: Lessons Learned from Support to Civil Society Organisations, Norwegian Agency for Development Cooperation, Olso. Norad (2011), Real-Time Evaluation of Norway’s International Climate and Forest Initiative Contributions to National REDD+ Processes 2007-2010 Country Report: Brazil, Norwegian Agency for Development Cooperation, Olso. Norad (2010), Real-Time Evaluation of Norway’s International Climate Change and Forest Initiative, Norwegian Agency for Development Cooperation, Oslo. Nelson A. and K. Chomitz (2009), “Protected Area Effectiveness in Reducing Tropical Deforestation: A Global Analysis of the Impact of Protection Status,” Evaluation Brief 7, Independent Evaluation Group, Washington, D.C. OECD (2015), OECD Environmental http://dx.doi.org/10.1787/9789264240094-en

Performance

Reviews:

Brazil

2015,

OECD

Publishing,

Paris.

OECD (2013), Scaling-up Finance Mechanisms for Biodiversity, OECD Publising, Paris DOI:10.1787/9789264193833-en OECD (2010), Paying for Biodoversoty: Enhanching the cost-effectiveness of payments for ecosystem services, OECD Publishing, Paris. DOI:10.1787/9789264090279-en Office of the Special Evaluator for Development Cooperation/SEO (2014), How green is our Development Aid? Thematic evaluation of the Belgian Cooperation in the Field of Environment, Federal Public Service Foreign Affairs, Foreign Trade and Development Cooperation, Brussels. Pattanayak S. K., Wunder S & Ferraro P. J. (2010), Show me the Money: Do Payments Supply Enviornmental Services in Developping Countries, Review of Environmental Economics and Policy, Vol. 4 (2), pp 254-274. Porter-Bolland, L., et al, (2011) Community managed forests and forest protected areas: An assessment of their conservation effectiveness across the tropics, Forest Ecology and Management, doi:10.1016/j.foreco.2011.05.034 11

Rasolofosan R. A., P.J. Ferraro, C.N. Jenkins & Julia P.G. Jones (2014), Effectiveness of Community Forest Management at reducing deforestation in Madagascar, Biological Conservation, Vol. 184, pp. 477-458. Samii C, Lisiecki M, Kulkarni P, Paler L, Chavis L (2014), Effects of Payment for Environmental Services (PES) on Deforestation and Poverty in Low and Middle Income Countries: A Systematic Review, Campell Systematic Reviews. Scientific and Technical Advisory Panel of GEF (2010), The Evidence Base for Community Forest Management as a Mechanism for Supplying Global Environmental Benefits and Improving Local Welfare, Scientific and Technical Advisory Panel of the Global Environmental Facility, Washington, D.C. SDC (2014), Reports of Effectiveness 2014: Swiss International Cooperation in Climate Change 2000-2012, Swiss Agency for Development and Cooperation, Bern. Spracklen, D. V., and L. Garcia-Carreras (2015), The impact of Amazonian deforestation on Amazon basin rainfall, Geophys. Res. Lett., 42, 9546–9552. UN-REDD Programme/ FAO, UNDP, UNEP (2015a), UN-REDD Programme Strategic Framework 2016-20 (Revised Draft-7 May 2015), UN-REDD Programme Fouthteenth Board Meeting, Washington, D.C. http://www.unredd.net/index.php?view=document&alias=14096-un-redd-pb14-2015-strategicframework&category_slug=session-3-strategic-and-policy-issues&layout=default&option=com_docman&Itemid=134 (accessed 12 October 2015). UN-REDD Programme/ FAO, UNDP, UNEP (2014b), Consolidated Management Response to the Programme Evaluation, UN-REDD Programme.http://www.unredd.net/index.php?view=list&slug=un-redd-programme-evaluation3266&option=com_docman&Itemid=134 (accessed 12 October 2015). UN-REDD Programme/ FAO, UNDP, UNEP (2014c), External Evaluation of the United Nations Collaborative Programme on Reducing Emissions from Deforestation and Forest Degradation in Developing Countries (the UN-REDD Programme), Volume I – Final Report, UN-REDD Programme. USAID (2014), Measuring Impact: Lessons Learned from the Forest, Climate and Communities Alliance, United States Agency for International Devleopment, Washington, D.C. World Bank Group (2004), Sustaining Forests: A Development Strategy, The World Bank Group, Washington, D.C. http://wwwwds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2004/07/28/000009486_20040728090355/Rendered/PDF/29 7040v.1.pdf

CITATIONS AND RESEARCH NOTES 1

(IPCC, 2007) (IPCC, 2007) 3 (Goodman and Herold, 2014) 4 This Insights draws mainly from recent evaluations of programmes by major actors in the sector. 5 [(Baastel and ECO Consult (on behalf of FCPF 2013), FAO 2015b, GEF 2012, ICF 2014, IEG 2013b, Miteva, D. A., et al. 2012, Norad 2014, UN REDD programme 2014c] 6 [(Baastel and Nordic Agency for Development and Ecology (commissioned by FCPF) 2011, GEF 2015, ICF 2014, IEG 2015b, IEG 2013b] 7 (FAO 2015b, GEF 2006, Gelo et al. 2013, Hardcastle 2010, IEG 2013a, IEG, 2013b, Samii 2014) 8 [(Baastel and Nordic Agency for Development and Ecology (commissioned by FCPF) 2011, GEF 2006, ICF 2014, IEG 2013b, IEG 2011, Norad 2014, Norad 2012] 9 (GEF 2015, Hardcastle 2011, ICF 2014, IEG 2013b, IEG 2011, Norad 2014, Norad 2012, UN REDD Programe 2014c) 10 (UN-REDD Programme/ FAO, UNDP, UNEP, 2014b) 11 (UN-REDD Programme/ FAO, UNDP, UNEP, 2015a) 12 (IEG, 2013b) 13 [(Baastel and ECO Consult (on behalf of FCPF),2013] 14 (CIF 2014, IEG 2013b, IEG 2011, Lele 2011, Norad 2014, Norad 2012, UN REDD Programme 2014c) 15 [(Baastel and Nordic Agency for Development and Ecology (commissioned by FCPF) 2011, GEF 2015, Hardcastle 2010, ICF 2014, IEG 2013b, IEG 2011, Norad 2014, Norad 2012, UN REDD programme 2014c] 16 (IEG 2012, UN-REDD Programme/ FAO, UNDP, UNEP, 2014b) 2

12

17

(IEG, 2011) (Norad 2014, UN-REDD Programme/ FAO, UNDP, UNEP, 2014b) 19 (Office of the Special Evaluator for Development Cooperation/SEO, 2014) 20 (SDC, 2014) 21 See for instance: (Spracklen, D. V., and L. Garcia-Carreras, 2015) 22 (Norad 2014, Norad 2012, IEG 2011) 23 (IEG, 2011) 24 (Norad, 2014) 25 (Norad 2014, Norad 2012, IEG 2011) 26 See for example (Norad, 2014) 27 (UN-REDD Programme/ FAO, UNDP, UNEP, 2014b) 28 (UN-REDD Programme/ FAO, UNDP, UNEP, 2014b) 29 (Norad, 2014) 30 (Samii et al., 2014) 31 (Samii et al., 2014) 32 (Samii et al., 2014) 33 (IEG, 2013b) 34 (IEG, 2013b) 35 (IEG, 2010) 36 (Kronenberg and Hubacek 2013) 37 (Pattanayak, 2010) 38 (Miteva, D. A., et al, 2012) 39 (IEG, 2010) 40 The spatial data on the location of forest fires and protected areas was used as a proxy indicator of deforestation, while controlling for confounding influences. 41 (IEG, 2013b) 42 The 2013 evaluation included a portfolio review of protected area projects implemented by the World Bank. (IEG, 2013b) 43 For instance, the 2013 IEG evaluation found that the indicators being used to measure biodiversity were process indicators related to the management of PAs (as a proxy for biodiversity outcomes) with a lack of data related to the protection of flora and fauna and that more needed to be done to understand the impact of PAs at the species level. (IEG, 2013b) 44 (IEG, 2013b) 45 (GEF, 2015) Notably, this evaluation also analysed changes in submitted ‘Management Effectiveness Tracking Tool (MEET)’ scores over time, which “ are considered to be a proxy for a PA’s potential to deliver desired conservation outcomes”. 46 (FAO, 2015a) 47 (Rasolofosan et al., 2014) 48 (Rasolofosan et al., 2014) 49 (Scientific and Technical Advisory Panel (STAP) GEF, 2010) 50 (Scientific and Technical Advisory Panel (STAP) GEF, 2010) 51 (IEG, 2013b) 52 (IEG, 2013b) 53 One recommendation made in the GEF evaluation (2015) to allow for results to be monitored consistently within a country is to establish long-term partnerships for biodiversity and socio-economic monitoring with country institutions that already have this as their mandate. 54 (Miteva, D. A., et al, 2012) 55 (Norad, 2014) 56 (FCPF, 2011) 18

13

Number 11 · April 2016

Evaluation Insights: Forests and Sustainable Forest Management

Evaluation Insights are informal working papers issued by the Network on Development Evaluation of the OECD Development Assistance Committee (DAC). These notes highlight emerging findings and policy messages from evaluations and share insights into the policy and practice of development evaluation.

Further reading Real-Time Evaluation of Norway’s International Climate and Forest Initiative. Synthesising Report 2007-2013 Norway’s International Climate and Forest Initiative (NICFI) was launched in 2007 with the aim of reducing greenhouse gas emissions resulting from deforestation and forest degradation in developing countries (REDD+). More than NOK 10 billion has been disbursed since 2008 to support such efforts both internationally and at the country level.

Managing Forest Resources for Sustainable Development: An Evaluation of World Bank Group Experience The World Bank has played a major role in shaping the dialogue on the role of forests and climate change and the development of forest carbon instruments. IEG’s review of the Bank managed Forest Carbon Partnership Facility found that it is providing an effective and highly inclusive platform for defining the modalities of REDD+. But it has also raised client expectations that will only be met if financial flows follow. IEG suggested the need for a high level internal dialogue about how the Bank will integrate countries’ REDD strategies into overall development planning.

These and many other development evaluation reports are available from DEReC: the Development Evaluation Resource Centre: www.oecd.org/derec

Further OECD publications

14