January 31, 2003

The IEO provides objective and independent evaluation on issues related to the IMF. The Office operates independently of IMF management and at arm's length from the IMF's Executive Board.

Evaluation of Poverty Reduction Strategy Papers and the Poverty Reduction and Growth Facility

Issues Paper/Terms of Reference

This document is available in French (352 kb PDF file), Portuguese (349 kb PDF file), and Spanish (363 kb PDF file) - [Disclaimer: Reasonable efforts to ensure accuracy of these documents have been made; however, only the English version produced by the IEO is the authentic version]

Full text in PDF Format (English) (348Kb) 

I. Background Information

II. Terms of Reference of the Evaluation

III. Envisaged Methodological Approach

Bibliography

List of Abbreviations

Evaluation Of PRSPs and the PRGF

Issues Paper/Terms of Reference

In late 1999, to enhance the contribution of their interventions to international poverty reduction efforts, the International Monetary Fund (IMF) and the World Bank (WB) adopted a new strategy for their assistance to low-income countries. The main planks of this strategy were twofold: (i) both institutions were to base their concessional lending and debt relief to low-income countries on Poverty Reduction Strategy Papers (PRSPs) prepared by the countries themselves; and (ii) IMF concessional lending was to be provided through a revised lending facility, the Poverty Reduction and Growth Facility (PRGF)--with a stronger poverty reduction focus.

Of the 77 countries that are eligible for the PRGF, 28 have produced interim PRSPs and another 23 have produced a full PRSP. Almost all of the latter countries are currently engaged in PRGF-supported programs.¹ There should now be sufficient evidence on how the approach is working in practice for an independent evaluation to be undertaken, so as to allow at a still relatively early stage for any adaptations of the approach that might appear desirable. In view of the relevance of the PRSP for the World Bank and of the joint IMF-WB nature of the many components of the process, the evaluation undertaken by the IMF's Independent Evaluation Office (IEO) will be conducted in parallel with and complemented by an evaluation of the PRSP process by the Operations Evaluation Department (OED) of the World Bank.²

The purpose of this paper is to lay out the main issues to be addressed by the evaluation. Section 1 provides some background information on the objectives of the PRGF and PRSP initiatives, the main findings of internal reviews undertaken to date, and key criticisms formulated by external stakeholders. Section 2 presents the main questions to be addressed by the evaluation. Section 3 discusses the methodological approaches envisaged. The latter two sections constitute the terms of reference for the evaluation.

A draft of this issues paper was circulated earlier to seek inputs from all interested parties on the questions to be addressed and the methodology to be followed. Comments or contributions on the substance of the issues covered are now invited from all interested parties and may be submitted at any time by e-mail addressed to ieo@imf.org.

I. BACKGROUND INFORMATION

Main objectives of the PRSP and PRGF initiatives

From 1987 to 1999, the bulk of IMF concessional lending was provided under the Enhanced Structural Adjustment Facility (ESAF). Internal and external evaluations conducted in 1997 and 1998, respectively, identified a number of problems that hindered the effectiveness of programs supported by this facility.³ In particular, the external evaluation highlighted the following as major problem areas: (i) lack of national ownership; (ii) weaknesses in the analytical and empirical bases of the social policy content of programs; and (iii) insufficient attention to tradeoffs involving policy choices that imply significantly different paths for growth and social welfare.

Initially, after considering the lessons from the two reviews, the Executive Board endorsed several recommendations of IMF staff and the external evaluators aimed at strengthening the effectiveness of ESAF-supported programs. Subsequently, it became clear that bolder and broader steps were needed, in particular to ensure that debt relief under the enhanced HIPC Initiative--which was also agreed upon at that time--had a significant poverty reduction impact.

A new approach to supporting reform and adjustment programs in low-income countries was therefore developed by the staffs of the World Bank and the IMF and subsequently endorsed by their respective Boards. The key element was to be the development by countries themselves of poverty reduction strategies, set out in a new vehicle: the Poverty Reduction Strategy Paper (PRSP). While stating clearly that the ultimate goal of the new policy was poverty reduction and the achievement of related international development goals, the initial policy framework did not spell out explicitly the objectives of PRSPs. It merely listed a number of process requirements, as well as expected outputs and intermediate outcomes.

• Process:

The process was to be based on five key principles: (i) country driven, with broad-based participation of civil society; (ii) results-oriented; (iii) long-term perspective; and (iv) comprehensiveness (to address the multi-dimensional nature of poverty and the policies needed to reduce it); (v) partnership (with all stakeholders and with/among donors, particularly between the IMF and the World Bank). The World Bank and IMF support the process, with their operations oriented around the objectives and strategy set out in the PRSP. In approving this framework, IMF Executive Directors stressed that its implementation would require a closer collaboration between the staffs of the Bank and the Fund, and they welcomed the clear delineation of responsibilities proposed, with the IMF focusing on macroeconomic policy and related structural aspects. They emphasized that IMF staff should not be expected to become directly involved in areas that are primarily the responsibility of the Bank.

• Primary outputs:
  • focused document laying out realistic but challenging poverty objectives, along with the policies needed to achieve them;
  • suitable basis for IMF and World Bank concessional lending;

  • key instrument for low-income countries' relationships with the donor community

• Intermediate outcomes:
  • development of participatory processes for the setting of poverty reduction goals and the monitoring of the implementation of poverty reduction strategies
  • improved understanding of links between policies and poverty reduction outcomes
  • setting of priorities and design of public actions to achieve poverty goals
  • development of appropriate diagnostics and indicators of progress in poverty reduction
  • deepening of a shared vision across civil society regarding poverty reduction strategy
• To complement this new tool, the IMF Executive Board endorsed the Managing Director's proposal to transform the ESAF into the PRGF. Prominent features that were to distinguish PRGF from ESAF-supported programs included:
• Broad participation and greater country ownership;
• Embedding the PRGF-supported program in a broader set of measures set out in an overall strategy for growth and poverty reduction;
• Government budgets that are more pro-poor and pro-growth
• Ensuring appropriate flexibility in fiscal targets;
• More selective structural conditionality;
• Emphasis on measures to improve public resource management/accountability; and
• Social impact analysis of major macroeconomic adjustment and structural reforms.

IMF Executive Directors, upon endorsing this change, again stressed that the IMF did not have the panoply of expertise needed to assess the quality of social spending and related issues and, therefore, that the social components of IMF-supported programs should draw, to the fullest extent possible, on the work of the World Bank or other relevant institutions.

Main findings of internal reviews to date

Since the adoption of the PRSP/PRGF approach, progress reports and internal reviews of early experience were conducted by IMF and World Bank staff in 2001 and 2002.⁴ Given the limited amount of time elapsed and of full PRSPs available, these reviews focused on process issues and on experience with I-PRSPs. The 2002 reviews also drew upon extensive external consultations (discussed in the next Section). One of the purposes of the current evaluation will be to assess how much things have changed now that a larger number of full PRSPs is available. The main findings of these internal reviews are summarized here to provide background to the evaluation. At this stage, the IEO does not have its own view on these findings.

Regarding the PRSP⁵

• broad agreement among all stakeholders involved on the validity of the objectives of the PRSP approach and its usefulness in tackling poverty;

• general recognition that there have been improvements over time in PRSP process and content;

• widespread agreement that the four key achievements of the PRSP approach to date are: (i) a stronger sense of ownership among most governments; (ii) a more open dialogue; (iii) a prominent place for poverty reduction in policy debates; and (iv) an acceptance by the donor community of the principles of the PRSP approach;

• focus should now shift beyond process to content and implementation; hence the importance of improving knowledge of the linkages between policies and poverty outcomes; and of utilizing and building local capacity in core areas;

• areas that will need continuing attention are: realism in the setting of goals and targets as well as in managing expectations; the importance of openness and transparency; the importance of flexibility, to allow for different country circumstances; the desirability of debate about alternative policy choices; and the importance of patience and perseverance with implementation;

• problems identified in a significant number of cases included: the lack of involvement of specific groups, including Parliaments, in the participatory process; unsatisfactory modalities for donor involvement (apart from the Bretton Woods Institutions); lingering weaknesses in poverty data and analysis and in target setting; lack of adequate poverty and social impact analyses; lack of satisfactory monitoring indicators and/or institutional arrangements; insufficient prioritization and specificity of public actions identified in PRSPs; uneven coverage of cross-cutting issues such as gender, HIV/AIDS, good governance and rural development; persistent weaknesses in public expenditure management systems and accountability in general; uneven integration of the PRSP process into existing decision-making processes; and need for more opportunities for learning and disseminating lessons and good practices.

While still refraining from providing strict directives as to the content of PRSPs and the implementation of the process, these internal reviews nonetheless identified a number of "good practices" in all the areas of the PRSP process.⁶

Regarding the PRGF

• In all three fundamental areas for change, namely program content, country ownership and the IMF's role, there has been substantial progress, but more can be done.

• Areas of progress include consistency between the policy goals and macroeconomic frameworks of PRGF-supported programs and PRSPs, increased allocation of resources toward poverty reduction spending, streamlined structural conditionality and better coordination with the World Bank.

• Areas leaving most scope for further improvements include the more systematic incorporation of poverty and social impact assessments (PSIAs) into program design, increased efforts towards improving the quality and management of public spending, and a perceived need to encourage deeper and broader discussion and analysis of the macroeconomic framework and the policies in PRGF-supported programs, with increasing focus on the analysis of the sources of growth.

Main external criticisms of the PRGF and PRSP initiatives⁷

While the PRSP process has been welcomed by many external commentators as putting increased emphasis on a country-driven, participatory process and on the role of poverty diagnostics in policy design, a number of concerns have also been raised. The following brief summary is not meant to be a comprehensive review of the various critiques, and the IEO does not yet have its own view on the issues raised, but the following broad sets of issues that have been raised are relevant for the evaluation.

Roles of the IMF and the World Bank

Some external critics have argued that the continuing dominance of the Bretton Woods Institutions in agenda setting and the choices of the reforms to be pursued, as evidenced by the need for a joint staff assessment (JSA) of the PRSP, "clearance" of this assessment by the Boards of the two institutions,⁸ and the HIPC link, remains a serious impediment to country ownership. Some characterize this process as little more than a relabelling of previous IMF and World Bank policy practices. Some observers have also argued that this dominance is an impediment to meaningful donor alignment since donor views are still not sufficiently taken into account.

Participatory process

Concerns also relate to the nature and extent of domestic participation. Some external critics argue that while stakeholders have been consulted, their influence on the choice, design, and implementation of policies has not increased markedly. This is partly related to the limited technical capacities of civil society and other groups that represent the poor but it also reflects, in their view, flaws in the design of the process, including an underestimation of the time and resources required to build the capacity to enable civil society to contribute effectively to policy formulation. In this context, some commentators see the need to limit the ambition and content of the PRSPs, which are currently too detailed and all encompassing, in order to avoid the risk of large divergences between realizable resources and the goals to be addressed.

Some critics also suggest that the level of civil society participation tends to decline or break down altogether as the PRSP process approaches key final stages. They argue that the final document tends to be substantially different in language from the penultimate one discussed with stakeholders. Thus, while the participatory process initially raises a number of expectations on the evolution of policies, the final version of the PRSP does not always reflect conclusions of the civil-society based consultation process. Other observers argue that the inability of the participatory process as implemented to address conflicting interests/policy tradeoffs means that it inevitably has a limited impact at the implementation stage. Such problems may be greatest for macro-economic policy formulation--where the IMF has primary responsibility.

Some external critics also suggest that the participatory process gives insufficient weight to the views of parliamentarians, the private sector, and, within civil society, those of rural dwellers--with adverse consequences for the comprehensiveness of the policy debate. In contrast, other observers have argued that the objectives set for the PRSP with regard to participation are unrealistic and vague, especially in the area of macroeconomic policies, and that in practice the PRSP process is often not well integrated with the framework for domestic macro-policy formulation, which can cause problems for the links between the PRSP and the PRGF (see below). They have also expressed concern that a participatory process that is largely separate from existing political institutions, including local governments, risks undermining these institutions.

Links between the PRSP, PRGF, and HIPC

Some external commentators see the need to delink the PRSP process from the HIPC framework arguing that the linkage has led to a rushed PRSP process.

With respect to the PRGF, commentators note that, for the reasons discussed above, the macroeconomic policies embedded in PRGF-supported programs are not always well integrated with PRSPs. In some cases, this reflects weaknesses in PRSPs themselves. It is thus difficult to tell at this stage the extent to which the PRSP process has informed the policy content of PRGF-supported programs. Some critics therefore see the policies supported by PRGF as traditional structural adjustment programs in another guise. More generally, there is a concern that by putting the emphasis on process changes, the PRSP/PRGF initiative overlooks the need for a reexamination of the traditional policy planks of structural adjustment programs based on a rigorous analysis of their impact on growth and poverty reduction.

Some civil society representatives and in-country NGO groups have also objected to the methods used to evaluate and monitor the impact of the PRSP/PRGF processes, which are often based on hastily collected and inadequate data, especially about poverty, and using methods on which there is little consensus. They highlight the need for independent and credible monitoring mechanisms, as part and parcel of the participatory framework.

One response made to these various criticisms is that the PRSP/PRGF initiative is meant to begin a fundamental change in approach, which cannot happen immediately, and that the degree of progress will depend on each country's particular institutions and starting conditions. Commentators who take this view argue that the key test is whether approaches in practice (including within the IMF and World Bank) are making significant progress in the right direction.

II. TERMS OF REFERENCE OF THE EVALUATION

The purpose of this evaluation is to assess the IMF's role in the PRSP process as well as the PRGF. Recognizing that a parallel evaluation of the PRSP is being undertaken by the World Bank's OED, the IEO will not assess important issues where the primary responsibility lies with the World Bank. The evaluation will seek to assess, in keeping with standard practice in the area of aid effectiveness, the following five aspects: efficiency, effectiveness, impact, relevance and sustainability. This approach effectively entails addressing three broad sets of questions:

• Are the objectives of these initiatives suitably defined and is their design consistent with the achievement of cost-effective and sustainable progress?

• Is the IMF delivering on its commitments embedded in the PRGF/PRSP and with what results?

• What accounts for any shortcomings diagnosed (including systemic constraints) and what suggestions emerge from the evaluation about how they can be remedied as far as the IMF is concerned?

A few important considerations should be acknowledged from the outset, all dictated by a need to keep the scope of evaluation reasonably focused and consistent with the IEO's comparative advantage:

• First, besides the PRGF, the evaluation will concentrate on the role of the IMF in the PRSP process and on those dimensions of the PRSP initiative that are directly relevant to the IMF's mandate. Clearly though, the ultimate success or failure of the PRSP and PRGF initiatives will depend to a considerable extent on factors that are beyond the scope of this evaluation, such as the role of other stakeholders, in particular country authorities and the World Bank, as well as the wider donor community, the global evolution of the volume and nature of ODA flows and broader developments in the world economy.

• Second, the evaluation will focus on the experience of countries with full PRSPs, based on the assumption that this experience has the greatest potential for generating lessons of importance for the IMF's role in the future. This focus implies that start-up problems (such as the time pressure initially caused by the linkage of I-PRSPs with the HIPC initiative), owing to their one-off nature, will not be directly assessed. Nor will the evaluation examine directly issues related to countries encountering protracted difficulties either to enter into the PRSP process or to move from the I-PRSP stage to the full PRSP (including post conflict cases). While we recognize that these issues are important, we think that they would be better addressed in a separate study and at a later stage when it is easier to distinguish countries facing truly protracted difficulties from those just requiring a little more time to complete the process of drawing up a full PRSP.

• Third, at this stage of the process, the evaluation will not be able to cast much light on final outcomes for growth and poverty.⁹ The focus will therefore be on inputs (the PRSP process and PRGF-supported programs' formulation), outputs (i.e., PRSP contents and PRGF-supported programs' design) and intermediate effects (i.e., institutional and policy changes).

In spite of these limitations, we expect this evaluation to have a substantial value-added over the internal review undertaken in late 2001/early 2002, for the following reasons. First, the sample of full PRSPs to be looked at will be expanded from 6 in the internal review to 23 in this evaluation, and six of those coincided with or preceded the adoption of a new PRGF arrangement (compared to none in the internal review sample). Second, most of the interactions with PRSP/PRGF stakeholders planned for this evaluation will take place in the second and third quarters of 2003, i.e. 1½ years after the consultations held as background to the internal review--a period during which significantly more experience has been gained by all stakeholders. Third, taking account of the particular comparative advantage of the IEO as an evaluation unit that is independent but also has access to internal documents and IMF staff, the evaluation aims to cast light not just on what has happened under these initiatives, but also on the why of any shortcomings in the approach (i.e. by examining constraints, incentives, and possible side effects that appear to be of general applicability) and on suggesting specific solutions.

The broad questions outlined above lead to a range of more specific issues involving the three dimensions just mentioned along with the overall architecture of the initiative. These issues are outlined below. Not all will be addressed in the same depth in the evaluation. Rather, we expect that a prioritization will emerge as the evaluation proceeds and as major issues are identified. For all issues, the evaluation will seek to investigate the causes of any shortcoming identified, and attempt to suggest remedies. To avoid undue repetition, this point has not been duplicated in every set of questions. The methodology envisaged to address these issues is discussed in Section 3.

One way of thinking about the scope of the evaluation would be in the context of a logical framework diagram (see White (1999) for an example of this approach). While we have not prepared a detailed mapping of this framework, Figure 1 gives a broad indication of the different stages. The evaluation will focus on only part of the overall framework, namely those elements of the first three stages (i.e., process, outputs, and effects) that directly concern the IMF's role.

A. Process issues

PRSP Process and IMF Inputs Into It

1. To what extent are PRSPs country-driven documents in the areas of the IMF's primary responsibility? Specifically, does IMF involvement strike the right balance between providing needed support and allowing the process to be genuinely driven by country authorities? Do the methodological tools provided by the IMF in its areas of expertise (e.g. the relevant sections of the PRSP Sourcebook) provide adequate guidance and allow sufficient scope for country-driven choices between alternative policies? Is IMF policy advice (including in the Sourcebook) based on concrete evidence of the links between policies and growth/poverty reduction?¹⁰

2. What is the nature of the IMF's contribution to the participatory process, in particular, in terms of : (i) provision of information to the various stakeholder groups? (ii) interaction with the donor community and civil society? Is the process by which the macroeconomic framework of PRSPs is formulated participatory and, if not, what are the practical obstacles to a participatory approach on such issues?

3. In the areas within its mandate, does the IMF adequately assess the ability of countries to undertake an effective PRSP process, including poverty and social impact analyses and what steps does it take--either directly or in coordination with other providers of technical assistance--to help countries strengthen their capacity for domestic policy formulation, implementation and monitoring?¹¹

PRGF process

4. Has the process of program formulation changed under the PRGF with respect to: (i) the IMF's own internal procedures (including the cycle of missions); (ii) the scope, depth, and transparency of IMF staff interactions with various stakeholders (including line ministries, parliaments, private sector, the poor, CSOs, and donors); and (iii) the analytical basis of the macroeconomic framework? In particular, how are poverty/social impact assessments (PSIA) incorporated into the design of the policy framework and what are the obstacles to greater use of PSIA? Where the PRGF program was preceded by a PRSP, was it clearly derived from the former?

B. Immediate outputs

PRSP Contents

5. In the areas of the IMF's primary responsibility, to what extent do PRSPs have the general components expected from them, including an analysis of the impact on growth and income distribution of past macroeconomic and related structural policies, a robust macroeconomic framework, along with related priority public actions and a credible financing plan ?

6. Do PRSPs improve the clarity of policy choices in the macroeconomic and budgetary areas by providing realistic goals and projections, an analysis of the costs and benefits of alternative policy options and the tradeoffs between them, and a suitable prioritized plan of actions?

7. In the areas of the IMF's primary responsibility, do PRSPs provide a framework suitably linked to implementation, including monitoring arrangements and feedback mechanisms? While taking into account the expanded time horizon over which their results will be seen, are PRSPs also sufficiently connected to shorter-term policy-making instruments (in particular the budget and the medium-term expenditure framework), so as to provide a framework within which a PRGF-supported program can be derived?

Joint Staff Assessment

8. Are JSAs clear and candid in their assessments of suitability of PRSPs as a basis for IMF concessional lending ? How are inputs from other stakeholders handled in making such assessments?

9. What is the assessment of the usefulness of JSAs by: (i) country authorities? (ii) the Executive Board? (iii) other PRSP stakeholders ? To what extent do JSAs play a signaling role of their own to aid donors?

PRGF-Supported Program Design/ content

10. To what extent do PRGF-supported programs incorporate the key features intended to differentiate them from ESAF-supported ones? In particular, to what extent are these programs aligned on PRSPs, in terms of their macroeconomic framework, targets and policy objectives, linkage and support to budgetary processes, and use of monitoring indicators? Is program design influenced by PSIAs?

11. Is streamlined structural conditionality suitably prioritized in PRGF-supported programs to those actions judged critical to successful implementation of macroeconomic policy? Is the World Bank's conditionality taken into account in that process? What has happened to the aggregate level of conditionality and how are gaps and overlaps between the agendas and priorities of both institutions handled?

12. Are program targets and underlying projections realistic (and consistent with the World Bank's assumptions)? Are they set in sufficiently flexible terms to allow a timely adaption in case of unexpected shocks (be they positive or negative)?

13. How are external financing requirements derived in PRGF-supported programs and how are these estimates linked to the longer-term objectives set out in PRSPs, including debt sustainability? How is the tradeoff between accommodating the higher spending plans embedded in PRSPs and preserving/progressing toward macroeconomic stability being handled in PRGF-supported programs?

C. Intermediate effects and outcomes¹²

Institutional Effects¹³

14. In macroeconomic and related areas, are PRSPs contributing to substantial improvements in domestic policy formulation and implementation? For example, are they contributing to improvements in the analytical bases (including relationships between macroeconomic and related structural policies and poverty outcomes, monitoring tools etc.) for policy debates and economic policy decisions? In particular, do the monitoring and evaluation arrangements provided for in PRSPs effectively feed back to policy formulation in these areas?

15. Are PRSPs linked to countries' budgets and public expenditure management frameworks on a permanent, institutionalized basis, as an integral part of domestic policy formulation?

16. Have the various institutional arrangements surrounding PRSPs brought about a closer, more effective collaboration between the IMF and the World Bank (including in their analytical work and setting of priorities for their respective country operations)? What has been the performance of the Joint Implementation Committee (JIC) in that respect?

Impact on Economic Policy and Preliminary Evidence on Outcomes¹⁴

17. How has the PRSP process affected the nature and scope of the policy debate, within the country and between the country and the IMF, as well as policy outcomes on key issues that are in the IMF's primary area of responsibility? How has PRGF-supported program implementation been affected? For example, what changes in policies have resulted from a more country-driven process?

18. What can be said of the evolution of expenditure composition parameters vis-à-vis suitable comparator group(s), looking at both budget allocations and actual spending? What has been the impact on tax structure/budgetary process and public expenditure management frameworks? What do the intermediate indicators used to monitor progress towards the MDGs suggest (focusing on those indicators that are most relevant to the IMF's role)?

19. While it is too early for a systematic assessment of the impact of the PRSP/PRGF on final objectives, the evaluation will collect what evidence is available on the evolution of various program outcomes (e.g. GDP growth and key macro indicators) compared, for example, with earlier ESAF-supported programs.

D. Questions of architecture and design of the PRSP/PRGF initiatives

The evaluation will also address several issues involving the design and architecture of the PRSP/PRGF approach, in order to assess its relevance and sustainability with respect to the ultimate objectives. Taking into account the parallel evaluation by the OED the goal would not be to examine all questions involving the overall architecture, but to address some specific questions concerning the IMF's role:

20. Does IMF staff receive sufficiently clear guidance from both internal and public policy documents regarding: (i) their involvement in the PRSP process? (ii) the formulation of PRGF-supported programs? (iii) the depth and modalities of their collaboration with World Bank staff? Are the supporting institutional arrangements and the incentives faced by staff consistent with a uniform and effective delivery of the promises embedded in the PRSP/PRGF approach as far as the IMF is concerned?

21. Are the IFI's expectations regarding the PRSP spelled out clearly enough for PRSPs to provide both an effective, operational strategic policy framework for poverty reduction and a vehicle for effective donor coordination ?

22. Are the function of the JSA process and the IMF's role in it appropriate? In particular, does the need to gain JSA endorsement lead to a form of self-censorship on the part of countries? Should changes be envisaged to the JSAs, e.g., by making them more inclusive of other stakeholders' views, so as to make judgments on PRSPs less of a "Washington thing" while providing a frank and clear assessment?

23. What actions could/should the IMF take, within its mandate and area of expertise (e.g., as part of its broader surveillance role), to link the PRSP objectives with policies in the advanced economies that would improve the external economic environment faced by the PRSP countries?

24. What does the evaluation suggest regarding what should be the long-run role of the IMF--and the PRGF in particular--in relation to the PRSP process in countries: (i) with no immediate short term balance of payments need? and (ii) with clear short-term balance of payments needs but where the poverty reduction strategy or its implementation is judged inadequate? In those cases, what suggestions emerge from the evaluation about what would be a suitable strategy for IMF involvement and what functions should the IMF perform in those cases, based on its comparative advantage?

III. ENVISAGED METHODOLOGICAL APPROACH

In order for the IEO to concentrate on its own areas of expertise and avoid duplicating work undertaken by others, while at the same time letting its own judgments be informed by external analysis, the following three-pronged approach is envisaged:

• The IEO's original contribution would focus on the areas where it has a comparative advantage, namely IMF operations in the relevant countries (e.g., IMF inputs to the PRSP process, PRGF-supported programs, surveillance, and technical assistance activities) and internal IMF processes.

• Aspects of shared interest between the IMF and the World Bank will be reviewed in collaboration with OED.¹⁴ IEO and OED will undertake some evaluative work jointly (see below) and will also share the findings of any work done independently. However, each evaluation unit will retain its independence of judgment on the issues at stake.

• Many of the cross cutting issues related to the PRSP process have also been studied by various research and development institutions in both the low income countries themselves and in donor countries, including through case studies. While final judgments from the evaluation will remain the exclusive responsibility of the IEO, this material will be systematically reviewed and, where appropriate, drawn upon by the IEO to inform its own analyses. A survey of this literature will be part of the evaluation.¹⁵ Authors of such evaluative or analytical material related to the issues discussed above are invited to bring their work to the attention of the IEO.

With respect to IEO's original contributions, the evaluation will follow a three-tiered approach consisting of: (i) in-depth cases studies; (ii) cross-cutting analyses of the full sample of countries with a full PRSP; (iii) control-type analyses. The case studies will contribute to answering all the questions, while cross country or institution-wide analyses will be undertaken to test how generalizable findings from the case studies are, and to provide additional evidence on each set of issues. The matrix presented in Annex II provides a summary mapping of key tasks and methodologies envisaged to tackle them.

(i)	Tanzania*
(ii)	Nicaragua*
(iii)	Mozambique*
(iv)	Tajikistan
(v)	Vietnam
(vi)	Guinea*

Cross-country analyses would be undertaken on the full sample of countries with a full PRSP as of December 2002. Their purpose would be to test on a broader scale the relevance of the findings made in the case studies as well as to bring out other messages of general significance. These analyses would rely upon systematic reviews of IMF staff reports on surveillance and lending arrangements, and IMF databases on program design/ implementation and economic outcomes; surveys of country authorities, IMF staff, donors and other stakeholders (including the civil society networks that took part in the PRSP participatory process). Care would be taken, in the conduct of these analysis, to distinguish within the sample between different vintages of PRSPs and PRGF arrangements, in order to identify any learning effects.

Control-type analyses are also envisaged, essentially of two sorts: first, broad-based statistical analyses of the full sample of PRGF-eligible countries, in order to compare the performance of program versus non program countries, PRSP versus non PRSP countries and ESAF versus PRGF-programs.¹⁸ Second, tentatively, a desk review of a small number of PRGF-eligible countries that chose not to engage in the PRSP process may be undertaken, in order to highlight any differences in the process of policy formulation and in actual policy choices and outcomes compared to PRSP countries. There are obvious problems of sample-selection bias in any choice of such a limited control group, but it may highlight some useful qualitative messages.

The modalities for cooperation between the IEO and the OED on their respective evaluations, will be guided by the following broad principles: (i) each institution will produce its own evaluation report in keeping with its own procedures, and these reports will be presented to their respective Boards; and (ii) the inputs to the evaluations will be organized in such a way as to avoid duplication of demands on member countries (e.g. with respect to country case studies). Joint case studies are expected to involve, inter alia, joint country missions and a common stakeholder survey, as well as single country reports.

The timetable of the evaluation contemplates submission of the final report to the IMF's Executive Board in the first quarter of 2004.

Back to top

Bibliography

Action Aid, 2002. Inclusive Circles Lost in Exclusive Cycles, Contribution to the First Global Poverty Reduction Strategies Comprehensive Review, mimeo, January.

Bevan, David and Christopher Adam, 2001. Poverty Reduction Strategies and the Macroeconomic Policy Framework. Mimeo- Department of Economics, University of Oxford, January.

Bretton Woods Project and World Vision, 2002. Blinding With Science or Encouraging Debate? How World Bank Analysis Determines PRSP Policies. London, September.

Campbell, Bonnie and Bruno Losch, 2002. Les Pauvres, Bénéficiaires ou otages des Stratégies de Réduction de la Pauvreté? Politique Africaine, October.

Catholic Relief services, 2001. Review of the Poverty Reduction Strategy Paper Initiative Based upon the Experiences and Comments of CRS Partners in Bolivia, Honduras, Zambia and Cameroon, December. Baltimore; Catholic Relief Services World Headquarters.

Christian Aid, 2002. Quality Participation in Poverty Reduction Strategies: Experiences From Malawi, Bolivia and Rwanda.

Christian Aid, 2001. Ignoring the experts: Poor People's Exclusion From Poverty Reduction Strategies. October.

Cling, Jean-Pierre, et al., 2002. The PRSP Initiative: Old Wine in New Bottles? Paper presented to ABCDE-Europe conference 2002.

European Network on Debt and Development (EURODAD), 2001. Many Dollars, Any Change? Part 1: The Changing Nature of Development Cooperation: Building Ownership. Brussels, October.

______, 2001b, Many Dollars, Any Change? Part II: Have Structural Adjustment Policies Failed the Poor? Brussels, October.

Gupta, Sanjeev, et al. 2002. Is the PRGF Living Up to Expectations? An Assessment of Program Design. IMF Occasional Paper No. 216.

Hubbard, M. 2001. Attacking Poverty, a Strategic Dilemma for the World Bank. Journal of International Development, Vol. 13, pp. 293-298.

International Monetary Fund, 1997. The ESAF at Ten Years: Economic Adjustment and Reform in Low-Income Countries.

______, 1998 a. External Evaluation of the ESAF: Report by a Group of Independent Experts.

______, 1998 b. Review of the IMF's Approach to Social Issues and Policies.

______, 1999 a. Economic Adjustment and Reform in Low-Income Countries. Studies by the Staff of the International Monetary Fund (Bredenkamp and Schadler, editors)

______, 1999 b. Poverty Reduction and Growth Facility: Operational Issues.

______, 2000. Key Features of IMF Poverty Reduction and Growth Facility Supported Programs.

______, 2001. IMF Lending to Poor Countries: How does the PRGF Differ From the ESAF?

______, 2002 a. Review of the Key Features of the Poverty Reduction and Growth Facility--Staff Analyses. (February)

______, 2002 b. Review of the Poverty Reduction and Growth Facility: Issues and Options. (February)

International Monetary Fund and World Bank, 1999 a. Heavily Indebted Poor Countries (HIPC) Initiative--Strengthening the Link between Debt Relief and Poverty Reduction

______, 1999 b. Poverty Reduction Strategy Papers--Operational Issues. (December)

______, 2000. Progress Report on Poverty Reduction Strategy Reports.

______, 2001. Poverty Reduction Strategy Papers--Progress in Implementation. (April)

______, 2002 a. External Comments and Contributions on the Joint Bank/Fund Staff Review of the PRSP Approach. Volume 1--Bilateral Agencies and Multilateral Institutions. (February)

______, 2002 b. External Comments and Contributions on the Joint Bank/Fund Staff Review of the PRSP Approach. Volume 2--Civil Society and Individual Contributions. (February)

______, 2002 c. Synopsis of External Comments and Contributions on the Joint Bank/Fund Staff Review of the PRSP Approach. (February)

______, 2002 d. Review of the Poverty Reduction Strategy Paper (PRSP) Approach: Main Findings. (February)

______, 2002 e. Review of the Poverty Reduction Strategy Paper (PRSP) Approach: Early Experience with Interim PRSPs and Full PRSPs. (February)

Klasen S., 2001, In Search of the Holy Grail: How to Achieve Pro-Poor Growth?, Background Paper to SPA Task Team: Growth and Equity. Eschborn: GTZ.

Maxwell S., 2001, Innovative and important, yes, but also instrumental and incomplete: the treatment of redistribution in the New Poverty Agenda, Journal of International Development, Vol. 13, pp. 331-341.

Mosley, P., 2001, Attacking Poverty and the Post-Washington Consensus, Journal of International Development, Vol. 13, pp. 307-313.

Oxfam International, 2001. Are PRSPs Working? Contribution to the World Bank/IMF PRSP review process. Washington DC.

Structural Adjustment Participatory Review International Network (SAPRIN), 2002. The Policy Roots of Economic Crisis and Poverty. A Multi-Country Participatory Assessment of Structural Adjustment.

United Nations Conference on Trade and Development (UNCTAD), 2002. Economic Development in Africa. From Adjustment to Poverty Reduction; What Is New? New York and Geneva: United Nations.

United States General Accounting Office (GAO), 2001. Few Changes Evident In Design of New Lending Program for Poor Countries. Report No. GAO-01-581. Washington DC.

White, H. (1999), Dollars, Dialogue and Development, Stockholm, SIDA Evaluation Report.

World Bank, 1999. Building Poverty Reduction Strategies in Developing Countries

ANNEX I

ANNEX II