January 13, 2003

The IEO provides objective and independent evaluation on issues related to the IMF. The Office operates independently of IMF management and at arm's length from the IMF's Executive Board.

Proposed Work Program for Fiscal Year 2004

This document is available in French (190 kb PDF file) and Spanish (192 kb PDF file) - [Disclaimer: Reasonable efforts to ensure accuracy of these documents have been made; however, only the English version produced by the IEO is the authentic version]

3. A similar process of consultation has been followed to determine the IEO work program for FY 2004. The 12 remaining topics from the original list of 15 in the medium-term program were reviewed internally and were narrowed down to the following short list of five topics, which appeared most suitable for the work program for FY 2004.

(i)	The PRSP/PRGF experience based on full PRSPs.
(ii)	Country case study of either Argentina or Turkey.
(iii)	The role of the IMF in providing Technical Assistance.
(iv)	The IMF's surveillance function.
(v)	The IMF's approach to capital account liberalization.

A discussion paper outlining the main issues in each of these topics was circulated for comments to selected individuals and to all departments in the Fund and also posted on the IEO website.² The paper was also circulated to management. The views of Executive Directors on the discussion paper were ascertained in a meeting of the Evaluation Committee held on December 5, 2002.

II. Work Program for FY 2004

4. While contrasting views on priorities and timing were expressed in the course of consultations, all the five topics listed above were deemed important subjects for IEO evaluation by a large proportion of stakeholders. However, only a subset of these topics can be included in the work program since the agreed capacity limit was that IEO would begin with three evaluations per year and expand to four in the first instance, with the possibility of expanding to five per year being kept open for decision at a later date.

5. Given the capacity constraint, the work program for FY 2004 will consist of the studies listed as (i), (ii) and (iii) above, with Argentina as the relevant case study in (ii). Our consultations revealed strong support for the IEO undertaking an evaluation of both Turkey and Argentina, often on the grounds that all cases of exceptional access should be subject to evaluation. Since the Turkey program is scheduled to end in December 2003, this case study could be taken up in FY 2005. In the case of Argentina, there was concern that the study should not interfere with ongoing discussions for a possible future program. This issue has been carefully considered and IEO's judgment is that it is possible to undertake the study without interference with ongoing operations. This is because the proposed evaluation will stop at the collapse of the last program at the end of 2001 and will not pronounce in anyway on the current situation or on any new program, which may be negotiated. Furthermore, the findings of the evaluation will not become available until early 2004. The precise timing of the study will be adjusted to avoid any undue burden on staff which might be otherwise engaged.

6. This work program comes close to the steady state capacity of completing four studies per year because the evaluation of the PRSP/PRGF experience is expected to be an unusually large project equivalent to about 1¾ "average" project.

7. As in FY 2003, it is expected that the IEO will be able to make a start on a fourth project in the latter part of FY 2004, which would be completed during FY 2005. No decision has been taken yet on which study might be chosen for this purpose. A final decision will be taken towards the end of 2003 in the context of determining the work program for FY 2005. The projects listed in (iv) and (v) in para. 3 will remain under consideration for possible inclusion in the FY 2005 work program.

8. The main issues to be covered in each of the projects included in the work program are discussed briefly below. Following IEO practice, detailed issues papers/terms of reference will be prepared for each project and the IEO will seek comments from internal and external stakeholders before the scope of each evaluation is finalized. This process is already well under way for the PRSP/PRGF project. The final terms of reference will also be posted on the website and concerned stakeholders will be invited to submit substantive inputs on any aspect of the terms of reference.

A. Evaluation of the Poverty Reduction Strategy Papers and the
Poverty Reduction and Growth Facility (PRSPs/PRGF)

9. The Poverty Reduction and Growth Facility (PRGF), which was established in 1999 and with its establishment member countries gave the IMF an explicit responsibility to help low-income countries to deal with structural balance of payments problems in a manner which supports the medium-term objective of promoting pro-poor growth. The programs supported by the IMF to achieve such growth are to be based on Poverty Reduction Strategy Papers (PRSPs), which emerge from a consultative and participatory process. This approach is intended to ensure a country-driven agenda of reforms. The PRSP is also expected to provide a mechanism for effective collaboration with the World Bank, which has primary responsibility for many of the structural reform areas, and for coordination with other donors.

10. The initial experience based on an analysis of "Interim-PRSPs" and associated PRGF arrangements between 1999 and 2001 showed only tentative improvements over the earlier ESAF programs, but this could be because the early PRGFs were based on Interim PRSPs. As of end-December 2002, 21 countries have completed full PRSPs and most are engaged in PRGF arrangements based on these PRSPs. Since PRSPs involve both the IMF and the World Bank it is proposed that aspects of the evaluation related to the PRSPs be undertaken in parallel with the World Bank's Operations Evaluation Department (OED). However, the IEO and the OED would each produce their own evaluation report, to be submitted to their respective Boards, with the focus on assessing their respective institutions.

11. The evaluation would seek to present an overview of experience in all countries, which have prepared full PRSPs based on cross country data combined with a detailed evaluation of about six case studies. Since the evaluation will be undertaken in the course of calendar year 2003, it is too early to evaluate performance against long-term objectives such as the Millennium Development Goals, but the available information on the first two and in some cases three years under PRGF-supported arrangements will provide useful insights on many important aspects.³ The focus of the IEO evaluation will be on the IMF's role in the process.

12. Some of the questions which will be addressed are:

• Has the PRSP/PRGF process helped to define more clearly the role of the IMF in low-income countries in a useful way and if not, how could the architecture on the execution by the IMF of these initiatives be improved?
• Have the macroeconomic programs underlying PRGF arrangements been explicitly derived from broader strategies for fostering growth and reducing poverty set out in the PRSPs and are therefore fully consistent with them? Do the latter provide sufficient basis for elaborating such macroeconomic frameworks, or are there large areas of ambiguity which give rise to misperceptions?
• Has the approach to negotiation and program design adopted for PRGF-supported arrangements been substantively altered to conform to the goals of the PRSP/PRGF approach, including especially the objective of making the process more country-driven? In particular, how effectively are alternative policy options and the poverty and social impact of such options considered in practice?
• How effective has Bank-Fund collaboration been in implementing the PRGF/PRSP approach?
• What does experience to date suggest about the IMF's longer-term role in low-income countries, especially "post-stabilization" cases, and the need for an "exit" strategy?

In addressing these questions, the evaluation would go beyond process issues and seek to assess the substantive policy changes brought about by these initiatives. It will also assess performance in the two or three years on which data will be available.

13. Preliminary work for this study has already commenced and a draft issues paper has been posted on the IEO website and circulated to Executive Directors and management for comments. The final version of the issues paper is expected to be posted on the website by early February.

B. Case Study of the IMF's Role in Argentina

14. The crisis in Argentina with the collapse of the currency board arrangement and sovereign debt default has raised a number of questions about the effectiveness of THE IMF crisis prevention efforts and the quality and impact of its policy advice. This is especially so since Argentina was continuously engaged in THE IMF-supported programs for most of the period since the adoption of the Convertibility Law in 1991 and was often presented as a success story.

15. Although the evaluation will assess the 2000 and 2001 programs, it will also take a larger view of the IMF's involvement with Argentina from 1991 onwards and would seek to address three broad sets of questions:

(i) Did the IMF provide adequate and effective policy advice? Issues to be looked at under this heading would include the quality, timeliness and impact of the IMF's policy advice on the currency board and possible "exit" strategies, on fiscal policy and debt sustainability, and on structural policies required for the successful operation of the currency board arrangement. Reasons for nonimplementation of the IMF policy advice would also be examined, including an examination of the lessons to draw regarding the ownership of policies.

(ii) What was the impact of the IMF's almost continuous engagement in Argentina through lending arrangements over that period? Issues to investigate would include program design adequacy, possible moral hazard effects of continued IMF involvement on both the authorities and financial market participants, the modalities of the internal decision making process at key stages of Argentina's programs, and private sector involvement.

(iii) Could the IMF surveillance have played a greater role in avoiding the crisis? For example, how could the impact of the IMF surveillance have been improved and did the IMF sufficiently take into account the regional dimensions of surveillance in judging the sustainability of the overall strategy?

16. The primary focus of the evaluation would be on drawing lessons from this experience that would contribute to improve the effectiveness of the IMF lending and surveillance activities in future. The timetable of the evaluation and the precise definition of the period under review will ensure that the IEO does not interfere with ongoing operations.

C. Technical Assistance by the IMF

17. Technical assistance (TA) is regarded as critical for capacity building and fostering ownership, which are viewed as increasingly important in both surveillance and program contexts. TA is potentially extremely important in low-income countries, which often suffer from severe capacity constraints that limit the effectiveness of policy formulation and implementation. It is also important in other member countries especially where the growing concern with crisis prevention points to many new areas where TA could help to strengthen the financial system and improve the quality of data provision.

18. A review of the IMF's TA activities was undertaken by an internal evaluation group in 1999 and a number of shortcomings were identified, including uneven effectiveness, owing in part, to weak follow up by the Fund and the lack of satisfactory internal evaluation and reporting procedures, and an insufficient complementarity between surveillance and TA activities. Several initiatives were taken to address these shortcomings including the establishment of a centralized Office of TA Management, the adoption of a policy framework to ensure that TA supply is suitably prioritized, and the opening of regional TA centers in the Caribbean and in Africa. In the past few years new areas of TA have emerged, including in the financial sector, where an evaluation of what has been done would be useful before expanding the scale of the effort.

19. Some of the questions which the study could address are:

• How effective are internal IMF processes for identifying TA priorities, allocating resources across countries and subjects, and assessing the results? Have the recent initiatives led to significant changes in the way in which TA effectiveness is judged, and has this led to a reallocation away from places in which it is not being used effectively? In this respect, the evaluation will cover the outcome of ongoing efforts to establish a common "best practice" evaluation methodology for all TA-providing departments and whether the results of such efforts are being incorporated into internal decision-making processes.
  
  
• Is there enough complementarity between TA and other major IMF activities, such as surveillance and use of IMF resources, and how are these activities integrated in practice?
  
  
• How effective has THE IMF TA been in building institutional capacity and fostering sustainable reforms and how could these effects be improved? How is absorptive capacity taken into account and how effectively is the policy advice tailored to the circumstances of each country? How is transfer of knowledge emphasized and what measures are taken to make TA more goal oriented?

20. In accordance with IEO practice, the work program will be posted on the IEO website after it has been reviewed by the Executive Board.

Topics for FY 2004 or FY 2005