IEO logo
IEO Home

Independence

Goals & Instruments

Transparency

Work Program

Ongoing Projects

IEO Publications

Staffing

Terms of Reference

Origins of the IEO

Links to Related Sites


Independent Evaluation Office - Work Program

December 5, 2001
The IEO provides objective and independent evaluation on issues related to the IMF. The Office operates independently of IMF management and at arm's length from the IMF's Executive Board.

IEO's 2002-2005 Work Program

Full text in PDF Format (725Kb)  

FY 2002/2003 (Firm)

  1. Fiscal adjustment in IMF-supported programs
  2. A group of three capital account crisis cases, i.e., Indonesia, Korea and Brazil
  3. Repeat use of IMF resources

FY 2004 and 2005 (7 to 9 of the following)

  1. The Poverty Reduction and growth Facility (PRGF) and the Poverty Reduction Strategy Paper (PRSP)process (to be undertaken jointly with the World Bank's Operations Evaluation Department, OED)
  2. The IMF's advice on financial sector restructuring after a crisis
  3. Structural conditionality in Fund-supported programs
  4. The role of IMF surveillance in crisis prevention
  5. The IMF's advice on exchange rate policy
  6. The Financial Sector Assessment Program (FSAP) and Financial Sector Stability Assessments (FSSAs)
  7. IMF Technical assistance
  8. Private Sector Involvement (PSI)
  9. The IMF's approach to capital account liberalization
  10. The role of multilateral surveillance
  11. Additional country case (possibly Argentina or Turkey)
  12. Low-income country case

Fiscal adjustment in IMF-supported programs

Fiscal adjustment, especially in low-income countries, has long been the subject of criticism in large part because it involves severe trade-offs between stability and growth, or stability and social expenditures, which are often not adequately articulated or quantified and involve distributional issues that are highly sensitive politically.

Based on a sample of (mainly low-income) countries that have limited access to private international capital markets, the study will examine the major features of fiscal program design, the nature of the dialogue between Fund staff and the country authorities and other groups, the extent of country ownership, and the results in terms of the efficiency, sustainability, and social impact of the fiscal adjustment. A "nested samples" approach may be adopted, with some issues being examined over the full sample of countries and others over smaller samples.

Inter alia, the study will address the following issues:

  • Were the stated objectives of the program successfully achieved? If not, what factors contributed to the failure? Where program objectives were met, was there improvement in growth, poverty reduction and key social indicators?
  • To what extent have programs helped to achieve a more sustainable fiscal adjustment? Are fiscal deficits and sustainability defined appropriately? To what extent has adjustment encompassed the broader public sector?
  • How were trade-offs considered, e.g. amount and speed of fiscal adjustment versus the social cost of adjustment; speed of deficit reduction versus inflation path; spending cuts versus tax increases; efficiency gains versus higher spending? How did programs deal with shocks?
  • Was there room to change the mix between adjustment and financing? How was debt dynamics treated?
  • How necessary were critical structural reforms for achieving adjustment objectives?
  • How did the negotiations of fiscal adjustment take account of constraints on implementation, explore alternatives and analyze the social impact of key measures? How were major differences and perceptions of trade-offs by government and civil society handled and how were programs crafted so as to foster domestic support? What institutional changes were proposed to monitor the composition of spending and actual implementation?
  • To what extent have programs contributed to building institutional capacity and fostering ownership? What has been the rate of IMF technical assistance in these efforts and how could it be made more effective?
  • How effective has Bank-Fund collaboration been in addressing public expenditure management and social safety net issues?
  • Do Fund-supported programs over time show improvements reflecting lessons learnt? In particular, has the PRGF process made a substantial difference to how these issues are addressed?

back to top

Group of capital account crisis cases (Brazil, Indonesia and Korea)

An evaluation of the role of the Fund in recent capital account crises is a high priority because of the systemic importance of these cases, and because there is considerable controversy among external observers about key elements of the strategy pursued. The proposed study would focus on three recent country cases to evaluate the effectiveness of the Fund's role in spotting vulnerabilities during the pre-crisis period, and the effectiveness of Fund-supported programs in resolving the crises. The proposed country cases represent contrasting examples of the factors underlying the crisis and also of outcomes. They will be used to assess a range of issues that have been raised by critics of the Fund approach, including:

  • What was the Fund's assessment of the countries' policies prior to the crises? Was the nature of the crisis correctly diagnosed and was the macroeconomic adjustment strategy-notably fiscal and monetary policy-suited to the characteristics of the crisis?
  • Why was the extent of capital outflows so underestimated? Since a restoration of confidence was central to the success of the programs, how could program design have dealt with the risk that confidence would not be quickly restored? How did the size or structure of official financing packages influence the restoration of confidence?
  • What were the tradeoffs involved in alternative approaches to involving the private sector in halting capital outflows?
  • Were the structural components of the program-including with respect to the financial and corporate sectors-helpful in crisis resolution? What was the extent and quality of coordination with the World Bank and other multilateral institutions?
  • Could the sharp initial declines in output (in the Indonesia and Korea cases) have been avoided and why was the impact on output in Brazil smaller? Could the adverse social impact of the crises have been mitigated?

The study will seek to identify lessons from the diversity of the experience across the three countries.

back to top

Repeat use of Fund resources

Some twenty-five countries have been indebted to the Fund for more than thirty years out of the last fifty. Sixteen countries have been under Fund-supported programs for twelve years or more out of the last eighteen. Such prolonged use risks turning the Fund into a source of long-term financing, in contradiction with the mandate set forth in its Articles of Agreement. Many of these countries have acute debt sustainability problems and most are now enrolled in the HIPC initiative. Drawing on a number of case studies, the project would aim to answer the following questions:

  • What are the causes of this pattern of repeat use of Fund resources? Are there specific program-design or other weaknesses that might have contributed to the pattern, including as regards institutional capacity building, assessments of debt sustainability, institutional arrangements for official debt restructuring, and the overall availability of long-term financing?
  • What have been the costs entailed by repeat use for the borrower, for the Fund, and for the rest of the membership?
  • The project will also assess possible remedies-whether in the design of lending facilities and programs, in the use of technical assistance, or in the division of labor with providers of long-term financing. Part of this assessment will include an analysis of the effects of the HIPC Initiative on prolonged users.

back to top

Poverty Reduction and Growth Facility (PRGF) and Poverty Reduction Strategy Papers (PRSP)1

Building on previous internal and external reviews, and coming at a time when significant experience with the PRGF would have been accumulated, the evaluation will assess the extent to which PRGF arrangements differ from those under the Fund's previous concessional facilities. It will address issues related to program design, ownership, implementation, as well as progress in reducing the incidence of poverty and in growth performance. In addressing these issues, particular attention will be paid to the treatment of countries seeking debt relief under the enhanced HIPC Initiative, to the adequacy of Bank-Fund collaboration mechanisms, and to issues related to institutional capacity building and the sustainability of reforms.

The following specific questions will be addressed:

  • What should be the role of the Fund in low-income countries, especially those that have attained macroeconomic stability?
  • Are the macroeconomic programs underlying PRGF arrangements clearly linked to the strategies for fostering growth and reducing poverty set out in the PRSPs?
  • Has the approach to program negotiations been modified in line with the PRGF operational guidelines?
  • How effective has Bank-Fund collaboration been in ensuring that programs include measures to protect the most vulnerable?
  • To what extent has conditionality become more selective in PRGF arrangements (compared to ESAF arrangements)?
  • To what extent has the degree of country ownership made a difference to program implementation and performance against program targets? Has good performance against program targets been associated with (i) improvements in poverty-reduction indicators; and/or (ii) level of donor support?
  • To what extent are capacity weaknesses identified in programs being addressed over time? How effective has Fund technical assistance been in helping to build capacity in areas of the Fund's competence?
  • To what extent is the PRGF having a measurable impact on growth and/or poverty reduction?

back to top

The IMF's advice on financial sector crisis resolution and restructuring

A substantial proportion of the Fund's membership has experienced financial sector crises over the last two decades and such crises have been at the core of many of the recent capital account crisis cases. There has been considerable debate among external observers about the appropriateness of the Fund's policy advice in the resolution of these crises. The evaluation would focus on a panel of countries broadly representative of the types of financial sector crises the IMF has had to deal with in recent years (e.g. those involving large external liabilities and those involving primarily domestically-driven asset price bubbles). It would address the following questions:

  • How was the nature of the financial sector crisis and the implications for the macroeconomic outlook diagnosed? Has the Fund approach to such crises evolved as experience has been gained?
  • What were the common ingredients of the crisis resolution strategies embedded in Fund programs (e.g. bank closures, deposit guarantees, bank recapitalization, regulatory framework)? Did the strategies pursued adapt to country circumstances? Was the policy response successful in minimizing disruptions, both at the macroeconomic and at the microeconomic levels?
  • How well did Bank-Fund collaboration work, including in the integration of debt workout and corporate restructuring strategies into financial sector resolution? Did policies adopted in Fund-supported programs improve the resilience of the financial sectors in a substantial way?

back to top

Structural conditionality in Fund-supported programs

The recent streamlining of conditionality attached to Fund-supported programs has been an important element in the Fund's response to criticism of the large expansion of both the scope and the volume of structural conditionality over the last decade. The areas in which the expansion of conditionality has been most noticeable are the financial sector and privatization. The fiscal sector has always been and remains the single most important focus of structural conditionality, while conditions related to the trade regime have been traditionally important and remain so.

The project will focus on those four core areas of structural conditionality and will assess the impact on the effectiveness of Fund-supported programs in achieving their core objectives. The methodology used will combine the analysis of a range of indicators across Fund-supported programs, both before and after the streamlining drive, with a more in-depth examination of a few post-streamlining case studies.
Specific issues to be investigated include:

  • The rationale and overall effectiveness of structural conditionality in the four core areas as well as the potential costs and benefits of alternative approaches to conditionality.
  • the extent to which the streamlining has resulted in a shift of conditionality towards sectors where the Fund has a clear comparative advantage over other institutions, and more generally how it has affected program design.
  • how the measures associated with the streamlining (in particular prioritization of conditionality, enhanced collaboration between the IMF and the World Bank, and targeted provision of technical assistance) have had an impact on the quality (in terms of economic efficiency and poverty reduction) and sustainability of adjustments promoted in programs.
  • whether the streamlining has resulted in a greater ownership of program objectives by country authorities and a lesser degree of social disruption.

back to top

The role of IMF surveillance in crisis prevention

In the wake of the Asian and Russian crises of 1997/98, crisis prevention has become a major concern of both the international financial community and critics of the IMF. The Fund has launched a series of initiatives over the past two years to try and address those concerns, both in its internal work processes and in its relations with member countries.

The study will evaluate the Fund's record in anticipating and preventing balance of payments crises before and after these initiatives were launched. The analysis will be based on a panel of countries which experienced crisis, and some countries where crises were at risk (as identified by various vulnerability assessment approaches and independent judgment) but were avoided. The group of countries will be chosen to reflect crises with different origins, but with the primary focus on capital account crises. The evaluation will distinguish between the Fund's ability to anticipate the vulnerabilities that lead to crises (i.e. crisis prediction) and its ability to identify and encourage policy corrections (i.e. crisis prevention). The evaluation will seek to identify lessons for improving further the effectiveness of Fund surveillance in crisis prevention.

Specific issues to be investigated include:

  • Has multilateral surveillance been constructively used to help anticipate and prevent crises? Were the implications of international developments for individual countries appropriately analyzed and incorporated into surveillance of individual countries? Did multilateral surveillance identify vulnerabilities associated with volatility of capital flows? Were contagion risks properly assessed?
  • Has bilateral surveillance contributed to the anticipation of balance of payments crises? How does the track record of the Fund compare with that of private agencies? How have private capital markets used the increased information now made available by the IMF and has greater transparency led to a better pricing of risk?
  • Did surveillance suggest sufficiently specific corrective steps? What factors affected the ability of surveillance to influence the policy response?
  • Are certain types of vulnerabilities better diagnosed and corrected than others? In particular, how well did surveillance handle vulnerabilities related to public debt dynamics, balance-sheet mismatches, and sustainability of exchange rate pegs?
  • To what extent is adequate data available to support vulnerability assessments?
  • How have the reforms introduced in recent years altered the way in which Fund surveillance identifies and responds to potential vulnerabilities in member countries? What further changes could improve crisis prevention? Is all available information effectively incorporated into systems that can give early warning of potential vulnerabilities?

back to top

The IMF's advice on exchange rate policy

There is considerable external criticism and misunderstanding about IMF advice on exchange rate policy, especially to emerging market economies. This evaluation would focus on the following issues, drawing inter alia on specific country case studies.

  • Are there common guiding principles governing the IMF's policy advice on the exchange rate and are they implemented consistently in practice?2 For example, the Fund's public position is that, in line with the Articles of Agreement, the Fund would abide by a member's preferred choice of exchange rate regime and advise on policies needed to support that choice. To what extent has this general approach been followed in practice both in surveillance and in program design?
  • How valid is the bipolar view of feasible exchange rate regimes for countries with an open capital account? What position has the Fund taken on this issue?
  • How effective was the Fund's policy advice on the strategy for exiting a pegged exchange rate regime, if the peg is inconsistent with underlying policies? Why have some exits been relatively successful, in terms of minimizing disruptions, and others not? How has the Fund influenced the timing of such exits and accompanying domestic policy commitments?
  • Has the Fund's advice on exchange rate policy paid sufficient attention to the regional context?

back to top

Financial Sector Assessment Program (FSAP) and Financial Sector Stability Assessments (FSSA)

The FSAP/FSSA is one of the major initiatives undertaken by the Fund in response to the crises of the late 1990s. It absorbs substantial Fund (and World Bank) resources and is expected to play an important role in bilateral surveillance of all member countries. By the end of FY 2003, approximately 80 assessments are expected to be completed. The evaluation would focus on the following areas:

  • How well do FSAPs and FSSAs identify the major risk factors in a country's financial system and have they then proposed appropriate and feasible measures to address them? Are they using the best approach to diagnose vulnerabilities or would other methods be more productive? How well are TA needs assessed and implemented?
  • Are recommendations articulated as a coherent "blueprint" for reform with proper regard to suitability, sequencing, and implementation capacity? How well are the assessments integrated into ongoing surveillance?
  • Does the FSAP/FSSA achieve its goals in a cost effective manner? Do countries perceive the FSAP as adding value to the domestic policy development process? Has it deepened the quality and coverage of discussions with authorities on financial sector issues?
  • How has the private sector reacted to the new information published in FSSAs and how has it impacted their activities and risk perceptions?

back to top

IMF technical assistance

Technical assistance (TA) is one of the major activities of the IMF. In FY 2002, it absorbed $125 million, or 18 percent of the total administrative budget. In 1999, a comprehensive internal review of TA identified a number of shortcomings in the management of those resources, and gave a mixed assessment of the overall effectiveness of the Fund's TA efforts.3 In response to those findings, a series of initiatives were taken since late 2000 to realign TA with the Fund's overall policy priorities and to standardize TA management across provider departments.

The project would focus on assessing to what extent these initiatives have provided adequate remedies to the identified shortfalls? Based in part on a number of case studies, the project would evaluate the following issues:

  • The framework used to allocate TA resources across countries and subjects, and the process for ensuring cooperation and setting priorities with other TA providers.
  • The effectiveness of TA, especially in building institutional capacity and fostering sustainable reforms. How is absorptive capacity taken into account and how effectively is the policy advice tailored to the circumstances of each country? How is transfer of knowledge emphasized and what measures are taken to make TA self-limiting?
  • The interactions between TA and other major Fund activities, such as surveillance and use of Fund resources.
  • The project will also assess the Fund's internal TA evaluation procedures, including the outcome of ongoing efforts to establish a common 'best practice' evaluation methodology for all TA-providing departments.

back to top

Private sector involvement (PSI) in crisis resolution

The need to involve the private sector effectively in the resolution of financial crises has been much discussed and it clearly impacts on decisions regarding the appropriate limits on access to Fund resources. There are wide differences of views on how to handle this issue, including within the official community. Although policy in the area is still evolving, it is likely to build to a more definite shape over the next two years and an evaluation of country cases could therefore be undertaken in FY 2005 to inform the broader debate. Specific questions to be addressed would include:

  • How are private market creditors being involved in the resolution of financial crises and are principles in this respect being applied consistently? Is debt sustainability/solvency analyzed in a consistent manner? Is there a tendency to be over-optimistic about the expected return to normalcy, e.g. expected reductions in risk premia?
  • What has been the catalytic role of Fund-supported programs and official financing packages in mobilizing private capital (in terms of both volume and terms of financing)? Why in many cases were private flows less than assumed in the program?
  • How effective have been the various concerted efforts, short of a standstill, to secure private sector involvement? What has been the outcome in terms of burden-sharing among creditors? Are further institutional changes required to facilitate PSI?
  • What lessons do the early cases suggest about the impact of concerted PSI on the real economy on the resumption of voluntary flows, and on the availability of financing to other countries?

back to top

The IMF's approach to capital account liberalization

This study will address issues on which there is significant disagreement and which are important for a broad range of Fund members. The evaluation would assess the Fund's policy advice on the scope, pace and composition of capital account liberalization. Drawing, inter alia, on a number of country case studies, it would address questions such as:

  • What has been the IMF's advice regarding capital account convertibility? How have policies towards the capital account changed in recent years in the light of experience with capital account crises?
  • Has the Fund's policy advice on capital account liberalization been consistent across countries with respect to the interaction with macroeconomic policy and the sequencing of other reforms, especially vis-à-vis the financial sector? What has been the effect of the proposed framework for taking account of trade-offs between strengthening intermediation and minimizing vulnerabilities and how is it being implemented in practice?4
  • Is multilateral surveillance giving sufficient attention to possible measures on the supply side to reduce the volatility of capital flows?
  • Given that occasional crises are inevitable, what role is there for temporary capital account restrictions as part of a crisis response?

back to top

The role of multilateral surveillance

Multilateral surveillance is a core activity of the IMF, which aims to provide a basis for analyzing the forces driving the world economy, identifying global vulnerabilities, and advising on appropriate policies, especially in the most systemically important countries, to avoid sub-optimal outcomes. Potential issues to be addressed would include the following:

  • The role of the World Economic Outlook and the International Capital Markets Report exercises in forecasting prospects for the global economy and identifying global vulnerabilities.
  • The effectiveness of the interaction between multilateral surveillance and bilateral surveillance of the major industrial countries. What is the value-added and how can it be improved?
  • The effectiveness of the IMF's inputs into the deliberation of various international groups such as the G-7 and various regional groupings (such as APEC and the Manila Group).
  • What has been the impact of the increased emphasis on capital market issues in multilateral surveillance and the enhanced interactions with private financial market participants?

back to top


Footnotes

1 The study of the PRSP process will be undertaken in collaboration with the World Bank's Operations Evaluation Department.

2 See, for example, the Chairman's summing up of the Executive Board discussions of Exchange Rate Regimes in an Increasingly Integrated World Economy (September 21, 1999 and November 15, 1999)

3 During the course of the IEO's consultations on the work program, a substantial number of observers within and outside governments also expressed the view that IMF TA programs, while generally providing sound policy advice, were not especially effective in building institutional capacity.

4 Capital Account Liberalization and Financial Sector Stability - Considerations for Sequencing (SM/01/186, June 25, 2001)