Prepared by the Policy Development and Review Department in consultation with other departments

March 14, 2003

Contents

---

Executive Summary

1. Introduction
   
   
2. Reviews of Fund Surveillance: Methodology and Main Conclusions
   
   
3. Operational Responses
   
   
4. The Agenda Ahead
   
   
5. Next Steps
   
   
6. Issues For Discussion

Executive Summary

In its September 2002 Communiqué, the International Monetary and Financial Committee asked for a report by its next meeting on ways of further enhancing the effectiveness of Fund surveillance. This paper responds to that request with a review of the conclusions of past reviews of surveillance, a stocktaking of the range of initiatives undertaken since the mid-1990s to strengthen surveillance, and a list of areas where further progress could help improve the overall effectiveness of the surveillance process.

Successive internal and external reviews of surveillance conducted since the Mexican crisis of 1994/95 point to five key ingredients of effective surveillance: (i) timely, comprehensive, and accurate information; (ii) high-quality analysis; (iii) openness to different perspectives to minimize the risk of "tunnel vision"; (iv) effective communication with the authorities and with the public; and (v) impact of the Fund's advice on members' policy decisions.

A great many initiatives have been undertaken to address the weaknesses in Fund surveillance highlighted in these reviews and, more generally, to strengthen the architecture of the international financial system. The strategy underlying these initiatives has three main strands: (i) to improve the Fund's diagnosis of risks and vulnerabilities confronting member countries; (ii) to develop standards and codes to assess and strengthen members' economic policy frameworks and institutions; and (iii) to enhance data disclosure and policy transparency to ensure accountability and induce better decision-making.

The initiatives have covered a wide spectrum, including data provision and dissemination, financial sector surveillance, standards and codes, debt sustainability assessments, surveillance of capital market developments, vulnerability assessments, focus and coverage of surveillance, surveillance in program countries, transparency, and a lending facility with eligibility limited to members with strong policies.

The paper argues that the present architecture of surveillance, as shaped by these initiatives, remains a viable framework. It further considers that the appropriate focus for the Fund is on taking advantage of the unexploited potential of this framework to contribute to better economic outcomes across the Fund's membership.

Nevertheless, further work could be beneficial in a number of areas. Among the issues worth exploring, the paper highlights (i) the appropriate calibration of the Fund's policy advice to reduce vulnerabilities in emerging market economies; (ii) consideration of cross-country experiences; (iii) ways to bring greater attention to the external effects of economic policies in systemically or regionally important countries; (iv) surveillance in program countries; (v) candor and transparency; (vi) ways to strengthen incentives for sound policies; and (vii) approaches to measuring and assessing the effectiveness of surveillance. The paper also suggests a set of next steps to move this policy agenda forward.

I. Introduction

1. During the 2002 biennial review of surveillance, Executive Directors examined the results of the many initiatives to strengthen Fund surveillance that had been introduced in the wake of the capital account crises of the 1990s. Directors concluded that substantial ground had been covered in enhancing surveillance and its effectiveness. Nevertheless, more needed to be done to ensure that the Fund's policy advice is sound and has impact. In this context, Directors saw merit in exploring further ways to strengthen the effectiveness of surveillance.¹

2. As background for this discussion, this paper takes stock of the findings of past reviews of surveillance, of various operational responses to these findings to date, and of issues that merit further attention. Section II briefly reviews the methodology and conclusions of the reviews of Fund surveillance carried out since the mid 1990s. Section III summarizes the operational responses to the shortcomings identified in these reviews. Sections IV examines priorities for the agenda looking forward. Section V discusses next steps.

II. Reviews of Fund Surveillance: Methodology and Main Conclusions

3. Fund surveillance has been examined in numerous reviews. In accordance with the mandate of the 1977 Surveillance Decision, the staff prepares regular, biennial reviews of the implementation of surveillance.² In addition, three special reviews have been conducted in the 1990s in the aftermath of major financial crises: a review of Fund surveillance in Mexico in 1993–94, an internal review of surveillance in the Asian crisis countries, and the 1999 external evaluation of Fund surveillance.³

4. These reviews have typically focused on examining the efficacy of the surveillance process. While there is general agreement that the effectiveness of surveillance needs to be judged by its impact on policy decisions and the development of sound economic institutions, both policy decisions and institution building are influenced by a variety of factors. It is thus hard to isolate the contribution of Fund surveillance. Given these methodological difficulties, reviews of Fund surveillance have tended to concentrate on various aspects of the surveillance framework and process. In particular, internal reviews have generally concentrated on the scope and quality of the coverage of Article IV consultations and on surveillance procedures, such as the frequency of Article IV consultations.

5. The external evaluation of surveillance offers a comprehensive assessment of a broad range of factors influencing the effectiveness of the surveillance process. It also provides some evidence on the impact of the Fund's policy advice based on selected case studies and on interviews with member country officials. The review of surveillance in Mexico in the run-up to the 1994–95 crisis and the review of surveillance in the Asian crisis countries focus on cases of failed surveillance and examine the factors that accounted for this failure. Biennial surveillance reviews have generally been comprehensive in their country coverage, but have inevitably concentrated on dimensions of effective surveillance that are well documented in country reports, notably factors affecting the quality of the Fund's analysis and policy advice.

6. The findings of successive reviews of Fund surveillance undertaken since the mid-1990s point to the following key ingredients for effective surveillance:

• Timely, comprehensive, and accurate information. The report on surveillance in Mexico in 1993–94 and the review of surveillance in the Asian crisis countries, among others, highlighted the impact of data limitations and gaps in disclosure on assessments by the Fund and market participants of the emerging risks in these countries.⁴
  
  
• Clear focus and adequate breadth of coverage, as well as high-quality analysis. All reviews of surveillance since the mid-1990s have identified weaknesses in the scope and quality of coverage. In particular, they have insisted on the need to strengthen financial sector surveillance, the analysis of capital flows, and the assessment of crisis vulnerabilities,⁵ including vulnerabilities arising from the exchange rate regime,⁶ institutional weaknesses,⁷ risks of contagion and spill-over from other countries,⁸ and concerns about the sustainability of public and external debt.⁹ Noting the expanded reach of surveillance, the 2002 biennial review of surveillance stressed the importance of keeping individual surveillance exercises focused on key issues to avoid diluting the Fund's policy advice.¹⁰
  
  
• Openness to different perspectives. Several reviews have highlighted the risk of "tunnel vision": the danger that the analysis underlying the surveillance exercise may miss key vulnerabilities, be unduly influenced by a country's good past performance or, in countries with Fund arrangements, by the agreed program framework.¹¹ These reviews have underscored the need to approach the analysis of a country's economic situation and policies from a fresh perspective. More generally, most reviews conclude that the Fund's policy advice could be strengthened significantly through more systematic consideration of perspectives gained from cross country experiences, the results of multilateral surveillance, the views of market participants, and relevant outside experts.¹²
  
  
• Effective communication with the authorities and with the public. Various elements of effective communication figure prominently in the report on surveillance in Mexico during 1993–94, in the external evaluation of surveillance, and in the 2002 biennial surveillance review. The first two reports stress the importance of candor in the policy dialogue with the authorities, and point to the risk that the collegial nature of Board discussions may turn peer pressure for adoption of sound policies into peer protection of existing policies.¹³ The review of Fund surveillance in Mexico in 1993–94 emphasizes the role of management and the Board in encouraging frank discussions of risks.¹⁴ The external evaluation of surveillance and the 2002 biennial surveillance review highlight the need for sensitivity to political constraints and the benefits of elaborating alternative policy proposals that take into account the political and social realities in the member country.¹⁵ The role of transparency vis-à-vis the public in enhancing surveillance has been extensively debated since the 1997 surveillance review, when Press Information Notices were introduced; reflecting on the initial experience, the report on the external evaluation of surveillance argued for routine publication of all Article IV staff reports.
  
  
• Impact of the Fund's advice on members' policy decisions. Both the report on Fund surveillance in Mexico and the external evaluation of surveillance recognize that member countries' willingness, or unwillingness, to act may define clear limits to the Fund's efforts to enhance the effectiveness of surveillance. The report on surveillance in Mexico during 1993–94 reasoned that better information, policy analysis, and communication would likely not have altered policies in Mexico, although it could have contributed to a better appreciation of the mounting risks, including the risks of contagion, among the membership.¹⁶ The external review of surveillance concluded that "while Fund influence will no doubt vary by country and within a country over time, depending on particular circumstances, it surely has to be accepted that surveillance is hardly ever going to be a primary influence on a country's policy actions." ¹⁷

III. Operational Responses

7. A great many initiatives have been undertaken since the mid-1990s and, particularly, since the Asian crisis to address weaknesses in Fund surveillance and to strengthen the architecture of the international financial system. The strategy underlying these initiatives consists of three main strands: (i) to improve the Fund's diagnosis of risks and vulnerabilities confronting member countries, including vulnerabilities in the financial sector; (ii) to develop standards and codes to assess and strengthen members' economic policy frameworks and institutions; and (iii) to enhance data disclosure and policy transparency to induce better decision-making by both private and public economic agents.¹⁸ The steps taken under this broad strategy have affected a wide range of Fund policies and practices.

8. Data provision to the Fund and data dissemination to the public. Staff has been instructed to assess the adequacy of members' data provision in all Article IV consultation reports, and, beginning with a comprehensive examination in 1995, data provision to the Fund for surveillance has become subject to regular reviews. In parallel, the data standards initiative has promoted better data provision to the public. The 2002 review of data provision to the Fund for surveillance noted significant progress in the provision of core data, even though about one third of the cases discussed in Article IV staff reports still indicate data inadequacies. Looking forward, the principal gaps and shortcomings to be addressed relate to the extent, timeliness and periodicity of reporting and disclosure of data on international reserves, public sector and external debt, financial sector indicators, and other measures of balance sheet risks. There is also room for improvement in the quality of reporting on creditor exposure.¹⁹ The next reviews of data provision to the Fund and the data standards initiative are scheduled for this year.

9. Financial sector surveillance. The Financial Sector Assessment Program (FSAP), which was established in 1999, has become the primary vehicle for conducting financial sector assessments as input to Fund surveillance.²⁰ A significant number of countries has been covered so far, but participation, which is voluntary, has been somewhat uneven across regions and among countries where an in-depth assessment of the financial sector would be particularly useful. Incomplete coverage and the low frequency of assessments have raised questions about the conduct of financial sector surveillance in the absence of, or in between, FSAP participation. The report for the 2003 review of the FSAP calls for more flexible tools for ongoing financial sector surveillance and FSAP follow-up, such as focused FSAP updates, participation of specialists in Article IV missions, and off-site monitoring through financial soundness indicators. It also argues for greater variation in the scope of FSAP assessments according to country circumstances, a reduction in the annual number of FSAP assessments and reassessments, and close management of the country selection process in order to free up resources for more continuous financial sector surveillance in countries with material financial sector vulnerabilities or countries of systemic importance. It nevertheless envisages that a broad range of countries would participate in the FSAP.²¹

10. Standards and codes. Standards and Codes were introduced to provide a systematic framework for organizing and presenting information on institutional issues, to assess members' institutional frameworks against established benchmarks on a voluntary basis, and to use these assessments (i.e., Reports on the Observance of Standards and Codes, ROSCs) to help members strengthen their institutions.²² Like FSAPs, ROSCs offer critical input to surveillance in areas that have not been part of its traditional mainstay but have played an important role in recent crises. While coverage of countries and areas has expanded significantly, it is far from complete. The report on the 2003 review of the standards initiative—while retaining the voluntary nature of ROSCs—supports a more selective approach to coverage, focusing on countries and areas with the highest return to domestic and international financial stability and to members' institutional capacity. The review also underscores the need for a sharpened focus of individual ROSCs on the main conclusions and recommendations, and for greater attention to, and flexible modalities for, ongoing follow-up.²³

11. Debt sustainability analysis. In order to strengthen the staff's debt sustainability analyses, a new analytic framework for debt sustainability assessments was developed; it was adopted in June 2002. The framework is to be used for all requests for use of Fund resources in the General Resources Account and in the context of surveillance of members with significant market access.²⁴ For these countries, it has become a key reference point for the staff's work in this area. The forthcoming review of the debt sustainability framework, which is planned for mid-2003, will take stock of how the framework has been applied, particularly in policy discussions during program negotiations and Article IV consultations, and will explore possible further improvements to the framework.

12. Capital account developments and market perspectives. The creation of the International Capital Markets Department and the establishment of the Capital Markets Consultative Group have significantly boosted the Fund's analysis of capital market developments and have expanded the dialogue with market participants. The insights gained from multilateral surveillance of international capital markets and from market intelligence are increasingly feeding into bilateral surveillance exercises, strengthening the analysis of capital account developments and broadening the staff's perspectives on individual countries' vulnerabilities. Nevertheless, further efforts are needed to ensure that this is done systematically.

13. Vulnerability assessments. Efforts to improve the analysis of crisis vulnerabilities have proceeded on two tracks.

• One track involved the analytical underpinnings of vulnerability assessments and the development of tools. This work has covered, inter alia, the development and evaluation of Early Warning System (EWS) models; a reassessment of exchange rate regimes in an environment of increased capital market integration; ²⁵ analyses of channels of contagion; and the development of debt- and reserve-related indicators of external vulnerability.²⁶
  
  
• The other track led to the development of an operational framework for high frequency assessments of crisis vulnerabilities in emerging market countries that are potentially exposed to sudden shifts in market sentiment.²⁷ This process was initiated in May 2001 and has evolved into comprehensive quarterly vulnerability assessments complemented by more streamlined interim updates. These assessments pull together information on global economic and market developments from multilateral surveillance, the views of selected functional departments on vulnerabilities in specific sectors, market intelligence, and analyses of vulnerabilities and financing scenarios prepared by area departments. The exercise involves substantial integration of bilateral and multilateral surveillance and provides a forum for drawing on the institutional wisdom from experiences in various countries, thus offering a valuable confrontation of different perspectives

14. Operational guidance on focus and coverage of surveillance. Given the expanded reach of surveillance and the risk of losing focus, the operational guidance for staff following the 2002 biennial surveillance review contains clear language on the focus and scope of individual surveillance exercises. This guidance reconciles focus and breadth by emphasizing the need for selectivity among a range of potentially relevant issues, based on country-specific circumstances and the Fund's hierarchy of concerns.²⁸ In addition, the guidance note contains instructions to ensure adequate quality of coverage. Guidance on effective communication emphasizes candor, sensitivity to the political and social implications of alternative policy solutions, transparency, and consideration of the impact of past policy recommendations.

15. Surveillance in program countries. Responding to the weaknesses of surveillance in program countries identified in the 2002 biennial surveillance review, the operational guidance note for staff on surveillance devotes substantial attention to the role and nature of Article IV consultations in a program context. Specifically, the guidance underscores the need for independent assessments that offer a fresh perspective on economic conditions and policies and provide a basis for a reassessment of the program strategy. Consultation cycles for program countries were made more flexible to allow consultations to take place at a time when they are most beneficial in the program context.²⁹ In addition the vulnerability assessment exercise for emerging market countries provides a framework for more continuous, independent cross-checks of vulnerabilities in program countries.

16. Transparency. Since the first release of Public Information Notices (PINs) on the conclusion of Article IV consultations in 1997, the Fund has successively broadened its transparency policy. Making Article IV consultation documents, including Financial System Stability Assessment reports and ROSCs, available to a broader audience is intended to strengthen surveillance by increasing the Fund's accountability, by facilitating better risk assessment in the private sector, and by helping mobilize support for policy actions in member countries. Executive Directors discussed the pros and cons of moving from voluntary to presumed publication of Article IV and UFR staff reports at the review of the Fund's transparency policy in September 2002. While roughly 60 percent of Article IV³⁰ and 50 percent of UFR staff reports are presently published,³¹ a substantial number of Fund members have been reluctant to move to a policy of presumed publication of staff reports. The Board agreed to reconsider the matter in June 2003.³²

17. Creating incentives for strong policies: the CCL. The Contingent Credit Lines (CCL) were established in 1999 as an instrument of crisis prevention. By linking eligibility for the CCL to a member's policy performance, the facility was expected to create incentives for the adoption of strong policies and adherence to internationally agreed standards, thus strengthening Fund surveillance. To date, no member country has applied for the CCL. The Board is reviewing possible modifications of the facility, as well as the option of letting it expire while exploring improvements in existing Fund lending instruments and surveillance.

IV. The Agenda Ahead

18. The initiatives undertaken in recent years define the core of the Fund's efforts to strengthen the effectiveness of surveillance. As outlined in the previous section, the initiatives underway are designed to help improve the effectiveness of the individual elements of the surveillance framework. On balance, the staff believes that the present architecture of surveillance, as shaped by these initiatives, remains a viable framework. Thus, the appropriate focus for the Fund is on taking advantage of the unexploited potential of the present framework to contribute to better economic outcomes across the Fund's membership.

19. Nevertheless, there are a number of areas where further work is needed, beyond those issues and initiatives that are subject to periodic review (e.g., data provision, FSAP, standards and codes). This section offers a preliminary discussion of some areas where further evolution in the Fund's approach would be desirable. The list of issues considered is not exhaustive. Among the issues that are worth exploring are: (i) the appropriate calibration of the Fund's policy advice to reduce vulnerabilities in emerging market economies; (ii) a more systematic consideration of cross country experiences as input into bilateral surveillance; (iii) ways to bring greater attention to the external effects of economic policies in systemically or regionally important countries; (iv) strengthening the Fund's capacity to bring a fresh perspective or a second look to decisions on strategy in program countries; (v) candor and transparency; (vi) possible ways to strengthen incentives for better policies and preemptive responses to emerging vulnerabilities; and (vii) approaches to measuring and assessing the effectiveness of surveillance.

Calibration of Policy Measures to Reduce Vulnerability

20. While advising member countries on policies to reduce vulnerabilities to external shocks has always been a central task of Fund surveillance, the substance of this policy advice has evolved with the changing nature of external crises. Following the emerging market crises of the late 1990s, a consensus has emerged that resilience to external shocks requires a sustainable—in most cases flexible—exchange rate regime; an appropriate level of liquid and unencumbered foreign reserves; a prudent level and composition of external obligations, including private sector obligations; a robust financial system; a prudent level and composition of public debt; sound budgets that leave room for maneuver; and effective social safety nets.

21. Questions about the appropriate calibration of the Fund's policy advice in these areas remain. For instance, the experience in Argentina and other countries suggests that prudent debt levels may be much lower than previously thought. The risks present in the composition of the debt stock in many countries have highlighted again the importance of debt management and borrowing strategies that provide more insurance against a range of potential shocks. Events in Uruguay and other countries have focused new attention on the magnitude of risks in highly dollarized economies, and on ways to mitigate those risks. Financial sector crises in a number of countries have shown the importance of factoring in implicit or explicit contingent public liabilities in analyses of debt sustainability. Work is underway on a number of issues to improve the analytical basis for our policy advice in these areas.

22. The Fund's policy advice could also benefit from a more systematic effort to analyze political factors influencing the ability to establish and maintain shock absorbers. Experience in a number of recent crises suggests that the ability to implement policies that reduce vulnerabilities depends on the robustness of domestic political institutions and the capacity to mobilize political consensus in favor of needed adjustment measures. Given the importance of these political factors, the staff has proposed in its response to the IEO's evaluation of prolonged use of Fund resources, more systematic analysis of, and reporting on, such factors. This effort would proceed in an evolutionary manner, building on treatment already accorded to such issues in post-conflict cases and in countries facing severe governance issues.³³

23. A well-defined set of criteria for sound policies in relevant areas could provide a framework for more objective and transparent assessments of members' vulnerabilities and policies. For instance, criteria such as the revised eligibility criteria proposed in the paper on the review of the CCL could offer helpful guidance to the staff for their assessments and greater clarity to the authorities about the standards for these assessments. Underpinned by sound analytical studies and used with due regard to country-specific circumstances, such criteria could contribute to a more systematic effort to use surveillance more effectively to help member countries put in place larger cushions against shocks, and build greater resilience to deal with the challenges that accompany economic and financial integration.

Consideration of Cross-Country Experiences

24. There is significant scope for integrating insights from cross-country experiences more systematically into surveillance exercises. The benefits of using the wealth of Fund's cross-country experience to gain a broader perspective on economic conditions and policies in individual countries have been highlighted in numerous reviews of surveillance.³⁴ The value of cross-country studies and comparisons included in Article IV consultation documents and WEO reports, the benefits of cross-country fertilization from the inter-departmental review process, and the experience with the framework for vulnerability assessments in emerging market countries all support this contention. Notwithstanding progress made on this front in recent years, more remains to be done to ensure that insights from cross-country studies and past experiences of Fund members are incorporated systematically into surveillance, particularly at an early stage of the staff's analysis. A strengthened inter-departmental review process offers one avenue for bringing greater cross-country input to individual surveillance exercises.

External Effects of Policies in Systemically or Regionally Important Countries

25. The systemic impact of large countries' policies deserves greater attention in Fund surveillance. Economic conditions and policies in the Fund's largest members can affect developments in other member countries through a multitude of channels, and are, therefore, matters of particular importance to the membership of the Fund.³⁵ This conclusion has been underscored in various surveillance reviews. It is also supported by the work on the World Economic Outlook reports, the World Economic and Market Developments sessions, the Global Financial Stability reports, and the vulnerability assessments in emerging market countries, which do pay close attention to economic and financial spillovers from advanced countries.

26. However, there has been a lively debate over the years about the impact of the Fund's advice on its largest members. Some have argued that these members—especially industrial countries—have been less receptive to Fund advice than other members as they are unlikely to seek financing from the Fund, and are, therefore, less constrained to take account of the views of the international community in the formulation of their policies. In light of these concerns, the Fund needs to take a close look at possible ways to raise further the profile of the externalities of large countries' policies, in both multilateral and bilateral surveillance.

Surveillance in Program Countries

27. Surveillance in program countries needs to become more independent from program considerations. Calls for greater independence of surveillance in program countries, and weaknesses of surveillance identified in the review of prolonged use of Fund resources resonate with the concerns about surveillance in program countries expressed in the 2002 biennial surveillance review.³⁶ Specifically, they confirm that Article IV consultations have not always been used effectively to step back from the program framework and reconsider the diagnosis of the problem and the policy strategy. The operational guidance following the 2002 biennial surveillance review constitutes a first step toward a regime that emphasizes the independence of the surveillance exercise from the program framework, and the need for a fresh look at economic conditions and policies.

28. A number of procedural changes are now being put in place to facilitate a fresh perspective in these cases, both in the Article IV process and on a more ongoing basis, including changes that would bring in fresh pairs of eyes to the Article IV process before or during Article IV missions. These include:

• a more systematic effort to ensure that the surveillance process is timed when it is most important to bring a broader perspective to inform program strategy;
  
  
• a more systematic process of ex post assessments of use of Fund resources and strategic planning, drawing lessons from previous programs and their implications for future Fund engagement, in line with the conclusions of the Task Force on Prolonged Use of Fund Resources; such a process could be complemented by greater use of outside experts, when needed;
  
  
• strengthened and differentiated review procedures for Article IV and UFR documents, including more systematic use of pre-brief meetings;
  
  
• augmentation of resources for Article IV consultations with staff from the concerned area department and/or functional departments, when needed to facilitate comprehensive reassessment of economic conditions and policies.

Also, as noted by Directors at the follow up discussion to the 2002 biennial surveillance review in July 2002, "management retains the prerogative to alter the composition of Article IV consultation teams to meet the needs of particular country circumstances." ³⁷

29. The role and conduct of surveillance in low-income, program countries deserves further consideration. In these countries, the economic policy agenda is framed by the Poverty Reduction Strategy Papers (PRSPs). The relation between the PRSP process and surveillance, including such elements as a backward-looking assessment of PRSPs' macroeconomic frameworks in Fund surveillance and the forward-looking role of Article IV consultations in disseminating the Fund's views on macroeconomic and macroeconomically-relevant policies to a wide range of stakeholders, requires a careful analysis.

Candor and Transparency

30. Candor and transparency are essential for effective surveillance. Virtually all reviews of Fund surveillance and all operational guidance notes have underscored the need for candor in the Fund's assessments of policy and in its policy advice. There has also been growing acceptance of the value of transparency by the Fund. Greater transparency in the Fund's surveillance activities can help increase the accountability of the Fund and of member governments, while providing the private sector with better and more continuous information for decision making.

31. However, there is inevitably a degree of tension between the importance of candor and clarity in the Fund's policy assessments and prescriptions in its dialogue with its members and the objectives of greater transparency. From the very beginning, the discussion on the Fund's transparency policy has been influenced by concerns about adverse effects of greater transparency on the candor of the staff's assessments and policy advice.³⁸ The 2002 review of the Fund's transparency policy noted that the prospect of publication may account for the large volume of corrections in Article IV staff reports, and may have encouraged the authorities to avoid sensitive issues—including discussion of adverse scenarios and potential policy responses under such scenarios—and staff to shade the presentation. Looking ahead, a key challenge for the Fund is to ensure full candor in the staff's presentation of its diagnoses and prescriptions to the Board, which is a sine qua non for effective surveillance. Subject to that, further efforts should be made to continue to boost rates of publication of Article IV and UFR staff reports.

Incentives for Sound Policies

32. The scope for strengthening incentives for sound policies may be worth examining. A variety of proposals have been advanced within the Fund and by outside observers to give Fund surveillance more traction. Most of these proposals fall into two categories: those that suggest the Fund issue public assessments of countries' policies with a more explicit rating or evaluation relative to other countries or against some explicit standard; and those that propose a tighter link between the quality of a country's policies and access to Fund resources.

33. However, the proposals put forward raise difficult questions. Publicized ratings of members' policies would place high demands on the standards or criteria used for such assessments.³⁹ In particular, these standards would have to be sufficiently specific to limit subjective interpretation, broad enough to cover all relevant policy areas, and applicable across the membership or, at least , across specific, well-defined subsets of the membership. Such standards are difficult, if not impossible, to design. Furthermore, explicit and publicized rating of a broad range of policies against these standards would risk weakening the focus and the candor of the policy dialogue with member countries.

34. Formal, publicized schemes linking policy assessments or ratings to access to Fund resources raise similar questions about the design of eligibility criteria and the impact of the staff's rating function on the policy dialogue with members. In addition, they raise questions about the signals associated with entry and exit. Less formal schemes without publicized pre-qualification could avoid these problems and still help clarify, and possibly strengthen, the links between present policies and the conditions for future access to Fund resources that exist already implicitly. However, such schemes offer less certainty about access to Fund resources than formal pre-qualification schemes, and entail, therefore, less strong incentives to pursue needed policy reforms. Notwithstanding these difficult questions, it may be worth exploring possible options for strengthening incentives for better policies in member countries through surveillance and greater selectivity in the Fund's lending policies.

Assessing the Effectiveness of Surveillance

35. Assessing effectiveness is a critical element in the continuing quest to enhance the effectiveness of surveillance. But it is not an easy task, given the impossibility of defining counterfactuals (i.e., what would members' policies have been in the absence of Fund surveillance?) and the difficulty of establishing causal links between provision of advice by the Fund and implementation of particular policies by member countries. Given these methodological difficulties, analyses of the effectiveness of Fund surveillance will need to continue to rely largely on the approaches employed in past internal and external reviews:

• Assessments of initiatives to improve specific aspects of the surveillance process. These assessments help ascertain to what extent, and how well, these initiatives are in fact being implemented, but they are limited by their narrow focus.
  
  
• Case studies of selected countries. Such studies have proven to be a useful tool to gain insights into numerous facets of the conduct of surveillance in particular countries.⁴⁰ However, given their limited country coverage these studies should not be expected to provide a global assessment of the effectiveness of surveillance across the membership.
  
  
• Surveying country officials and staff through seminars or interviews. These surveys can offer useful insight into aspects of surveillance that are difficult, if not impossible, to examine on the basis of country documents, such as the effectiveness of communication, and into member countries' perspectives on the Fund's policy advice and on the factors affecting its impact. However, these surveys inevitably reflect subjective views rather than objective facts.

36. Beyond these approaches, it will be interesting to evaluate the new requirement in Article IV consultation reports concerning the effectiveness of surveillance. Since the conclusion of the 2002 biennial surveillance review, Article IV consultation reports are expected to contain a brief assessment of the authorities' response to the key policy challenges addressed in previous Article IV consultations.⁴¹ These assessments are intended to raise questions about the effectiveness of surveillance and the factors affecting the impact of the Fund's policy advice in individual Article IV consultations. The experience with these assessments and possible ways of enhancing them could be considered in the context of the next biennial surveillance review.

37. The foregoing is not an exhaustive list of areas in which further progress needs to be made. Work is continuing on a number of initiatives that are subject to regular review (e.g., FSAP, financial soundness indicators, standards and codes). In addition, a variety of other areas are subject to ongoing thinking: for instance, helping missions develop an appreciation for the nature and characteristics of political constraints; enhancing the potential contributions of Fund missions to authorities' efforts to build a consensus in favor of needed reforms; and providing mission chiefs with the necessary training to boost their ability to handle the political and external relations dimensions of their assignments.

38. While progress in the areas outlined above would help strengthen the surveillance framework further, it would be unrealistic to expect policy recommendations under Fund surveillance to commonly be the prime driver of policies in member countries. The paper for the 1997 biennial surveillance review noted quite appropriately "that Fund surveillance is only one influence, and generally not the predominant one, on members' policies and performance."⁴² And the external review of Fund surveillance concluded that "the best the Fund can realistically hope to do is contribute over time to building or maintaining a consensus across the membership on the broad policy framework; and to tip the balance in situations where policies are in fact genuinely in the balance."⁴³

V. Next Steps

39. To move forward on the policy agenda presented in the previous section, several steps are proposed for the period between the 2003 and 2004 Spring meetings.

40. The Board agenda between now and the 2003 Annual meetings should include the following discussions:

• On the analysis of risks, vulnerabilities, and shock absorbers, reviews of the debt sustainability framework adopted in July 2002, and of progress on financial soundness indicators; a discussion of a liquidity management framework; as well as seminars on the balance sheet approach to financial crisis, and on macroeconomic policies and financial stability in dollarized economies.
  
  
• On transparency, a review of the Fund's transparency policy.
  
  
• On the links between surveillance and access to the Fund's financial resources, a discussion on contingent lending, precautionary arrangements, and surveillance, which would follow up on the recent review of the Contingent Credit Lines.

41. Also, in the period to the 2003 Annual Meetings, guidance to staff on surveillance in program countries would be promulgated by management. Such guidance would cover the range of proposals described in paragraph 28, and would complement the Operational Guidance Note for Staff Following the 2002 Biennial Surveillance Review (SM/02/92, 9/13/2002). It would be circulated to the Board for information.

42. In the months immediately following the 2003 Annual Meetings, an informal Board meeting would be held to frame the 2004 biennial surveillance review (2004 BSR), which would take place in spring 2004. This meeting would help to set the review's agenda. It would also be expected to provide input into the nature and scope of assessments of the effectiveness of Fund surveillance (e.g., surveys, case studies) to be undertaken in the 2004 BSR. This informal Board meeting could be complemented by a seminar with outside experts on pertinent surveillance issues, such as the role of Fund surveillance in systemically important countries.

43. As part of preparations of the 2004 biennial surveillance review, Executive Directors may wish to consider whether there would be merit in the Board undertaking a review of its role in Fund surveillance, particularly its role as a source of encouragement for the adoption of sound policies by member countries.

VI. Issues For Discussion

44. Executive Directors may wish to address the following questions:

• What can Fund surveillance be expected to achieve (paragraph 38)?
  
  
• Is the diagnosis of the key ingredients of effective surveillance correct (paragraph 6)?
  
  
• Does Section III provide a fair summary of the progress made to strengthen Fund surveillance since the mid-1990s as well as of previously identified gaps and shortcomings in a number of areas (e.g., data provision, financial sector surveillance, standards and codes)?
  
  
• Do Directors agree that the present architecture of Fund surveillance, as shaped by the initiatives taken since the mid-1990s to strengthen surveillance, remains a viable framework, and that the focus of the work ahead should be on exploiting fully the potential of this architecture (paragraph 18)? Alternately, do Directors see a need for fundamental modifications to this architecture?
  
  
• Going one step further, should further efforts to strengthen surveillance focus on the issues listed in paragraph 19? Alternately, should priority be given to consideration of some other matters?
  
  
• Are the procedural steps sketched in paragraph 28 a valuable complement to the measures adopted at the conclusion of the 2002 biennial surveillance review to bring a fresh perspective to Article IV consultations in program countries?
  
  
• Do Directors agree that assessments of the effectiveness of Fund surveillance will have to continue to rely primarily on the various approaches used in past reviews—i.e., review of policy initiatives, case studies, surveys (paragraph 35)? If not, what other approaches would they recommend using? Is there value in considering more substantial treatment of the effectiveness of surveillance in individual Article IV staff reports, building on the step adopted at the conclusion of the 2002 biennial surveillance review?
  
  
• What are Directors' views on the next steps laid out in Section V?