Recent years have witnessed an increase in the frequency of currency and balance of payments crises in developing countries. More important, the crises have become more virulent, have caused widespread disruption to other developing countries, and have even had repercussions on advanced economies. In some cases, the crises have also been quite unexpected and have affected countries with strong economic performance. Even when economies were perceived as more vulnerable to crises, the timing of speculative attacks has often caught observers and policymakers by surprise. This course of events, in part related to the increased volume and volatility of private international financial flows, has stimulated research on the prediction of balance of payments crises.¹

Clearly it would be desirable to have a system for identifying looming crises so that steps could be taken to avoid them. With the increasing liberalization of capital movements and the globalization of financial markets, unsustainable policies may at times be disciplined by an early reaction of financial markets, while on other occasions, they may be exacerbated by strong capital inflows that are later reversed as investor sentiment suddenly changes. The resulting crisis may itself be associated with economic disruption and welfare losses that go beyond those justified by the economic fundamentals, making crisis prediction and prevention an important objective. This paper aims to assess the progress of attempts to develop systematic empirical frameworks for predicting balance of payments crises.

A number of projects were initiated after the Mexican crisis of December 1994 to design an early warning system (EWS).² These efforts have multiplied since the onset of the Asian crisis.³ Early warning systems apply some statistical method to predict the likelihood that a country will face a currency or balance of payments crisis, defined in a precise way, over a given time horizon.⁴ Their frameworks focus on economic and financial variables that are likely to provide an early indication of a vulnerable balance of payments position or unsustainable exchange rate level. These variables typically include indicators of domestic macroeconomic imbalances and banking sector weakness, such as fiscal deficits and domestic credit growth; overvaluation of the exchange rate, such as measures of relative prices or costs, the current account deficit, and export growth; and of external vulnerability and contagion, such as external liabilities relative to international reserves and the incidence of crises in other countries. In sum, the range of possible variables encompasses both the fundamentals of the domestic economy and the vulnerability to changes in market sentiment and the global environment.

To predict crises, their causes must be clearly understood. Two competing strands of theories--called first- and second-generation models--are reviewed in this paper (Section II). The first focuses on the consequences of such policies as excessive credit growth in provoking a depletion of foreign exchange reserves and making a devaluation inevitable. The second emphasizes the trade-offs between internal and external balance that the policymaker faces in defending a peg: if the costs in terms of unemployment or financial sector fragility of using interest rates to defend the exchange rate are too high, the authorities may choose to devalue. The endogeneity of the policy choice, together with the fact that expectations affect the trade-off faced by the government, raise the possibility that crises may be self-fulfilling. For example, investors who expect a devaluation may, by raising the cost of servicing outstanding debt, make a crisis more likely. Crises are likely to be more difficult to predict in this latter case, since they depend in part on investor sentiment. Finally, there is some evidence that crises may also be triggered by contagion effects.

Ideally, an early warning system should try to identify situations that pose a distinct risk of a crisis affecting the external payments of a country or a large devaluation of its currency. But what are the events or situations that a system should provide warnings about? In some cases, such as the abandonment of a peg, a collapse in the value of the domestic currency, or a default in international payments, there is no ambiguity in identifying the event. But should the definition also include cases that involve sizable but unsuccessful currency attacks, such as Argentina in 1995 or Brazil in 1997? It would seem apparent that the economic authorities should be alert to any situation in which the external position is vulnerable; this also includes currency attacks that ultimately result in "close calls" but no devaluation. In fact, whether an attack ends in a currency crisis or not may well depend on the resolve of the government in defending its currency, on accompanying policy measures, and on the availability of external financing rather than on the conditions that originated the attack. Adopting this broader definition of crisis may make the problem of prediction more difficult, to the extent that failed attacks are truly different from currency collapses, but the broader measure will be more useful to policymakers. The paper discusses the specific empirical implementation of alternative crisis definitions (Section III).

How well have existing early warning systems done in predicting currency or balance of payments crises? Section IV examines the performance of three representative frameworks (plus an extension developed in the paper). The short answer is, they have had mixed success. The predictions from the most promising models contain substantial information about the risks of crisis, but they often provide false alarms. A representative model produces a warning signal (indication that a crisis is approaching) in about 50 percent of the cases in which it should have signaled because a crisis did indeed happen at some point over the following two years. This means, in a typical case, that during the two years before a crisis the model issued a warning signal on about 12 of the 24 months. But the warnings issued by the typical early warning system model are not very reliable. About 60 percent of the times that the typical model issued a warning, no crisis occurred during the following two years.

Some care should be taken in interpreting false alarms as "mistakes" made by the models. First, the authorities in these countries may have taken policy measures to ward off a looming crisis. More generally, it may be that these false alarms occurred in situations in which countries were truly at risk but did not suffer a crisis because of favorable external developments or even good luck. As discussed in Section II, theoretical models that emphasize self-fulfilling crises imply that, for a range of values of fundamentals, crises may happen but are not inevitable. The best an early warning system could do would be to assign higher probabilities of crisis to countries that are more vulnerable to a shift in expectations. This would still be useful information, as the model would essentially generate correct warnings, even though many of these warnings would be counted as false alarms.

Models tend to perform much more reliably for the historical period for which they were designed and estimated ("in sample") than for later periods ("out of sample"). This is particularly true when the out-of-sample period considered is one that took place after the research was conducted, and thus could not have been, implicitly or explicitly, incorporated into the design of the system. For example, Section IV studies the performance in relation to the Asian crisis of three representative models that were designed and estimated before 1997, in addition to another one developed after the Asian crisis. In the former three cases, the researchers did not know about the factors present in the unfolding of the Asian crisis, so that those events did not affect their choice of variables or any other aspect of the specification of the model. Applying this more stringent test, the out-of-sample and out-of-researcher-awareness period, the models examined do indeed tend to perform somewhat more poorly. The most successful model suffers some degradation in performance out of sample. The number of correct warning signals during the two years prior to the crises of 1997 drops to about 34 percent. However, the number of false alarms (when the model indicated that a crisis was approaching but no crisis took place during the following two years) also drops to about 51 percent.

Although it is possible, in principle, to calibrate models to forecast a larger fraction of the emerging crises, this would imply a cost in terms of the reliability of the signals. That is, the number of false alarms--predicted crises that do not materialize--rises as the number of missed crises falls. The precise calibration of the early warning systems thus depends on the relative cost associated with each type of error. In principle, since the purpose of such a system is to anticipate currency crises, one would prefer a specification that is fairly sensitive to symptoms and would not fail to issue warnings very often when a crisis is indeed approaching. But one should be aware that, in such cases, even the best-performing model is bound to produce its share of false signals--currency attacks, and even currency collapses--that are predicted by the model but will fail to materialize.

What is the standard against which to judge the value of forecasts generated by an early warning system? Available measures of expectations by market participants display a poor record with respect to recent events in anticipating crises. For example, a direct measure of exchange rate expectations from the surveys carried out by the Financial Times Currency Forecaster (which surveys banks, multinational companies, and professional forecasters on a monthly basis) suggests that markets had little inkling of the approaching crises in Mexico in 1994, or in Thailand in 1997 (Figure 1).⁵ The 30-day-ahead exchange rate forecast showed virtually no increase in expectations of a devaluation in the months just before the currency collapsed nor did the 12-month forecast. In fact, in the case of Mexico, the 12-month-ahead expectations turned more pessimistic in the second half of 1995 and in 1996, when Mexico was actually already emerging from the crisis. In the Asian crisis, countries' expectations of exchange rate depreciation went up sharply in October 1997, when markets in Hong Kong Special Administrative Region (SAR) and throughout the region displayed significant turbulence, but abated in November 1997, just before the collapse of the Korean won and the renewed weakness of some other currencies in the region.

Market expectations as embedded in asset prices also have not provided clear signals in recent episodes. Interest differentials (the spread between domestic and foreign interest rates, themselves a function of monetary policy and other factors as well) did not widen significantly prior to the Mexican crisis.⁶ Spreads on Brady bonds and Eurobonds, instead of providing an early signal of worsening of confidence, appear to have widened only at the time currency pressures were already well under way. In the Asian crisis, spreads hardly increased in the months prior to the flotation of the baht, and rose only in late October 1997, four full months after the event.⁷

This suggests that, although early warning systems are hardly infallible, they can contribute to the analysis of external risks in emerging countries (Section V). While they look at many of the same economic and financial variables that most analysts do, their strength is that they process the information contained in the rather large number of relevant variables in a systematic way that maximizes their ability to predict currency and balance of payments crises, based on the historical experience of a large number of countries. Often an early warning system can translate this information into a composite measure of vulnerability. Being based on a well-defined methodology, it is less likely to be clouded by preconceptions about the expected economic performance of particular countries. Furthermore, an early warning system can be a useful tool to rank the relative vulnerability of a group of countries, which is more difficult to assess on the basis of a country-by-country analysis.