Independent Evaluation Office Announces Release of Report on the Evaluation of Technical Assistance Provided by the Fund

French (72 kb, pdf file) | Spanish (48 kb, pdf file)

This evaluation examines the technical assistance (TA) provided by the IMF to its member countries. It is based on desk reviews of a sample of countries, cross-country data on TA, six in-depth country case studies¹ (including field visits and interviews with public officials and donors, reviews of past evaluations, and interviews with IMF staff and other stakeholders).

To derive operational lessons, the evaluation unbundles TA into the following three stages:

• Prioritization: How are countries' TA needs identified? What can be done to make the process more strategic so as to increase the relevance of IMF TA activities?
• The delivery process: What factors influence the effectiveness of the various modalities for TA delivery?
• Monitoring progress and evaluating impact: How is progress tracked and what factors contribute to the impact of TA?

A. Main Findings

How are TA priorities set?

• The IMF provides annually about 300 person-years of direct TA, amounting to about 10 percent of the gross administrative budget of the institution. Seventy percent of IMF TA is directed to countries with per capita income below $1,000, thus IMF TA is well targeted to low-income countries. The volume of TA provided to countries is also positively associated with having a PRGF- or EFF-supported program, the amount of external financing available, and whether the country is emerging from conflict.
  
  
• There is a weak link between TA priorities and Poverty Reduction Strategy Papers (PRSPs) or with key policy issues identified in Article IV consultations. In most cases, the PRS process has still not been able to clearly identify major capacity-building needs to be taken up by TA. This is a major shortcoming because the PRSP was expected to become the vehicle to provide guidance on broad priorities for the IMF in low-income countries. Thus, TA activities do not appear to be guided by a medium-term country-based policy framework that would set priorities in the IMF areas of expertise across sectors, program needs, and institutional initiatives, and that would balance TA demands stemming from short-term policy needs with medium-term capacity-building needs.

The process of TA delivery

• Country officials have generally been satisfied with the resident experts provided by the IMF, particularly their hands-on role in training and coaching, accessibility, and emphasis on teamwork. However, the evaluation finds the involvement of the authorities in the preparation of terms of reference (TOR), particularly for long-term experts, to be generally passive. This tends to reduce ownership and often masks important differences in expectations between the authorities and staff about final objectives.
  
  
• Country officials suggested that more informal and iterative discussions on a broader set of options before the wrap-up meetings at the end of a TA mission would contribute to enhancing ownership of recommendations by ensuring that constraints on the ground are taken into account.
  
  
• The evaluation found many instances of weak coordination between the IMF and donors working in similar areas. While coordination with donors is ultimately the authorities' responsibility, this is often not the case owing to weak institutional capacity and the fact that the PRSP is not yet sufficiently operational to play such a role in most low-income countries. As a result, the burden of coordinating donors' efforts often falls to a major donor or multilateral institution. Moreover, when the involvement of donors is strong in a particular country, and the IMF provides a relatively small fraction of TA, it is not always possible or even appropriate for the IMF to provide leadership in coordinating overall TA efforts even in its core areas of expertise. However, it should still seek to coordinate better with donors working in similar areas and, in low-income countries, should help governments make the PRS an effective vehicle on which it can align its own efforts.

Monitoring the impact of TA and evaluating factors influencing it

• The case studies show that progress has generally been achieved in enhancing the technical capacity of the agencies that the IMF typically supports. Significant variability was found, however, on whether agencies have been able to make full use of the increased capacity in order to have an impact on the ground or in achieving the ultimate objectives of TA. It is critical that the IMF should understand fully what prevents agencies from doing so.
  
  
• Part of the problem is that IMF documentation and reporting does not clearly unbundle and track the different stages of progress toward meeting the final objectives of TA. Specifically, documentation is weak in:
  
  
  • defining at the outset indicators (benchmarks) to judge whether progress is occurring, and explicitly discussing these indicators with the authorities;
    
    
  • unbundling between short- and medium-term indicators that capture different stages of the results chain, for example, (a) indicators that track the improved technical abilities of agencies receiving TA; (b) indicators that show whether these agencies are actually using that increased know-how, for example, whether they are performing their responsibilities; and (c) indicators that track the economic outcomes of that enforcement.
    
    
• The absence of a clear unbundling of these stages, and the factors influencing the lack of progress, limits the ability to use track record in implementing TA in making decisions about future TA. This is critical because there may be good reasons why TA recommendations have not been implemented.
  
  
• Frequently political interference or lack of support by the authorities prevent agencies from using effectively the knowledge transmitted by TA. Indeed, the case studies suggest that resistance by vested interests may mount as these agencies improve their ability to act. The evaluation found that in these cases the reporting from the field on constraints to progress has often not been candid enough, so that the ways to address such obstacles were generally not discussed frankly with the authorities.

B. Main Recommendations

The main recommendations of the evaluation are as follows:

The IMF should develop a medium-term country policy framework for setting TA priorities, incorporating country-specific strategic directions and linked to more systematic assessments of factors underlying past performance.

• In low-income countries, the PRS process provides the natural framework to identify TA capacity-building priorities, although it has infrequently been used effectively for this purpose. The IMF needs to engage countries to help them articulate their medium-term capacity-building needs in the IMF's areas of responsibility and in accordance with the PRSP. For other countries where there is a relatively significant provision of IMF TA, the framework may require periodic in-depth consultations with the authorities comprising a analysis of past progress and a forward-looking exercise to identify priorities.
  
  
• Area departments and resident representatives could play a greater role in developing these frameworks and this role should be explicitly acknowledged.

The IMF should develop more systematic approaches to track progress on major TA activities and to identify reasons behind major shortfalls

• At the outset of major TA activities, IMF staff and the authorities should agree on how the success of the TA activity will be measured. The IMF staff should unbundle more clearly the different stages through which TA has an impact, and then monitor these stages. Specifically, it should differentiate between:
  
  
  • progress in improving the technical capacities of agencies receiving TA;
    
    
  • whether agencies are making effective use of that increased technical capacity; as well as reasons on why this is not happening; and
    
    
  • the impact on the ground in terms of economic outcomes.
    
    
• Resident experts and headquarters staff in charge of backstopping activities should be candid in reporting obstacles to progress, including political interference or lack of support from the authorities that prevent agencies from making effective use of their improved technical capacity.

Greater involvement by the authorities and counterparts in the design of TA activities and arrangements for follow-up should be emphasized as a signal of ownership and commitment

• IMF staff should request the authorities and specialized counterparts to fully participate in the preparation of the TORs and devote sufficient time to help design the activity. Willingness to do so should be one of the factors taken into account in decisions on TA resource allocation.
  
  
• For more complex multiyear TA activities, a letter of agreement between the authorities and the IMF could specify commitment and resources including (a) mutually agreed benchmarks of progress, (b) commitments by both the IMF and the authorities-to assure sustainability beyond the life of the TA activity; and (c) critical policy steps that are required from the authorities to ensure necessary institutional changes, such as decrees or legislation that complement the TA activity.
  
  
• Greater efforts need to be made to disseminate the lessons of specific TA activities within relevant government departments and agencies.

Stronger efforts should be made by TA experts to identify options and discuss alternatives with local officials prior to drafting TA recommendations

• The receptivity of TA recommendations seems to be enhanced greatly when IMF experts engage counterparts early on in the design of the activity, explain its motivation, and try to assess the institutional subtleties of the specific environment. There is also a need to allow enough time for informal discussions prior to issuing recommendations and the wider consideration of options. TA missions should allow enough time to incorporate these factors even if the result is somewhat longer missions and correspondingly fewer TA activities.

The report was discussed by the IMF's Executive Board on February 18, 2005. The Board welcomed the report and endorsed the thrust of its findings, lessons, and recommendations. The report, including individual case studies, and along with IMF management and staff responses and the Summing Up of the Board discussion are available on the IEO's website at www.imf.org/ieo.

¹Cambodia, Honduras, Niger, Ukraine, Yemen, and Zambia.