Enhancing IMF Economics Training: AI-Powered Analysis of Qualitative Learner Feedback
Electronic Access:
Free Download. Use the free Adobe Acrobat Reader to view this PDF file
Disclaimer: IMF Working Papers describe research in progress by the author(s) and are published to elicit comments and to encourage debate. The views expressed in IMF Working Papers are those of the author(s) and do not necessarily represent the views of the IMF, its Executive Board, or IMF management.
Summary:
The International Monetary Fund (IMF) has expanded its online learning program, offering over 100 Massive Open Online Courses (MOOCs) to support economic and financial policymaking worldwide. This paper explores the application of Artificial Intelligence (AI), specifically Large Language Models (LLMs), to analyze qualitative feedback from participants in these courses. By fine-tuning a pre-trained LLM on expert-annotated text data, we develop models that efficiently classify open-ended survey responses with accuracy comparable to human coders. The models’ robust performance across multiple languages, including English, French, and Spanish, demonstrates its versatility. Key insights from the analysis include a preference for shorter, modular content, with variations across genders, and the significant impact of language barriers on learning outcomes. These and other findtpeings from unstructured learner feedback inform the continuous improvement of the IMF's online courses, aligning with its capacity development goals to enhance economic and financial expertise globally.
Series:
Working Paper No. 2024/166
Subject:
Artificial intelligence Technology
Frequency:
regular
English
Publication Date:
August 2, 2024
ISBN/ISSN:
9798400286124/1018-5941
Stock No:
WPIEA2024166
Format:
Paper
Pages:
37
Please address any questions about this title to publications@imf.org