Download PDF

Bob Simison profiles Stanford’s Susan Athey, who brings machine learning to economics

It was just plain luck when Susan Athey glanced out the kitchen window one day last January. A coyote was making off with one of the family’s pet chickens clutched in its fangs.

Athey grabbed a broom, fought off the coyote, and got Viola, a buff-colored chicken, to a veterinarian to be stitched back together. After a month-long convalescence in the family home on the Stanford campus, Viola rejoined the nine-chicken flock.

“The idea that I kept a chicken in my house for a month would be horrifying to my relatives in Alabama,” where her grandmother once raised chickens, says Athey, a Stanford economist. As for the choice of chickens as pets, that was simple resource-allocation economics, says Guido Imbens, a Stanford econometrics professor who’s been married to Athey since 2002. Chickens are lower-maintenance than dogs or cats, and they lay a couple of dozen eggs a week.

The Superwoman episode is no surprise to those who know Athey as an academic superstar. At the age of 48, she is the economics of technology professor at the Stanford Graduate School of Business and has won just about every award imaginable. She has published an array of papers on some of the hottest issues in economics, pioneered the role of tech economist, and helped confront her profession’s #MeToo history.

Through academic positions at Stanford, Harvard, Yale, and the Massachusetts Institute of Technology (MIT), as well as consulting for Microsoft and other companies, she has since broken ground studying the economics of the internet, applying auction theory to online services such as search-related advertising, and developing the emerging field of tech economics.

“It’s astonishing the breadth and depth of what she has worked on,” says Robert Marshall, an economics professor at Penn State University. “That’s not usual. Scott Adams, the creator of the comic ‘Dilbert,’ has said that there are 100 people on the planet at a given time who will make a difference. She is one of them.”

In 2007, Athey won the John Bates Clark Medal for outstanding economists under the age of 40, the first female to do so. It normally goes without saying that winners are a good bet for a Nobel. Marshall says it anyway. Marshall was a mentor of Athey’s when she was a Duke University undergraduate in the late 1980s. Before she landed a job as Marshall’s research assistant, Athey focused on math and computer science.

The Clark medal recognized her work in solving complex statistical problems, advancing knowledge of market design and the mechanics of auctions, econometrics, and industrial organization. Some of her early findings supported the Federal Reserve’s adoption of inflation targeting for managing monetary policy.

For an academic of her prominence, Athey has few visible critics, according to those who know her and her work well. Some of them say there may be quiet resentment over her success academically and economically as a woman.

“Susan is really a model of what I think economists should be,” says Matthew Gentzkow, a Stanford economics professor who won the Clark medal in 2014. “She combines absolutely cutting-edge engagement with research on the frontiers of economics, while having a deep level of engagement with communities outside of economics. She’s translating science into an impact on the real world.”

Athey sees the diverse elements of her career on a continuum. Her early research on timber auctions and pricing schemes flowed into work on technology markets such as designing search-engine advertising auctions, she says. When she realized that the tools for finding cause-and-effect links using machine learning or artificial intelligence didn’t exist, she set out to develop them. That led to her current interest in using technology, machine learning, and other tools of tech economics to help address social problems, she says.

“Market design is a cross-cutting theme all the way through,” Athey says. “We want to think of the economist as engineer, that we want to get out and actually use the tools of economics to make markets work better.”

Athey is known as a hard worker. The day after Christmas 2004, she came to school to help Catherine Tucker, then a Stanford PhD candidate and now a professor at MIT, prepare a key presentation. As an assistant professor, she worked all hours, says Stanford business dean Jonathan Levin. Athey responded to an email from Joshua Gans, an economics professor at the University of Toronto and a former grad school colleague, by phoning him in Australia from the labor and delivery room while having the first of her three children.

The daughter of a physicist and an English teacher, Athey enrolled at Duke University in Durham, North Carolina, at the age of 16 after growing up in the Maryland suburbs of Washington. She was active in a sorority and was president of the Duke field hockey club. Then a friend introduced her to Marshall, who was working on procurement auctions at the time.

“I asked my research assistant, who was leaving, to find me someone who was as good as she was or better,” Marshall says. “She came back with what looked like a high school student.” Athey soon impressed him as eager, thoughtful, deliberate, and brilliant. At Marshall’s suggestion, Athey began searching for information about the timber industry and found a source who had digitized the records from thousands of timber auctions. This formed the basis of several research papers for Marshall, and years later Athey returned to write several papers using the same data set.

“Susan was instrumental in doing work on my paper,” Marshall says. “She made me very much more productive. I told colleagues that she was smarter and better than I was.”

As a newly minted, 24-year-old Stanford PhD in 1995, Athey was the subject of a New York Times profile proclaiming her “the top draft pick in economics” and reporting that she fielded two dozen job offers, from which she chose MIT.

Athey went on to publish paper after paper on auctions and government procurement, showing how market structures can encourage collusion among buyers and sellers and how government agencies were leaving vast amounts of money on the table. She points out that she designed the timber auction system now used by the government of British Columbia, one of the world’s largest producers of lumber.

Bidding war

Athey put her knowledge of markets and auctions to practical use in 2001, when she and husband Imbens found themselves in the quintessential California experience, a four-way bidding war for a house. It showed how she approaches problems, methodically gathering all the information, according to Imbens. Athey says it wasn’t rocket science, just Auctions 101.

“I quizzed the real estate agent and went through previous types of auctions he’d been part of,” she says. That helped her calculate just the right bid to win the house.

Athey caught the attention of Microsoft’s then-CEO Steve Ballmer in 2007. He says he read about her Clark award in a Harvard publication and recruited her as a consultant. It lasted until 2014, when the company asked her to come on full-time and she chose to stick with her academic career.

In 2007, Microsoft was working to make its search engine—which eventually became known as Bing—competitive with Google. Such companies sell advertising space for search results through auctions, Athey says.

“The early academic literature on these auctions hadn’t really accounted for the fact that auction design affects the quality of the advertisements, and further, how much attention consumers pay to the advertisements depends on that quality,” she says. Together with MIT economist Glenn Ellison, Athey published a paper “that brought the consumer into the picture.”

It was an important insight that applies not just in search, but also to online marketplaces like Airbnb and others, Athey says. It pointed toward paying more attention to the role of auction design on the quality of the user’s experience, which feeds back into the advertisers’ incentives to participate and create high-quality advertisements. “When the advertisements are a better match to the consumer intent, the advertisers will pay more for a click,” she says.

Athey’s work at Microsoft extended well beyond market design and included work on mapping economic objectives into the measurements that were used to guide and operate the search advertising business.

Athey’s contributions were significant, says Ballmer, the billionaire businessman who stepped down as Microsoft CEO in 2014. That’s why other top tech companies have taken to hiring PhD economists by the hundreds, he says.

“We’ve entered a world in which computer science, instead of figuring out absolute answers, is using data to guess at answers statistically,” Ballmer says. Athey was one of the first economists to help develop that approach, he says. “Economics and computer science are both evolving, and economists are using statistical technology to think about all things economic.”

Athey cites her role in “pioneering tech economics” alongside economists such as Google’s Hal Varian as one of her proudest contributions.

“When I think about tech economics, it’s actually a very broad thing,” Athey says. “Tech economics includes market design, but it also includes machine learning and understanding the impact of technology on the economy.”

Women in economics

Colleagues both male and female credit Athey for providing an important role model for women in a traditionally male-dominated discipline. Of the 45 PhD holders she has advised—an astonishing total, other academics say—more than a third have been women.

One of them is Amalia Miller, now an economics professor at the University of Virginia. She says Athey provided “inspiration and role-modeling,” including starting a family while serving as an adviser to Miller. Now Miller has a four-year-old daughter of her own.

“When I told Susan the good news that I was expecting, she wrote back to me with pages and pages of advice,” Miller says. It included suggestions on “all kinds of practical issues associated with being productive” as a professor while parenting, and which rocking chair to buy. Miller bought the rocker, she says,

Athey herself says there was a real shortage of female role models for her.

“There was a stereotype of what a good mathematician looks like, and I didn’t fit it,” she says. “So I tried to look serious, because people questioned how serious I was. People question how smart you are.” The challenges changed at each stage of her career, she says.

“You claw your way up,” Athey says. “At the time I got my PhD, it was an open question whether many women would be able to get tenure if they had kids, but then the women in my cohort blew that issue out of the water. At first it seemed as if the biggest gender issues were solved.” But in fact they weren’t, she says. “Everything was supposed to get better. It’s disappointing that it didn’t.”

Athey was among female economists who confronted the profession’s history of sexual harassment, bullying, and discrimination at the American Economics Association’s (AEA’s) annual meeting last January. In a panel discussion, Athey said she wore khakis and loafers to fit in with the men at MIT, according to the New York Times.

“I spent all my time hoping that no one would remember I was female,” the Times quoted her as saying.

A subsequent survey of more than 9,000 economists by the AEA found significant evidence of sexual harassment and discrimination. Seventy percent of the female respondents said their work was taken less seriously than that of men. The organization responded by announcing several measures to fight discrimination and harassment.

Athey is faculty director of the Stanford business school’s Initiative for Shared Prosperity and Innovation (ISPI). The year-old project aims to use technology to address social problems, including poverty and inequality.

“Through all my work in tech markets, I became aware of all the things that you can do with data,” Athey says. “That has led me to my very latest set of interests, which is using technology to solve social problems.”

Early projects for the initiative include applying machine learning in educational technology companies and improving approaches to measuring impact. This is critical because technology companies typically engage in rapid, incremental improvements guided by data from many experiments, Athey says. It is especially important for social impact projects that often rely on philanthropy or government funding. Being able to demonstrate effectiveness can complement approaches that tie funding to measurable benefits, such as pay-for-outcomes plans. The initiative is also studying other ways of encouraging innovation, including income sharing for training and prizes for innovation, she says.

“Delivering services digitally or through digital platforms is a natural area of growth for social impact work,” Athey says. “I view the work at SPII as combining market shaping and incentive design with machine learning for social impact. It is also a natural extension of my work as a tech economist—applying tech economics to the social impact space.”

Athey will continue having a significant impact on economics for some time, based on the numbers of students she’s nurtured at all levels, her undergraduate mentor Marshall says. At retirement dinners in two or three decades, he predicts, “there will be an astonishing number of her students to attest to the difference she made in their lives.”

For her part, Athey says, “My greatest hope is that in a few decades, people will be able to point to significant achievements in what I’ve pivoted to recently, trying to take all these learnings from economics, market design, and machine learning, and applying them to social problems.”

Opinions expressed in articles and other materials are those of the authors; they do not necessarily reflect IMF policy.