About

Twitter

Facebook

CUP Web site

RSS Feed

New Books

Author Interviews

Author Events

Keep track of new CUP book releases:
e-newsletters

For media inquiries, please contact our
publicity department

CUP Authors Blogs and Sites

American Society of Magazine Editors

Leonard Cassuto

Mike Chasar / Poetry and Popular Culture

Erica Chenoweth / "Rational Insurgent"

Juan Cole

Jenny Davidson / "Light Reading"

Faisal Devji

William Duggan

James Fleming / Atmosphere: Air, Weather, and Climate History Blog

David Harvey

Paul Harvey / "Religion in American History"

Bruce Hoffman

Alexander Huang

David K. Hurst / The New Ecology of Leadership

Jameel Jaffer and Amrit Singh

Geoffrey Kabat / "Hyping Health Risks"

Grzegorz W. Kolodko / "Truth, Errors, and Lies"

Jerelle Kraus

Julia Kristeva

Michael LaSala / Gay and Lesbian Well-Being (Psychology Today)

David Leibow / The College Shrink

Marc Lynch / "Abu Aardvark"

S. J. Marshall

Michael Mauboussin

Noelle McAfee

The Measure of America

Philip Napoli / Audience Evolution

Paul Offit

Frederick Douglass Opie / Food as a Lens

Jeffrey Perry

Mari Ruti / The Juicy Bits

Marian Ronan

Michael Sledge

Jacqueline Stevens / States without Nations

Ted Striphas / The Late Age of Print

Charles Strozier / 9/11 after Ten Years

Hervé This

Alan Wallace

James Igoe Walsh / Back Channels

Xiaoming Wang

Santiago Zabala

Press Blogs

AAUP

University of Akron

University of Alberta

American Management Association

Baylor University

Beacon Broadside

University of California

Cambridge University Press

University of Chicago

Cork University

Duke University

University of Florida

Fordham University Press

Georgetown University

University of Georgia

Harvard University

Harvard Educational Publishing Group

University of Hawaii

Hyperbole Books

University of Illinois

Island Press

Indiana University

Johns Hopkins University

University of Kentucky

Louisiana State University

McGill-Queens University Press

Mercer University

University of Michigan

University of Minnesota

Minnesota Historical Society

University of Mississippi

University of Missouri

MIT

University of Nebraska

University Press of New England

University of North Carolina

University Press of North Georgia

NYU / From the Square

University of Oklahoma

Oregon State University

University of Ottawa

Oxford University

Penn State University

University of Pennsylvania

Princeton University

Stanford University

University of Sydney

University of Syracuse

Temple University

University of Texas

Texas A&M University

University of Toronto

University of Virginia

Wilfrid Laurier University

Yale University

June 25th, 2012 at 9:48 am

Jacqueline Stevens: Political Scientists Are Lousy Forecasters

Political Scientists Are Lousy Forecasters

“It’s an open secret in my discipline: in terms of accurate political predictions (the field’s benchmark for what counts as science), my colleagues have failed spectacularly and wasted colossal amounts of time and money.” — Jacqueline Stevens

This past Sunday, The New York Times published “Political Scientists Are Lousy Forecasters,” an opinion article by Jacqueline Stevens, professor of political science at Northwestern University and the author of States Without Nations: Citizenship for Mortals. In this article, Professor Stevens addresses the anger in the academic political science community about the recent amendment to a bill passed by the House of Representatives which will eliminate National Science Foundation Grants for political scientists. While Professor Stevens does believe that political scientists make important public contributions, she claims that she has found the N.S.F.-funded research done by political scientists troubling for decades: “he government — disproportionately — supports research that is amenable to statistical analyses and models even though everyone knows the clean equations mask messy realities that contrived data sets and assumptions don’t, and can’t, capture.”

The problem, as Professor Stevens sees it, is a simple one: because it is nearly impossible to predict political events, political scientists have a bad track record in predicting important events and trends.

It’s an open secret in my discipline: in terms of accurate political predictions (the field’s benchmark for what counts as science), my colleagues have failed spectacularly and wasted colossal amounts of time and money. The most obvious example may be political scientists’ insistence, during the cold war, that the Soviet Union would persist as a nuclear threat to the United States. In 1993, in the journal International Security, for example, the cold war historian John Lewis Gaddis wrote that the demise of the Soviet Union was “of such importance that no approach to the study of international relations claiming both foresight and competence should have failed to see it coming.” And yet, he noted, “None actually did so.” Careers were made, prizes awarded and millions of research dollars distributed to international relations experts, even though Nancy Reagan’s astrologer may have had superior forecasting skills.


Equally troubling for Professor Stevens is that the money provided for these political predictions is often tainted by ideologies:

Political prognosticators fare just as poorly on domestic politics. In a peer-reviewed journal, the political scientist Morris P. Fiorina wrote that “we seem to have settled into a persistent pattern of divided government” — of Republican presidents and Democratic Congresses. Professor Fiorina’s ideas, which synced nicely with the conventional wisdom at the time, appeared in an article in 1992 — just before the Democrat Bill Clinton’s presidential victory and the Republican 1994 takeover of the House.

Alas, little has changed. Did any prominent N.S.F.-financed researchers predict that an organization like Al Qaeda would change global and domestic politics for at least a generation? Nope. Or that the Arab Spring would overthrow leaders in Egypt, Libya and Tunisia? No, again. What about proposals for research into questions that might favor Democratic politics and that political scientists seeking N.S.F. financing do not ask — perhaps, one colleague suggests, because N.S.F. program officers discourage them? Why are my colleagues kowtowing to Congress for research money that comes with ideological strings attached?

Professor Stevens points to an important study of political experts as evidence for political science’s lack of predictive success:

How do we know that these examples aren’t atypical cherries picked by a political theorist munching sour grapes? Because in the 1980s, the political psychologist Philip E. Tetlock began systematically quizzing 284 political experts — most of whom were political science Ph.D.’s — on dozens of basic questions, like whether a country would go to war, leave NATO or change its boundaries or a political leader would remain in office. His book “Expert Political Judgment: How Good Is It? How Can We Know?” won the A.P.S.A.’s prize for the best book published on government, politics or international affairs.

Professor Tetlock’s main finding? Chimps randomly throwing darts at the possible outcomes would have done almost as well as the experts.

While she is opposed to the continuation of N.S.F. grants in particular, Professor Stevens makes it clear in the conclusion of her article that she is not opposed to governmental support for political scientists:

Government can — and should — assist political scientists, especially those who use history and theory to explain shifting political contexts, challenge our intuitions and help us see beyond daily newspaper headlines. Research aimed at political prediction is doomed to fail. At least if the idea is to predict more accurately than a dart-throwing chimp.

To shield research from disciplinary biases of the moment, the government should finance scholars through a lottery: anyone with a political science Ph.D. and a defensible budget could apply for grants at different financing levels. And of course government needs to finance graduate student studies and thorough demographic, political and economic data collection. I look forward to seeing what happens to my discipline and politics more generally once we stop mistaking probability studies and statistical significance for knowledge.

Post a comment