The problem with punditry

The media often favor bad ideas, especially when the truth it toomessy to be packaged neatly.

By PHILIP E. TETLOCK
May 16, 2006 20:06
4 minute read.

 
X

Dear Reader,
As you can imagine, more people are reading The Jerusalem Post than ever before. Nevertheless, traditional business models are no longer sustainable and high-quality publications, like ours, are being forced to look for new ways to keep going. Unlike many other news organizations, we have not put up a paywall. We want to keep our journalism open and accessible and be able to keep providing you with news and analyses from the frontlines of Israel, the Middle East and the Jewish World.

As one of our loyal readers, we ask you to be our partner.

For $5 a month you will receive access to the following:

  • A user experience almost completely free of ads
  • Access to our Premium Section
  • Content from the award-winning Jerusalem Report and our monthly magazine to learn Hebrew - Ivrit
  • A brand new ePaper featuring the daily newspaper as it appears in print in Israel

Help us grow and continue telling Israel’s story to the world.

Thank you,

Ronit Hasin-Hochman, CEO, Jerusalem Post Group
Yaakov Katz, Editor-in-Chief

UPGRADE YOUR JPOST EXPERIENCE FOR 5$ PER MONTH Show me later

Every day, experts bombard us with their views on topics as varied as Iraqi insurgents, Bolivian coca growers, European central bankers, and North Korea's Politburo. But how much credibility should we attach to the opinions of experts? The sanguine view is that as long as those selling expertise compete vigorously for the attention of discriminating buyers (the mass media), market mechanisms will assure quality control. Pundits who make it into newspaper opinion pages or onto television and radio must have good track records; otherwise, they would have been weeded out. Skeptics, however, warn that the mass media dictate the voices we hear and are less interested in reasoned debate than in catering to popular prejudices. As a result, fame could be negatively, not positively, correlated with long-run accuracy. Until recently, no one knew who is right, because no one was keeping score. But the results of a 20-year research project now suggest that the skeptics are closer to the truth. I describe the project in detail in my book Expert Political Judgment: How good is it? How can we know? The basic idea was to solicit thousands of predictions from hundreds of experts about the fates of dozens of countries, and then score the predictions for accuracy. We find that the media not only fail to weed out bad ideas, but that they often favor bad ideas, especially when the truth is too messy to be packaged neatly. The evidence falls into two categories. First, as the skeptics warned, when hordes of pundits are jostling for the limelight, many are tempted to claim that they know more than they do. Boom and doom pundits are the most reliable over-claimers. Between 1985 and 2005, boomsters made 10-year forecasts that exaggerated the chances of big positive changes in both financial markets (e.g., a Dow Jones Industrial Average of 36,000) and world politics (e.g., tranquility in the Middle East and dynamic growth in sub-Saharan Africa). They assigned probabilities of 65% to rosy scenarios that materialized only 15% of the time. In the same period, doomsters performed even more poorly, exaggerating the chances of negative changes in all the same places where boomsters accentuated the positive, plus several more (I still await the impending disintegration of Canada, Nigeria, India, Indonesia, South Africa, Belgium, and Sudan). They assigned probabilities of 70% to bleak scenarios that materialized only 12% of the time. Second, again as the skeptics warned, over-claimers rarely pay penalties for being wrong. Indeed, the media shower lavish attention on over-claimers while neglecting their humbler colleagues. WE CAN see this process in sharp relief when, following the philosopher Sir Isaiah Berlin, we classify experts as "hedgehogs" or "foxes." Hedgehogs are big-idea thinkers in love with grand theories: libertarianism, Marxism, environmentalism, etc. Their self-confidence can be infectious. They know how to stoke momentum in an argument by multiplying reasons why they are right and others are wrong. That wins them media acclaim. But they don't know when to slam the mental brakes by making concessions to other points of view. They take their theories too seriously. The result: hedgehogs make more mistakes, but they pile up more hits on Google. ECLECTIC FOXES are better at curbing their ideological enthusiasms. They are comfortable with protracted uncertainty about who is right even in bitter debates, conceding gaps in their knowledge and granting legitimacy to opposing views. They sprinkle their conversations with linguistic qualifiers that limit the reach of their arguments: "but," "however," "although." Because they avoid over-simplification, foxes make fewer mistakes. Foxes will often agree with hedgehogs up to a point, before complicating things: "Yes, my colleague is right that the Saudi monarchy is vulnerable, but remember that coups are rare and that the government commands many means of squelching opposition." Imagine your job as a media executive depends on expanding your viewing audience. Whom would you pick: an expert who balances conflicting arguments and concludes that the likeliest outcome is more of the same, or an expert who gets viewers on the edge of their seats over radical Islamists seizing control and causing oil prices to soar? In short, the qualities that make foxes more accurate also make them less popular. At this point, uncharitable skeptics chortle that we get the media we deserve. But that is unfair. No society has yet created a widely trusted method for keeping score on the punditocracy. Even citizens who prize accuracy have little way of knowing that they are sacrificing it when they switch channels from boring foxes to charismatic hedgehogs. Here, then, is a modest proposal that applies to all democracies: the marketplace of ideas works better if it is easier for citizens to see the trade-offs between accuracy and entertainment, or between accuracy and party loyalty. Wouldn't they be more likely to read pundits with better track records? If so, pundits might adapt to accountability by showing more humility, and political debate might begin to sound less shrill. Granted, it is not easy to create methods of keeping score that are credible across the spectrum of reasonable opinion. But in a world where, as Yeats said, "and the best lack all conviction, while the worst are full of passionate intensity," it is worth trying. The writer is Professor of Business Administration, Political Science, and Psychology at the University of California at Berkeley. - www.project-syndicate.org

Join Jerusalem Post Premium Plus now for just $5 and upgrade your experience with an ads-free website and exclusive content. Click here>>

Related Content

Jamal Khashoggi
November 15, 2018
Saudi Arabia seeks death penalty against 5 suspects in Khashoggi killing

By REUTERS