Inside fact-checking: A conversation with Lucas Graves, author of 'Deciding What's True'

Fact-checking has flourished since 2007, when new projects like the Tampa Bay Times’ PolitiFact launched to fact-check American politics. In the years since, more news organizations have put attention and resources toward fact-checking, so that it’s now an established part of political campaign coverage.

Lucas Graves, a professor at the University of Wisconsin, has been studying the fact-checking movement throughout this time period. His new book, Deciding What’s True: The Rise of Political Fact-Checking in American Journalism, explores the recent growth of fact-checking and why it resonates with both journalists and the public.

Graves’ approach to studying fact-checking is immersive. He briefly worked as a fact-checker himself and shadowed journalists as they went about their work. His book provides detailed accounts of fact-checkers at PolitiFact, at the University of Pennsylvania’s Factcheck.org and at the Washington Post Fact Checker. Following in the footsteps of academic works like Michael Schudson’s The Sociology of News, Graves attempts to understand fact-checking from the perspective of the fact-checkers themselves. He describes their work in a real-world context, where competitive publishing pressures coexist with idealistic aims of bringing truth to a democracy.  

Among the many journalists Graves interviewed for his book is PolitiFact Editor Angie Drobnic Holan. Graves and Holan recently talked about his new book, the impact of fact-checking, and how fact-checking might affect the upcoming presidential debates between Donald Trump and Hillary Clinton. Below are edited excerpts of their conversation.

Holan: Let’s talk about the big question first. Why does fact-checking matter?

Graves: Fact-checking matters in a few different ways. The most important one for me is that it represents a new kind of commitment from journalists to try to pierce political rhetoric and hold politicians accountable. It’s a cultural shift in journalism.

You write in your book that fact-checking is a reform movement in journalism. What does that mean?

Fact-checking is a reform movement because it makes an argument about how political journalism should be practiced. It’s part of a reaction against "he said, she said" reporting which has been building for decades. It fits into a much larger historical pattern of cultural shifts and even reform movements in journalism.

The research for my book involved spending a lot of time with fact-checkers at PolitiFact and elsewhere. From the start, what jumped out at me was that fact-checkers had a message not just for the public, but for other journalists. In a sense, the most important audience for fact-checkers is other journalists. That’s because fact-checking is most effective at dissuading politicians from exaggerating or peddling falsehoods when you have some consensus among journalists that a claim has been disproven, that it’s false.

Fact-checking really works best when reporters are willing to look at what their peers have already written about a claim. Then they can either challenge the claim when it’s repeated, or just leave it out of their reporting altogether.

Chris Wallace of Fox News has been tapped to moderate one of the presidential debates, and he said recently that he didn’t intend to "truth squad" the candidates during the debate. I thought that was interesting because Chris Wallace is a very astute interviewer. He may not fact-check the people he interviews per se, but he often asks pointed follow-up questions that capture the spirit of fact-checking.

It raises a really interesting question, right? On the one hand, there’s an argument that in a debate, it’s the candidates -- the debaters themselves -- who are keeping each other honest. And it’s their job to fact-check each other and the moderators’ job to stay out of the way, so the public can make up their own minds. But that really runs against the spirit of fact-checking.

The whole point of fact-checking is that it’s not enough to let the candidates yell at each other and leave it up to the viewers to figure out the truth. If all we end up seeing is Trump and Clinton contradicting each other, that’s not going to be a very enlightening debate. So in my opinion, when a claim has been thoroughly debunked, then the moderator can step in and at least point to the evidence.

Now the idea shouldn’t be just to cut the candidate off. The person being fact-checked should have the opportunity to respond as to why they might disagree with the fact-check or to provide some context. But anything we can do to prevent the debate from just being candidates disagreeing with each other, and to push the dialogue onto firmer ground, is a good thing.

I agree. But what if it’s not clear-cut? What do you do in those cases? Some would say that Candy Crowley’s correction of Mitt Romney in 2012 fell into that category. The candidates were arguing over how soon President Barack Obama had called the attack in Benghazi an act of terror, and Obama’s comments the day after the attack were hedged or at least had some nuance to them. Some people thought it wasn’t as settled as Crowley made it seem.

I think Candy Crowley’s fact-check of Mitt Romney in 2012 was an example of how you don’t want fact-checking to happen in a presidential debate. Instead of an off-the-cuff fact-check that’s aimed at one candidate, you want a mechanism that’s been carefully thought out and set up in advance, one that the candidates are expecting, so that it is more fair and the moderators only step in where the facts are really well-established.

I liked this statement from your book: "Critics have called this constant slide into interpretation a problem of the sorts of claims fact-checkers choose to investigate, arguing that they stray too easily into the realm of opinion. But it is really a problem of language and truth." What kind of problem is that, specifically?  

Most of us walk around with this idea of truth as being fairly black and white, at least in everyday life. We think of facts as atoms of understanding, something that can’t really be debated, something that everybody agrees upon and requires no interpretation. And of course, in a lot of cases it seems to work that way.

But when you look closely at almost any factual debate, you generally find that things are more complicated. And any fact, even an established scientific fact, is in principle open to challenge and may be set aside when we find some better way of understanding the world. So facts often aren’t as settled as we imagine them to be. The world tends to be a bit messier and more complicated than we usually think. So one of the problems is that we expect a certainty from the world that we rarely get.

Yes, in journalism as well as life.

Absolutely. And if you ask an expert in any discipline, whether it’s history, physics, biology or anything else, they understand the complexity and the nuance that’s involved in making sense of the data that they have. But in everyday life, we have an expectation that things will be really indisputably true in a way that is rarely the case.

Is fact-checking changing traditional ideas of what it means to be an objective reporter?

I definitely think it is. It fits into a much longer pattern of change we’ve seen since the objectivity norm first started to become established after World War I. There’s been a pretty steady increase in the willingness of journalists to interpret the world on behalf of their readers, and to bring not only their reporting but their analysis to the articles that they write, and that’s precisely because the world is a complex place.

We have many examples in history of how mechanical ideas about how to be objective can be taken advantage of by politicians who want to deceive the public. One of the ways that journalism changes is in response to these embarrassing episodes. After the red scare in the 1950s, journalists spent a lot of time examining the really lousy reporting that helped Joseph McCarthy spread his wild claims. You saw the same thing after the Vietnam War and after Watergate, when once again journalists argued it's not enough just to tell the public what officials say — you also have to investigate the truth behind those claims and challenge those claims and cast a skeptical eye on the version of events coming from government officials.

I think the reporting in the lead-up to the Iraq War in 2002 and 2003 is the most recent example of this kind of journalistic embarrassment that really pushed a lot of reporters to be more aggressive and to start to think differently about their jobs. I think that episode helped promote the new culture of fact-checking.

We get a lot of feedback from readers who really love fact-checking, but we also get a good number of angry emails from people who either don’t like fact-checking or don’t like the way we do it. In your book, you write that fact-checkers "are inundated in hostile and sometimes unhinged communication … even more than peers at traditional news outlets." Why should it be that fact-checkers get more pushback than other journalists?

I don't know of a good way to measure that, but my sense is that it is true, precisely because you are taking sides in political debates. One of the reasons that journalists have traditionally avoided contradicting things politicians say is to avoid angering their readers or being accused of bias. Don’t you think you get more angry emails than someone who’s filing straight news reports from the campaign trail?

I suspect we do, but I’m used to it. PolitiFact has been around for nine years, and it seems like angry emails are just part of the job.

What’s really interesting is that now reporters are much more exposed to that kind of input than they used to be. One of findings of the early sociological studies of American newsrooms was how insulated the reporters were from the things their audience was saying. You had separate departments at magazines and newspapers that handled all of the correspondence from the public. Their job was to shield reporters from that material. And frankly, many reporters weren’t that interested in what their audience actually thought. They preferred to write for this idealized public that looked a lot like them and was assumed to be really interested in the things that they were writing.

That’s obviously less true today, when every reporter gets emails directly from the public and can see every day all of the things that bloggers and other commentators are saying about their work online. That’s a really important shift.

You write, "Only a reasoning democratic public can make fact-checkers’ work meaningful." Do you think the public is doing its part?

You should tell me if you think that’s true or not. I've noticed that when fact-checkers talk about their work as objective journalism, they tend to focus on how it brings information to the public. That’s a way of saying they're not here to change anybody’s mind or to stop politicians from lying or eliminate untruths from political discourse, or any of the other things that people sometimes wish fact-checking could accomplish. Instead, they say the job of a fact-checker is just to provide information that people can take into account, but that they're free to disagree with.

I think, and you can tell me if you agree, I think there is a little bit of a tension there, because at the same time most fact-checkers I’ve talked to believe in the verdicts that they’re reaching, and they don’t actually think that in a lot of cases people should be disagreeing with them. Questions like President Barack Obama’s birthplace, or climate change -- when we think about the health of our democracy, we really wish more people would accept the evidence about human-driven climate change, because then we could move ahead and decide what we want to do or not do about that as a democratic public. So there’s a tension between that idea of shaping democratic discourse, and a more removed role in which it’s not a journalist's job to change the conversation.

I would say that society is complicated, and the media is one factor of many that influences public policy and politics. I think fact-checkers should have some humility as they go about their work, because that allows them to have maximum open-mindedness, which is what we need to do good fact-checking. I don’t think fact-checkers should say, "We know best, we know what people should think," because that closes down your ability to look at things in a new way and ask the question, "Is this true or not?"

I agree with that completely, and that’s one of the points I’m trying to make in the book. But there’s a tension there, right? There are really good reasons for fact-checkers not to become campaigners or advocates for particular points of view. One is that people won’t trust them as much, and another is that, just as you say, they won’t be able to look at the world objectively, because once you become invested in a campaign, it can start to influence your analysis.

But at the same time, you are out there saying that you’re reaching decisive factual conclusions about important questions, and you have to be able to stand behind your verdicts. Most of the fact-checkers I’ve talked believe in what they’re doing and believe they’re getting at the truth.

So that’s just a way of saying that it’s complicated. Fact-checkers are in this funny position where they’re a little bit more involved than traditional reporters, they’re out there pushing one account as the decisive truth, but they still have to stay out of the political fray to the extent that they can. And that’s what makes it so interesting. It’s a really delicate balancing act.