Deflategate and the limits of science

Remember the idea that science might answer once and for all what happened to the New England Patriots’ under-inflated footballs? It was a nice thought, but recent information has basically told us to forget it.

As quarterback Tom Brady begins his appeal to reverse the NFL’s findings, the truth will remain, it appears, in the eye of the beholder.

We fact-checked a claim from Patriots coach Bill Belichick that atmospheric conditions could push a football’s pressure "down approximately one-and-a-half pounds per square inch." Ultimately, we said Mostly True, noting that it is indeed possible.

In the months since, much has happened, and some people have asked to take a fresh look. Ever the humble fact-checkers, we are happy to do so.

As followers of this saga will know, the NFL-commissioned Wells Report said that the "preponderance of evidence" led investigators to believe that two Patriots equipment staffers let air out of the footballs and that Patriots quarterback Tom Brady was at least generally aware that they did it. The NFL leveled penalties on the team and suspended Brady for four games.

The most damning evidence in the report were the texts the two staffers had sent to each other in the months before the game. In colorful language (read cuss words), they talked about ball pressure and Brady’s demands. One guy called himself "the deflator."

But hard forensic evidence was very much part of the report, too.

For this, lead investigator Theodore Wells and his team contracted with Exponent, a national engineering consulting firm, and physicist David Marlowe, former chair of the Princeton University Physics Department.

Scientists examined the pressure measurements taken by officials of 11 Patriots footballs from the AFC Championship game against the Indianapolis Colts. (They also had the pressure readings for four balls used by the Colts.) Knowing that the internal pressure would drop when a ball moved from a warm room indoors to the chill 50-degree weather outside, they factored in the temperature shift. They tried to account for how the balls were handled by each team and where they were kept during the game.

And this is what they found:

"Our scientific consultants ultimately informed us that the data alone did not provide a basis for them to determine with absolute certainty whether there was or was not tampering. Based on extensive testing and analysis, however, Exponent concluded that, within the range of game conditions and circumstances most likely to have occurred on game day, they could identify no set of credible physical or environmental factors that completely accounts for the magnitude of the reduction in air pressure of the Patriots footballs or the additional drop in air pressure exhibited by the Patriots game balls, as compared to the drop in air pressure exhibited by the Colts game balls.

"Dr. Marlow agreed with this conclusion. This absence of an explanation based on natural factors tends to support a finding that human intervention may have been a factor."

There are many details behind this, but in plainspeak, while they didn’t have exact pre-game measurements, the record showed that all the Patriots footballs started at close to 12.5 psi, the Colts’ at about 13 psi and the Patriots footballs uniformly lost more pressure than the Colts footballs.

There was only way they could get the pressure drops in the footballs of both teams to line-up. They had to assume that at halftime, the Patriots balls were brought indoors (where it was warmer) and measured immediately and were wet. (It was a rainy day.) The Colts footballs would have sat longer indoors. That would give them more time to warm up and with that, their pressure would rise.

The Exponent engineers said based on witnesses, they didn’t think this sort of timing gap took place.

Not so fast

The Wells Report came out May 6; about a week later, the Patriots had put up a special website to counter the report’s findings. We’ll focus on the science part of that site.

The Patriots brought in Roderick MacKinnon, a professor at Rockefeller University and a Nobel laureate in chemistry. MacKinnon drilled down on the basis for the entire case -- the actual pressure in the gameday footballs.

As it turns out, officials used two different kinds of pressure gauges and, as the Wells Report noted, officials didn’t record which gauge they used. MacKinnon used that flaw to argue that eight of the Patriots balls (only 11 were tested) showed pressure drops that could be explained by temperature shifts alone.

The other three, he argued, could be explained by errors in measurement.

"The scientific analysis in the Wells Report was a good attempt to seek the truth, however, it was based on data that are simply insufficient," MacKinnon wrote.

MacKinnon zeroed in on why at halftime officials only measured four Colts footballs compared to 11 from the Patriots. MacKinnon postulated that officials ran out of time, meaning the Colts footballs had sat at room temperature for some time before they were measured. A longer wait led to a higher temperature and so, a higher pressure.

A private center, HeadSmart Labs, also argues that gameday conditions could explain the drop in pressure in the Patriots footballs. Founder Thomas Healy said he came up with different results from the Exponent team because he put more water on his test footballs.

"If footballs are sufficiently wet, the footballs will experience a greater drop in pressure than was achieved by Exponent in the Wells Report.," Healy wrote. (Healy is trying to crowdsource funding to do more research.)

Brady got some help from an unexpected quarter, the American Enterprise Institute, a free-market think tank. Three economists shredded the Wells Report purely on statistical grounds. In layman's terms, they argued that the Wells number crunchers produced "biased results" because they dropped key variables from their equations. When the economists ran the data, the only significant deviation was the Patriots footballs had more pressure than the laws of physics would predict. In their view, the best explanation is that the Patriots footballs were measured early in the half-time break, and the Colts footballs were measured late.

Our favorite physics teacher

When we first looked into Deflategate we turned to Robert Hilborn, the second-in-command at the American Association of Physics Teachers. In addition to sharing his own analysis, Hilborn took this episode as a teachable moment and invited high school physics students to run their own experiments.

Hilborn told us he gave a close reading to the technical section of the Wells Report and that it appeared that the testing was done "rather carefully." On the other hand, he too noticed that it is unclear how the footballs were handled at halftime. Hilborn said that if the Patriots footballs were carried inside in a bag and taken out one at a time, that might explain the measurements.

"There would be little opportunity for the (Patriots) balls' temperature to come up to the officials' room temperature while the Colts had only four balls which may have rested on a table in the officials' room until they were tested and hence had a chance to warm up significantly," Hilborn told us.

The limits of science

Sometimes, the scientific evidence is overwhelming. If the thermometer tells us it’s 90-degrees outside, we know we’ll feel hot. But the process at the AFC Championship in January was not so simple and the record is not as clear. For the NFL, the preponderance of evidence told them all they needed to know. As Hilborn put it, the science "was not inconsistent with deflation."

But that leaves room for doubt. Brady has appealed the findings, hoping those doubts give him an opening he can break through and reach a better place on the legal playing field.