Huffington Post's Sam Stein says 'no one beta-tested' health insurance marketplace website

The rollout of the Obama administration’s health insurance marketplaces website has been so roiled with problems -- from a pricing glitch, to flawed data, to widespread log-in fails -- that even journalists from left-leaning media are calling for top officials to be hauled before Congress.

But while appearing on MSNBC’s Morning Joe, Huffington Post political editor Sam Stein went a bit too far in criticizing the agency’s prep work before the Oct. 1 launch of healthcare.gov.

Or at least that's what he told us when we asked him.

"It seems from all of the reporting, and from what I can gather, that up until very recently they (the Obama administration) didn’t realize how bad this was going to go out," Stein said on MSNBC. "And no one beta-tested the site, which is almost criminal when you think about it."

The beta-testing line got picked up by conservative outlets such as the National Review and the Blaze, and we wanted to check it out.

Stein directed us to a Washington Examiner report with a headline that somewhat contradicted his statement, which he acknowledged: "Troubled Obamacare website wasn’t tested until a week before launch."

"Unfortunately, in the rush of cable news, I didn't add the clause ‘until a week before launch,’ " Stein told us. "I should have done that and apologize."

That’s true. But Stein is more right than even he thinks, in part because he inadvertently used the word "beta"  to describe the specific website testing. In the IT world, there are many kinds of tests under many kinds of names.

Put another way, Stein goofed -- but by goofing, he got closer to the truth.

Here’s what you need to know.

The Examiner story, quoting someone speaking anonymously, said officials did not allow testing on the website until just days before it went live Oct. 1. The biggest problem, according to this report and others, was that leaders of the Centers for Medicare and Medicaid Services decided to act as the central coordinator for the project (CMS would not tell the Examiner if the agency holds that role).

The next day, the Washington Post described how government officials and contractors proceeded with the rollout despite a botched crucial test days earlier. They ran a simulation, unsuccessfully, in which a few hundred people tried to log onto the website at the same time. The failure proved an early warning of the bumpy road to come. Officials went forward with the launch, and the website "locked up" almost immediately when 2,000 users tried step one, the Post reported.

Unnamed sources told the Post that an end-to-end trial run of the process did not happen by as late as Sept. 26, the week before the launch.

(Here at PolitiFact we don't rely on unnamed sources, but the reports from the Examiner and the Post appear credible.)

"Beta testing" is a very specific term in the tech world that most professional software goes through before launching. It comes after a product is in an "alpha" phase, or the earliest version of software that is subject to some tests to find any big issues. When a product reaches beta, it is tested by a larger group of people not connected to its development in an effort to gather feedback and make more fixes before a large-scale launch.

Big tech companies like Google sometimes launch products in beta mode, but enterprise and government software vendors don’t usually do it that way, said Alexander Howard, former Washington correspondent for tech-centric O’Reilly Media and a fellow at the Tow Center for Digital Journalism at Columbia University. They stop adding features in the months preceding a launch to "start testing the heck out of it," Howard said.

The development of healthcare.gov has been an entirely different story, he said, with no use of those terms.

"The only thing we have is reporting that says the first testing occurred the last week before it went live, and then it went live," Howard said. "There was testing, but it’s not clear that it was in a beta version."

According to published reports, healthcare.gov got additional features quite close to the launch, with no evidence of a consumer-centric test that went outside of the government. So Stein might have inartfully used the word beta-testing, but he might also have a point.

Featured Fact-check

X posts

stated on October 22, 2024 in posts on X

A photo shows The Atlantic published a story headlined “Trump is Literally Hitler.”

"Even if those tests nominally did occur, they were as good as nonexistent based on the complexity of the project," said Patrick Ruffini, a Republican digital strategist.

Testing of any kind was difficult because the specifications for the website were evolving the month before the website went up, said Gail Wilensky, senior fellow at Project Hope and former Medicare director under former President George H. W. Bush.

The Post story, by the way, said a test group of 10 insurers that had access to the site advised the Centers for Medicare and Medicaid Services that the site was not ready to launch nationwide. Insurers have been in "regular communication" with government officials and contractors to test the back end of the site dealing with enrollment, Robert Zirkelbach, spokesman for the industry group America's Health Insurance Plans, told us.

But most people are getting hung up on the earliest steps of setting up an account and verifying who they are.

We reached out to CMS and CGI Federal, one of the main contractors for the website, but did not hear back. (We suspect they’re busy.)

Our ruling

Here’s what Stein said: "No one beta-tested" healthcare.gov.

Stein, in fact, meant to say something else, he told us. But at PolitiFact we check the statements as they are made.

In this case, Stein is closer to the truth than maybe even he thought.

Stein specifically referred to beta testing on MSNBC, which is a phrase that traditionally means certain members of the public were allowed to access the website well before it opened. There's no evidence to suggest that happened, and the federal government isn't talking.

Ultimately, this is a glitch of a statement talking about the glitches of a website.

But it rates Mostly True.