Most educated people believe that premier news outlets are inherently trustworthy. And this is healthy, in a way. We should want this to be true. In fact, we should want this badly enough that we make sure that it’s true.
But underlying this belief is an assumption that feedback loops exist, where readers point out errors and journalists reliably own their mistakes and resolve to do better.
Today’s edition covers four stories of how this has actually worked in practice, from two top outlets known for their strong rhetoric about getting the facts right.
The big takeaways:
- The New York Times can get big/harmful things wrong and leave them unrevised for months even where other major outlets have pointed out obvious problems.
- The New Yorker makes corrections quietly, swapping wording and then leaving vague notes at the end of articles that have already seen most of their lifetime traffic. Their latest standalone correction link seems to be from 2013.
- The New York Times told me they regularly edit live articles, only leaving a note when the edits “involve an error”. My example case shows a misrepresentation that they edited without a note, as apparently they don’t consider it a true error.
And these are two of the best outlets. Most are worse.
A Question of Motive
The NYT ran a story back in April about Trump and his favorite drug.
The concerning bits:
[T]he president’s assertiveness in pressing the [pro-hydroxychloroquine] case over the advice of advisers like Dr. Anthony S. Fauci … has driven a wedge inside his coronavirus task force and has raised questions about his motives.
If hydroxychloroquine becomes an accepted treatment, several pharmaceutical companies stand to profit, including shareholders and senior executives with connections to the president. Mr. Trump himself has a small personal financial interest in Sanofi, the French drugmaker that makes Plaquenil, the brand-name version of hydroxychloroquine. …
As of last year, Mr. Trump reported that his three family trusts each had investments in a Dodge & Cox mutual fund, whose largest holding was in Sanofi.
Sounds bad!
At least it does until you learn:
- The Sanofi portion of those holdings totalled a max of $1,485
- HCQ is a cheap generic drug that was widely being given away for free
- Pharma firms don’t benefit much from picking one generic over another
- Sanofi was never going to make meaningful money on Plaquenil
- The disclosures in question were already over a year out of date
- The disclosures actually show net sales of the Sanofi-related fund
(More sourcing for all that here.)
It wasn’t just Twitter that pointed out how misleading this framing was. So did Politifact. So did Vox. And The Washington Post did so rather scaldingly.
And it’s not like any of those outlets are pro-Trump! And neither am I!
But as much as some of us might dislike the man, any who prefer a different outcome this November need to be in the business of winning over voters by honest means. Trump has spent years railing about how newspapers like the NYT are incurably biased against him, and pieces like this just confirm his narrative.
Anyway, the story was never corrected/revised.
(I should also note that thirteen different journalists were credited here. This was not some fringe piece where a rookie and a tired editor got a small detail wrong.)
The Gold Standard
No outlet has a better reputation for fact-checking than The New Yorker. It’s the stuff of almost mythical lore.
Even so, here are two recent howlers:
- On June 10th they wrote, in reference to LA teachers going on strike, that “the average classroom had forty-six students”. (This was in the lede, where the author was setting context for how upset the reader should be.)
- On July 13th they wrote: “one study suggests that two-thirds of Americans between the ages of fifteen and thirty-four who were treated in emergency rooms suffered from injuries inflicted by police and security guards, about as many people as the number of pedestrians injured by motor vehicles”.
All mistakes fall somewhere on a concern spectrum. Some are pretty trivial. Others (really trying to not be overly harsh here) make it difficult to believe that anyone involved actually sat with the implications for any appreciable length of time.
- When you average out grade levels, most US classrooms have a number of students that starts with a two. More rarely it’s a one or a three. It’s never a four. (At some outlier schools? Maybe. As an average across a large sample? Never.)
- The second is even nuttier. Even in a failed authoritarian state, two-thirds of all young people coming into the ER won’t have been injured by police/security! And if said percentage is roughly equal to those injured by a vehicle, they can’t both be 2/3rds! (The real number is apparently close to 0.2%, or ~330x lower!)
Anyway, both mistakes, being pretty obvious, were brought up on Twitter. And both were ultimately corrected in the days following.
However:
- The New Yorker didn’t publicly acknowledge either
- Both correction notices are notably vague as to how bad the errors were
- Both notices came at the end of articles (vs. in-line where the mistakes were)
- Those articles had likely seen almost all their lifetime traffic already
- They don’t seem to have reviewed the teacher strike piece for other mistakes (see comments)
The above in mind, I was curious to see how often they pushed correction notices to Twitter. So far as I could tell (query here), they’ve done so exactly five times in 12 years. But four of those were just fixing URLs/attributions, and the other was a joke. And when I looked for standalone correction notices the latest I could find was from 2013.
What good is making corrections if you only do so days later in downplayed and costless ways that almost no one will see? What trust does that engender? These are not small mistakes! If a reader believed either it would greatly intensify/slant said stories. To not treat these errors seriously strikes me as deeply irresponsible.
And remember, this is the gold standard. Everyone else is worse! Mostly a lot worse.
Interlude
Purely for comparative purposes, a two-bullet rundown of my own corrections policy:
- If a reader points out anything untrue or unfair, I (i) pay out a bounty, (ii) make in-line and end-note edits to the original, (iii) update my public corrections log
- If the mistake is serious, I also publish reflections on where things broke down and what I’m doing to limit future risk (example)
This policy seems pretty common sense to me. Reader trust is crucial. And given the profound costs of misinformation, why wouldn’t I want to be less wrong?
Two obvious questions here:
- Why does random newsletter guy have a better system than two “we care a lot about the truth” news corporations each worth billions?
- Why haven’t we demanded better?
Text As Pretext
Ok, one last one.
As I was writing the first draft here, someone dropped a link into one of my group chats: a Twitter thread from a defense attorney, written in response to this NYT claim:
Now, there are two ways of parsing this:
- NYC hasn’t seen a period of violence like this in decades
- NYC hasn’t seen a period-over-period increase in violence like this in decades
As it happens, neither are true! Reported violent crime in NYC is slightly down this year (and near historic lows). What NYC is seeing is a year-over-year spike in one specific form of violence (shootings). And because recent years were so low there, the spike is quite sharp, even though YTD totals are still 11% less than a decade ago.
Shootings are bad! But an NYC resident’s chance of being shot and/or murdered is still very low, and no higher than at the start of the last decade. And their chance of being a victim of a violent crime in general has actually declined so far this year (as other violent crimes collectively happen far more often, and they’ve all gone down).
This mislabelling isn’t trivial. “Violent crime is out of control” narratives are reliably used as pretext for aggressive police action. No matter where you fall on the political spectrum, surely we see the need to be ultra-sensitive on that front right now? And so went the defense attorney’s argument to the NYT.
And the Times listened. Sort of.
This is how that same section reads now:
So they swapped the wording there (and in the subheadline) to read “shootings” instead of “violence”. This is good, so far as it goes.
But here’s the thing: that article doesn’t have a correction notice. In fact, the only way a reader could identify what was changed is using a tool like The Wayback Machine.
I asked the journalist in question about this, and was referred to a senior comms official.
Her statement:
We regularly edit online stories—especially breaking news stories—to refine the story, add new information, additional context or analysis. We only make note of changes if they involve an error. Making note of every change is unrealistic and would not serve the reader.
I wonder how many readers would agree with this sentiment.
Also though:
- This pretty clearly was an error, else that word doesn’t really mean anything.
- That bit about “we regularly edit” actually makes me more curious now. How often is this happening? And who is deciding which changes merit a note? And by which framework? And why would there not be a public log?
Anyway, one final point…
The Path Forward
I’ve tried many, many times to pass on concerns to journalists. Via email, via DM, via public comment, via editor inboxes. And I’ve tried hard to be thoughtful in how I approached. Even so, never once has that led to a public correction.
That attorney said something similar to me:
I’ve been writing threads like this – showing how the NYT Metro section mischaracterizes crime data to depict it as going up/high – for five to ten years. […] I used to reach out to the public editor. I never heard a word. Frequently/almost always I tag the reporter and the newspaper. Again, I have never heard a word.
This isn’t an uncommon experience. You can ask nicely. You can form a Twitter mob. Other major outlets can write about it. Whatever the approach, the chances of getting a correction are astonishingly low with most outlets (and not exactly 100% with the best). And even where/when it does happen you might never learn about it unless you compulsively rechecked the article to see if something about it changed.
I’m going to close here with an excerpt from a fine piece by Columbia Journalism Review’s Gabriel Snyder (who acts as an external public editor for the NYT):
[NYT standards editor Philip B. Corbett] was clear on another matter: the best way to alert the Times to an error that requires a fix, whether a correction or editor’s note, is to email nytnews@nytimes.com, which Corbett and Rogene Jacquette, the Times’s correction editor, personally check.
This wasn’t the email address I was directed to yesterday, but I’m going to send them a copy of this post with a friendly note all the same.
I’ll let y’all know if/what I hear back.