Essential Terms of the Authority Crisis

Through no small amount of dishonesty and incompetence, institutions of authority are crumbling and the nature of expertise is changing.

This last week, the CDC held their ACIP meeting to discuss whether or not they should recommend the COVID vaccines for children 6 months to 5 years old. While presenting on the danger of the virus for children, a slide was shown claiming that COVID presented as one of the leading causes of death for children.

Katelyn Jetelina @dr_kkjetelina

COVID19 is the 4/5 leading cause of death for ALL pediatric age groups

Kelley, who runs covid-georgia.com, saw this slide and immediately knew it was false. She has been tracking COVID data in excruciating detail in Georgia since the beginning of the pandemic and has recently become an expert on the CDC’s pediatric death data simply because it was such a disaster and she wanted to get down to the truth of the matter.

This slide above is no small error. Not only did it count the wrong number for pediatric COVID deaths, it compared all pediatric COVID deaths in a 26-month period to annualized deaths from other causes. This is a massive data error, and yet it persisted through a supposedly rigorous data check from 11 authors and was selected by top-tier scientists for their landmark presentation to the most knowledgeable experts in the field.

No one in any of these meetings recognized this error. This slide was presented uncritically to the nation’s top doctors and epidemiologists who are in charge of setting the national policy on COVID vaccines for children and no one even noticed it. It was spread uncritically by dozens more experts, including a former Surgeon General of the United States.

And this error was caught by a woman who tweets using just her first name only and runs fact-checker on the world’s most eminent scientists in her free time.

On the one hand, I’m delighted that what Kelley does makes an impact. It looks like her persistence will result in a re-evaluation of the paper from which this chart was taken. In that way, the system works. But the system utterly failed before it worked and it’s only working now because an internet rando is more knowledgeable and paying closer attention than our top scientists and doctors.

A Record Running on Repeat

In the last two years, we’ve watched this story play out with alarming frequency. High-profile experts who are running policy for the nation (or even the world) show themselves to be woefully uninformed in their field of expertise. Over the last two years, a huge portion of this very newsletter has been about looking at nationwide narratives concerning COVID data and asking if they are actually true.

I’ve started asking myself: Why are we continuously playing whack-a-mole with bad policy drawn from poor science? It is impossible to do this forever, and for many people, it’s impossible to do this at all. Most people don’t know where to go to get the data and they wouldn’t know what to do with it if they got it.

Alex Meshkin, GED @alexmeshkin

We are living through the death of credentials and experts

This is the core of the authority crisis. After so many devastating and public failures at the highest levels of expertise, it seems untenable to give them credence simply based on their credentials and institutional positions.

But what is the alternative? I’ve noticed that even those who claim to be the most skeptical of credentialed experts haven’t abandoned the concept of knowable truth. They haven’t even abandoned the idea that they can know truth through science. They still listen to and highlight credentialed experts, just different ones.

I’ve been thinking about this for a while. I want to answer it slowly because I believe we are entering a time of history in which we need to carefully evaluate how we come to believe true things about the world around us.

What We Think We Know

When we think about how most people come to understand a new scientific truth, it’s helpful to establish a vision of how this process works.

A very simplistic diagram of scientific knowledge creation, for reference only.

Ideally, we start with the World As It Is. Much of this world is a mystery to us, many unexpected things happen and we wish to understand them.

In order to understand these things, we start collecting data about our world. What temperature is it? How many people are there? Where did they come from? Are they short or tall? Fat or thin? Where do they live? How do they live? We pull together various metrics as a way of sorting the world into categories and measurements.

This data is looked at Very Closely by scientists. Often they use Math or its bastard cousin Statistics to look at the data. Then they make charts and write about what they think is happening in the data. Ideally, several scientists will look Very Closely at the data and write about it several times, coming to similar conclusions.

If enough scientists do this enough times, Big Experts start to notice that the data seems to be consistently pointing in the same direction. They will make these conclusions Official by having a panel of experts agree that, yes, the conclusions seem to be soundly based on repeated analysis of the data.

These conclusions will be reported in the news or written about in books that are interesting enough for people to buy them. Over time, the public comes to accept these conclusions and we all agree that this thing about the world is true.

This is the rough sketch of how most people assume an idea moves from “something we think is happening” to “an established truth that all right-thinking people acknowledge.” An important component here is that this process typically takes a lot of time. These ideas are not established in months, but through decades of careful data gathering, analysis, checking and re-checking. Truth generated this way is a slow-moving beast but all the more reliable for its ponderous nature.

The Trust Problem

We take it for granted (or perhaps we took it for granted) that every step along the path above is permeated with the presumption of trust. We have to trust the data is being gathered the right way by people who care only about gathering accurate data. We have to trust the people interpreting the data are passionate to incorporate all the data they can get and eager to correct mistakes when they are discovered. We have to trust the institutional powers to consider all available analysis with a dispassionate eye. We have to trust that the findings are being accurately reported to the public in the popular media.

If we can trust the people moving knowledge down the appropriate path, we can read the newspaper or the latest popular science book and feel confident that we have new insight about the world.

The vast majority of my public work over the past 14 years has been focused on the final step in this process. Popular media is hideous at accurate reporting on detailed scientific information. But I have largely assumed that by the time something gets to the point of a published scientific paper, it has gone through most of the important steps above guided by individuals concerned primarily with accuracy and truth.

I’m not sure I believe that anymore.

Over the next few issues, I hope to lay out both the failures in the process as it currently exists and a vision of how to move forward. My plan is to lay out a record of what has gone wrong in every step of this vision of knowledge creation. People have become skeptical about every level in the knowledge creation pipeline and that skepticism is corrosive.

Lest we jump to conclusions, I want to note that this skepticism is not partisan. It is constant across party lines. Everyone is skeptical of all information they consume that does not reinforce their existing vision. There are attacks on knowledge from all sides across the entire knowledge generation pathway.

I don’t believe that knowledge creation is broken. Most people still end up applying the model above. But we are seeing massive ripples in how people apply this model and along what lines. The vision of a centrally managed authority for knowledge verification is dead as a doornail. We need to think about what comes after that.

This project is about figuring out how people in a world untethered from authority are building their own vision of the path to truth. I’ll be investigating how we are creating knowledge, how people are manipulating the knowledge creation process (to the detriment of the all-important virtue of trust), and how we can shelter our minds and build a vision of the world that is as close as possible to how it actually works.

Disney Shorts: Donald Gets Drafted (1942)

This is a short that was almost certainly fast-tracked into production to support the US entry into World War II. It was released just 5 months after the attack on Pearl Harbor & sets Donald as a patriotic draftee eager to join the war effort. Donald happily strolls past the military sandwich boards, signs up, goes through his physical, and ends up in boot camp with Pete as his drill instructor.

This short is a pretty lighthearted view of military training, making fun of military propaganda, the Army physical (“what color is this green card?” “blue?” “close enough”), and the demands of boot camp. It seems like it was written and produced to give a worried public a laugh at the prospect of thousands of young men being suddenly thrust into the military.