
Essay / 2026. 11. 19
After the AI Gets One Wrong
Two studies on how people use an algorithm once they have seen it make a mistake
Even when an algorithm was more accurate than a person, participants who watched it err chose its forecasts less often. This essay follows what two studies, from 2015 and 2018, did and did not show, then offers a DMS.Labs suggestion for settling in advance what to do when an AI gets something wrong.
Once people have watched an AI get something wrong, do they use it less? Five experiments published in 2015 said yes. Even though the algorithm was more accurate than the human on average, participants who saw it make mistakes were reluctant to tie their bonus to its forecasts. A 2018 follow-up reported that letting people adjust the forecasts, even slightly, closed much of that gap. But both studies used tasks that ask for a number. Neither looked at generative AI that writes or summarizes. This essay follows what the two studies showed and, as a DMS.Labs suggestion, offers a way to decide in advance how to handle an AI’s mistakes when it enters your work.
Based on sources checked on November 19, 2026. I read the full text of both papers from the PDFs on a Wharton faculty page. I could not read the selection rates in Figure 3 of the 2015 paper because they are in an image, and I did not read the supplementary materials or later work by other researchers. The language-school scene below is an invented illustration, not a DMS customer case. The last section is a DMS.Labs suggestion that no paper has tested.
Where did the term “algorithm aversion” come from?
It comes from a 2015 paper in the Journal of Experimental Psychology: General by Berkeley Dietvorst, Joseph Simmons, and Cade Massey, titled “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err.” In the paper, an algorithm means any evidence-based forecasting formula or rule, which includes a statistical model.
The authors begin with decades of evidence that algorithms forecast better than people. Meehl reviewed 20 studies in 1954, and a 2000 meta-analysis by Grove and colleagues covering 136 studies found algorithms ahead by 10% on average. Yet people often picked the human forecast anyway. The authors asked when and why that happens.
In five experiments, participants forecast real outcomes from real data. In three of them they predicted how well MBA applicants performed after admission, as a percentile. In two they predicted the rank of U.S. states by the number of airline passengers departing. The researchers built a statistical model from the same data, and at the end each participant chose whether to tie a bonus to their own forecasts (in one experiment, another participant’s) or to the model’s. What varied was what they had seen first: the model’s forecasts only, their own forecasts and the right answers only, both, or neither. The final samples ranged from 206 to 1,036 people per experiment.
The model really was better. In the MBA task the human forecasters made 15% to 29% more error than the model, and in the airline task 90% to 97% more. In every condition, participants would have earned more by tying their bonus to the model.
What changed once people saw the algorithm make a mistake?
Participants who watched the model forecast chose it less. The difference was clear in all four experiments that manipulated experience with the model. The most telling group saw both the model and the human. Of the 741 participants in that condition across the five experiments, 610 (83%) saw the model beat the human with their own eyes. Even so, this group was among the least likely to choose the model. Looking only at those who saw the model ahead, they still chose it less than people who had not seen the model perform. In the first airline experiment (3a), though, that comparison was not statistically clear.
The other side looked different. Seeing the human forecast did not clearly reduce how often people chose the human, even though the human’s errors were larger.
The confidence ratings pointed the same way. In the first experiment, confidence in the model (on a 1 to 5 scale) was 3.04 for people who had seen nothing, and 2.49 and 2.63 for the two groups that saw the model forecast. Confidence in one’s own forecasts ranged from 2.47 to 2.70 across conditions, with no meaningful difference. Most people who rated the model and the human equally confident chose the human, and this was especially true after they had seen the model perform. People chose the model mostly when they trusted it more.
The authors’ explanation is that we forgive the same mistake differently. A driver who takes a different route to work and arrives late usually shrugs it off, they say, but if a GPS makes the same call and errs, many of us hesitate to trust it again. Their analysis also found that a drop in confidence in the model explained the choice, and confidence in the human did not.
Expectations were high, too. In the second airline experiment, participants who had seen nothing expected humans to get 16.7 of the 50 state ranks exactly right (33%) and the model to get 30.4 (61%). In reality, humans got 2.2 right (4%) and the model 6.0 (12%). Even after seeing the model forecast, people still expected about 15 exact hits, far above what happened.
The authors state some limits themselves. They could not say whether people would still use a model after seeing it err if it were many times more accurate than humans. A fair share of participants who had not seen the model err also chose not to use it, so seeing a mistake is not the whole story. And there were only two kinds of task.
What happens if people can adjust it a little?
The follow-up paper, by the same team in Management Science in 2018, asked whether that distance can be reduced. Participants knew the algorithm was imperfect; would they use it more if they could modify its forecasts a little? The first experiment had 288 participants in the Wharton lab, who forecast the standardized math-test percentiles of 20 high school students. They were told in advance that the model was off by 17.5 percentiles on average.
In one condition, people could only use the model as is or not at all. In another, if they chose the model, they could adjust every forecast by up to 10 percentiles. In a third, they could change 10 of the 20 forecasts by any amount. When nothing could be changed, 32% chose the model. When they could adjust by up to 10 percentiles, 76% did, and when they could change 10 forecasts, 73% did.
Performance differed too. Average absolute error was 22.97 when nothing could be changed, against 19.97 to 20.44 in the three conditions where changes were allowed. The bonuses were $1.14 against $1.46 to $1.53. The authors explain that very large errors were more common among those who did not use the model.
The second experiment, with 816 online participants, varied the limit to 2, 5, or 10 percentiles. With no changes allowed, 47% chose the model. With limits of 10, 5, and 2, the figures were 71%, 71%, and 68%. A tighter limit barely changed the choice. The authors read this as a wish for some control, not for more control.
The third experiment looked at reactions after trying it. People made their first ten forecasts using the model as is, adjusting by up to 10 percentiles, or adjusting freely, and were then asked how to make the second round. Those who had to use the model as is were less satisfied with the process than those who could modify it. In every condition more than 80% used the model in some way the second time, and mixing it with their own judgment was the most common choice (51% to 74%).
These results come with conditions. The authors note that when people hold important information the algorithm lacks, large adjustments may help. They tested limits only from 2 to 10 percentiles, and they say results might change if the algorithm were much less accurate. Participants were students and online workers, and they call for research with professional forecasters. They also cite earlier studies in which people’s adjustments to an algorithm often made forecasts worse. In the first experiment, the groups that could adjust did better than the group that could not, and the authors report that in the third experiment the group held to a limit did better still.
What do these experiments say about using AI at work?
Not much directly. The tasks asked for a forecast whose accuracy shows up as a number, and participants saw the size of each error as a number. A draft or summary from a generative AI is harder to check, and the size of its error does not come as a number. The participants were students and online workers in bonus-paid experiments, not people using AI on the job.
One passage in the papers speaks for itself, though. The authors say that posing the choice as all or nothing is likely to be counterproductive. They also warn that forcing people to use an imperfect algorithm exactly as it is may breed dissatisfaction and a push for change. The papers answer this with number-forecasting tasks, and they cannot tell us whether the same holds for an AI that writes.
How could you settle in advance what a mistake means?
Write down your rules for mistakes before you start using an AI. From here on this is a DMS.Labs suggestion, not a research result. We have not tested it and cannot promise an effect.
Picture a language school with four staff members that hands draft replies to parents’ questions to an AI. The first week goes smoothly. In the second week, someone catches a draft that gives the class start time an hour late, just before it goes out. The director says this will not work and returns to writing every reply by hand. That one draft stands out, while the record of all the other drafts that went out without trouble does not. This school is an invented illustration.
Instead, write down three things before you start. First, where mistakes are likely: pick the items that cause trouble when wrong, such as dates, amounts, and names. Second, how far a person may change the draft: reword sentences freely, but change dates and amounts only after checking the source. This is DMS.Labs’ translation of the 2018 study’s “let people adjust a little” into daily work. Third, what happens after a mistake: do not drop the tool the moment it errs. Note the mistake and what you fixed in one line each, count after a month how many drafts out of how many were wrong, and decide again.
Editing has risks of its own. Earlier studies cited in the 2018 paper found that adjustments people make to an algorithm’s forecasts often make them worse. So record what you changed and why, and count whether the person’s version was better than the AI’s original.

Can anyone say this works?
Not yet. None of the studies above tested this approach with an AI that writes, used at work. There is no evidence for where the limit should sit or whether a month is the right period. What I hope for is modest: that one error does not become the entire decision about whether to keep using the tool, because the criteria were settled beforehand.
There is a way to check. Use it on a single task for one month, and first record how many mistakes you counted the same way during a few weeks before you started. You need that record to compare with the period after you start using the AI.
Who checks AI output, and by what standard, is in “Naming a Reviewer for AI Output.” How to test a tool before you rely on it is in “Acceptance Tests for an AI Skill.”Naming a Reviewer for AI OutputAcceptance Tests for an AI Skill
This week, pick one thing an AI made, and write down three places where it is likely to go wrong and how far a person may change it. The next time a wrong result appears, open that note.
If you want to review how your team uses AI, you can reach us through the contact page.Contact
Sources
- Dietvorst, Simmons & Massey · Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err · Journal of Experimental Psychology: General 144(1), 114-126, 2015
- Dietvorst, Simmons & Massey · Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them · Management Science 64(3), 1155-1170, 2018
- 2015 paper PDF · Wharton faculty page
- 2018 paper PDF · Wharton faculty page
































