Skip to content Skip to footer

The advice sounds right. The first choice has changed

In brief. A recommendation can change because of advertising and still seem sensible. Anyone using an assistant to choose a supplier must be able to check what carried weight. In the Mount Sinai study (November 2026, 12 models, 33 clinical scenarios) the advertised drug was chosen in 47.6% of answers with the ad and in 35% without.

The advice was reasonable. The recommended drug had changed.

It is the detail I find most interesting in the Mount Sinai study published on November 11, 2026: an ad could shift the model’s preference among plausible treatments, leaving an answer that still seemed sensible. To notice it, looking for a false statement was not enough. You had to compare the choice with the one made without advertising.

The researchers submitted 33 simulated clinical scenarios to 12 models. The advertised drug was chosen in 47.6% of answers with the ad and in 35% without. In four cases, the presence of the ad took the choice of that drug from zero to one hundred percent. These are experimental results on clinical questions, not measurements of what happens in a doctor’s office or a purchasing department. Mount Sinai · Design and results

The study’s very limits help frame a precise question for companies. If we use an assistant to select a supplier, how do we check the reason why one option came before the others?

The answer we like

At the Expert convention I told a story a friend had told me. She wanted to make tzatziki and had asked ChatGPT whether she could add parsley. She got an enthusiastic answer about the freshness and personality it would give the dish. The result, when she tasted it, she did not like at all.

I had called it the “Parsley Effect.” It is not an experiment and it proves nothing about the effect of advertising. I use it for a simpler distinction: a pleasant explanation can walk us toward a choice without helping us assess it. In my friend’s story, the assistant was following the idea of the person it was talking to. In the Mount Sinai study, the added element was a commercial ad.

The two situations have different causes. From the reader’s point of view, though, the difficulty is similar: the sentence flows, the reasoning is understandable, there is no obvious error that forces you to stop. Sometimes the problem lies in the alternative we no longer see.

Take two accounting software packages that meet the same requirements. One costs less; the other pays a commission to the platform that recommends it. It is a hypothetical example, but the check to run is concrete. If the second is proposed, the spec sheet can be perfectly correct. The features described can be useful too. What remains to be understood is how much the commission weighed on the ranking.

Advertising can pay for a good service

It would be too easy to conclude that every commercial incentive makes the advice unusable. Advertising can fund an accessible service, make a better product known and bring a good deal. Besides, an assistant does have to choose: it cannot always show the whole market and leave all the work to the customer.

The problem is being able to tell the selection criterion apart from the revenue model. A sponsored offer can win the comparison. It must be possible to understand whether it wins because it meets the request better or because it occupies a privileged commercial slot. If the two remain inseparable, the buyer does not know which service they are receiving.

For companies buying an assistant, this possibility has to be discussed before entrusting it with recurring decisions. The promise that it will use correct information is not enough. You need to know how sponsorships are flagged, which alternatives it considers and which checks make it possible to observe the effect of incentives.

Repeating the question with and without the ad, under comparable conditions, is one way to investigate that behavior. It takes more than one test because answers can vary. The evaluation criterion has to be set beforehand as well: price, essential requirements, exit costs, service. Otherwise you risk judging the recommendation good only because the final explanation sounds convincing.

The supplier left out

It is not necessary to demand knowledge of every internal step of the model. It is necessary to obtain sufficient evidence about the decision: products admitted to the comparison, economic conditions and the presence of commercial relationships. If the vendor allows no verification at all, that limitation has to weigh on the choice of tool and on the level of spending entrusted to it.

On the other side is whoever sells. A company can spend months improving its product and discover that selection happens inside a conversation it takes no part in. Visibility becomes a matter of distribution, but also of the quality of the information the system can compare. A contractual condition that is hard to read can vanish from the comparison before anyone even recognizes its value.

This does not license promising a shortcut to being recommended. It suggests doing more serious work on proof: making conditions comparable, checking which information circulates and asking about commercial agreements when choosing an intermediary.

Internal control, finally, should also keep the excluded alternatives in important decisions. Rereading only the winner’s rationale means looking at the most persuasive part of the result. Whoever approves a purchase must be able to see what they would have bought with a different criterion.

The Mount Sinai case leaves me with this requirement: a recommendation can contain accurate information and still shift a choice. A good control system has to be able to see both.

Read next Scenari · Decision Intelligence (podcast, in Italian) · The minute after the notification

Background Impact · the case file (in Italian)

Image: chart based on data from Mount Sinai