We need to talk about fake infographics

There is a growing belief that AI has become accurate enough that we can largely trust what it produces. Compared to where we were only a couple of years ago, I almost agree. Models that once struggled to produce recognizable logos can now generate (relatively) convincing market maps within seconds. It really is one of the most remarkable technological advances we’ve seen.

But that’s exactly why we need to become more careful with what we trust. (Even if there are trust issues, I do recommend you subscribe to my newsletter).

Recently I asked AI to generate a market map of the largest hotel technology companies. It looks quite ok, but it is a hot mess once you actually scroll down the list.

To someone working in hotel tech, the mistakes are obvious. To someone outside the industry, however, it looks entirely believable. That’s where the real danger begins.

AI now makes mistakes that look professional, and based on the way generative AI is designed, these mistakes will look more and more professional. The better the presentation becomes, the harder it is to separate confidence from correctness.

In my post I suggested that people at least put the disclaimer that it isn’t real research but AI outputs. Reading the comments I guess I am being too optimistic. People who are chasing clicks aren’t going to voluntarily say it isn’t real.

The thing is in fields we know, we see these things quite rapidly. But in fields where we are not experts ourselves it is much more insidious. If I ask AI about hotel technology or marketing, I can usually spot the problems because I know the right answers. But if I ask about medicine or law, how would I know which parts are wrong? More importantly, how would I know which parts are almost right but still lead me to the wrong conclusion?

AI works on probabilities. It predicts the most likely answer based on patterns in data. Reality requires a constant stream of judgement. We humans are full of context, exceptions, trade-offs, and grey areas (yeah sometimes too much). But generative AI is designed to give patterns. It will take a lot work for AI to understand something like judgement. Judgement is the millions of shades of grey between absolutes. Computers work in absolutes.

That is the human skill we need to cultivate with ourselves, our teams, our kids and everyone we know. It always was. But with convincing “conclusion machines” in our pockets - we need to seriously up our game here.

Hallucinations haven’t disappeared. They’ve simply become much harder to spot.