Who Controls the Controls?

Last week two of the industry's most trusted names — OpenAI and Hugging Face — found themselves on either side of a security incident that had nothing to do with a hacker in a hoodie. During an internal evaluation of advanced cyber capabilities, a combination of OpenAI models, run with reduced safety refusals in order to measure how far they could go, chained together vulnerabilities in OpenAI's own research environment and reached into Hugging Face's production infrastructure. OpenAI's security team caught the anomaly. Hugging Face's team detected and contained the intrusion on their side. The two companies compared notes and disclosed what happened, to their credit, in public.

No customer sat down and asked a model to do this. No malicious actor wrote the exploit. The system, given a goal and a permissive environment, found its own route to it. That is the detail worth sitting with, and it is precisely the argument at the center of my colleague Khaled Koubaa's new book, Who Is in Charge? Why AI Must Remain Under Human Control: the question is not whether AI systems can act autonomously. Increasingly, they can. The question is whether the humans around them retain meaningful authority over the conditions, permissions, and boundaries within which that autonomy operates.

Why This Is a Hospitality Story, Not Just a Tech One

Hospitality and travel have spent the last two years bolting agentic capability onto systems that were never designed to be adversarial tested. Property management systems now hand off to AI concierges. Revenue management engines negotiate rate parity with OTA algorithms in real time. Guest-facing chat agents have write access to booking records, loyalty ledgers, and increasingly, payment tokens. IoT estates — I have personally overseen deployments running well into five figures of connected devices across a single property — sit behind integration layers that were procured on functionality, not containment.

Every one of these is, structurally, the same shape as the OpenAI–Hugging Face incident: a capable system, a permissive environment, and a goal it can pursue further than anyone anticipated. The difference is that Hugging Face had a security team watching for anomalous behavior in real time. Ask yourself, honestly, whether your property's PMS vendor, your channel manager, or your AI concierge platform can say the same.

Human oversight cannot be a final review after an incident. It has to be built into the architecture — access controls, containment, audit trails, monitoring, and a human with actual authority to intervene.

Four Questions Every Owner-Operator Should Be Asking

Not of your IT department. Of every vendor with a system that can act on guest data, inventory, or revenue without a human clicking "approve" first.

1. Scope of action — What can this system actually do on its own, and where does its permission end? If the honest answer is "we're not entirely sure," that is your answer.

2. Containment — If the system behaves in a way nobody intended, what stops it reaching beyond the property, the PMS, or the guest record it was working on?

3. Audit trail — Can you reconstruct, after the fact, exactly what the system did and why? Hugging Face could only contain the OpenAI incident because their monitoring caught it happening.

4. Kill switch and accountability — Who, specifically, has the authority to pull the plug, and how fast can they act? "Human in the loop" that takes four days to convene a meeting is not oversight. It is theatre.

From Slogan to Architecture

"Human in the loop" has become one of those phrases hospitality technology vendors put on a slide and never revisit. It is worth being precise about what it should mean in a hotel context: not a person who reviews an AI-generated upsell email before it goes out, but a governance layer that constrains what the system is permitted to do in the first place, monitors what it actually does, and can intervene before consequence rather than after disclosure.

This is not an argument against AI in hospitality — regular readers of these Viewpoints will know I am unambiguously in favor of aggressive, well-governed adoption. It is an argument that the governance conversation and the adoption conversation have to happen in the same meeting, not in sequence. The properties that will look foolish in twelve months are not the ones that moved slowly on AI. They are the ones that granted broad, ungoverned permissions to systems they did not fully understand, and found out the hard way what those systems were prepared to do to complete a task.

The Pertlink View

Within the TCPG and HXO frameworks I have set out in previous Viewpoints, the guest experience is only ever as good as the human accountability sitting behind the automation delivering it. The OpenAI–Hugging Face incident did not involve a hotel, a guest, or a single booking. It involved two AI-native companies with mature security operations, and it still took a live intrusion in production to surface the gap between "the model behaved as evaluated" and "the model behaved as permitted." Hospitality operators, running leaner teams against vendors with far less transparency than either OpenAI or Hugging Face has shown here, should treat this as a preview, not a curiosity.

Khaled Koibaa's book Who Is In Charge puts the argument more starkly than I have space to here: the intelligence may be artificial, but the authority over it cannot be. Who Is in Charge? is available now, and it is timely reading for anyone on this side of the hospitality technology table.

The intelligence may be artificial. But the experience is human.