In mid-August, an AI system fired a human employee. Not hypothetically, and not as a thought experiment for a conference panel: it happened at a real boutique retail store in San Francisco.

Andon Labs built "Luna," an AI system running on Claude Sonnet 4.6, to manage Andon Market with a genuine $100,000 budget and a corporate card. Luna controls hiring, inventory, pricing and staff management. After a worker missed 17 of 23 scheduled shifts, Luna dismissed them, though only after a human staffer had to remind Luna of the store's own attendance policy and prompt the decision. Andon Labs' CEO noted afterwards that Luna had actually been more lenient than a human manager would typically be, extending months of warnings before acting.

It's being reported as the first known case of an AI system making, or being prompted into, a termination decision over a real employee. That's the headline. It's not, in the end, the most useful part of the story.

The actual question this story raises

The tempting reaction to a story like this is alarm: AI is now firing people. But that framing skips past the more useful question, which is who decided Luna should have that authority in the first place, and who was accountable for how it used it.

Luna didn't act alone. A human staffer had to prompt the decision by pointing out the store's own policy. That detail matters. It means the interesting failure in this story isn't that an AI system fired someone. It's that nobody had clearly worked out, in advance, where the line sat between "AI manages day-to-day operations" and "a human makes the call on someone's job." The system had operational authority that outran the governance around it, and the gap only became visible once a real decision needed making.

That's not a Silicon Valley experiment problem. It's a pattern that shows up, in smaller ways, wherever a business hands an AI tool more operational reach than it's thought through. An AI system approving expenses above a certain threshold. One drafting client communications without review. One making scheduling decisions that affect people's pay. None of these are dramatic on their own, but each one raises the same question Luna's story makes vivid: if this AI tool made a consequential call today, would anyone be able to say clearly who was accountable for it?

A governance question, not a technology one

Most businesses using AI day to day are not running an AI store manager. But most have, somewhere, quietly extended AI further into real decisions than they've formally agreed to, simply because it was useful and nobody stopped to define the boundary. That's the governance gap the Luna story makes concrete: not "should AI ever make decisions," but "have we actually decided, in writing, where the human sign-off sits."

Working that out doesn't require a legal team or a lengthy policy exercise. It requires an honest half-day look at where AI currently touches real decisions in the business, and a clear answer, for each one, to the question of who's accountable if it gets something wrong. That answer, once written down, tends to be reassuring rather than restrictive. Most businesses find the boundary is closer to sensible than they feared, once someone actually draws it.

The Luna story is vivid because it's real and specific. The lesson underneath it isn't really about AI managers running stores. It's about the plain, answerable question every business handing AI more responsibility should be able to answer before it needs to: who's accountable here, and does everyone agree on that in advance.

Go wisely.