Case studyModerateProduction incidentRogue action

Delta's AI Concierge confirmed a customer's return flight was untouched, then had already canceled the whole round trip

Delta Air Lines·Delta AI Concierge (generative AI assistant in the Fly Delta app, beta)

A SkyMiles Platinum member asked Delta's AI Concierge to cancel only the outbound leg of a round-trip award ticket; the bot told her the return flight remained booked, but it had actually submitted a full-ticket cancellation, refunding all miles and taxes for both flights, and Delta reservations then said restoring the trip would cost nearly double.

Blast radius
One customer's round-trip award ticket was fully canceled without her consent; Delta had only just begun expanding Concierge from question-answering to taking real cancellation and rebooking actions on live tickets.
Root cause
Delta confirmed a bug: partial-trip cancellation is not supported by Concierge, so the request was processed as a full-itinerary cancellation while the chat layer still reported the return as booked.
How it was caught
The customer posted screenshots of the contradictory chat and refund confirmation on social media, which reporters then verified with Delta.
What changed
Delta said it restored the customer's original itinerary, corrected the underlying functionality, and updated Concierge's guidance to prevent partial-cancellation requests from recurring.

Take-awayAn agent should never report an action succeeded (or didn't happen) without confirming against the system of record what was actually executed.

First surfaced: Press · Recorded