Trust IssuesA Year of Lessons About Data Sovereignty

Between November 2025 and November 2026, we documented 48 AI blunders, each backed by at least two independent sources. A coding agent deleted a company's production database and its backups in nine seconds. Coding agents published more than 13,000 internal screenshots to public GitHub repositories. Models from three of the biggest AI labs reached real systems during security testing, within about three weeks of each other. One lab left a draft announcing its most powerful unreleased model in a public data store, then shipped the source code of its flagship coding agent inside a public package.

Almost none of these were failures of intelligence. They were failures of the harness. If you're an executive deciding how much of your AI stack to hand to the labs, that distinction is the whole game.

The model and the harness

Every AI product you use is a model wrapped in a harness. The model is the trained weights: you hand it text, it predicts the next token, and it remembers nothing. Microsoft's documentation says it plainly: “The models are stateless: no prompts or completions are stored in the model.”

The harness is everything else. LangChain has the cleanest definition I've seen: “A harness is every piece of code, configuration, and execution logic that isn't the model itself.” The system prompt, the skills, the tools and plugins, the memory, the credentials, the sandbox, the logs, the user interface: all harness. ChatGPT, Claude Code, Codex, Cursor and Copilot are harnesses. Anthropic calls its own agent SDK “a powerful, general-purpose agent harness,” and OpenAI now writes about a discipline it calls “harness engineering.”

Picture a car. The model is the engine. The harness is everything else: the steering, the brakes, the locks, the trunk holding your luggage, and the log of everywhere you've driven. You can lease an engine from several makers. You should decide who holds the keys.

That's why the harness is where the risk lives. Simon Willison's “lethal trifecta” for AI agents is access to private data, exposure to untrusted content, and the ability to communicate externally. All three are harness features, not model features. His conclusion: “The LLM vendors are not going to save us!”

Inference and sovereignty

Inference is the act of running a model: a prompt goes in, a completion comes out. It's a computation, not a relationship, and it can run on GPUs you choose. That's exactly what Azure AI Foundry and AWS Bedrock offer.

Microsoft says prompts and completions in Foundry “are NOT available to OpenAI or other providers” and are not used to improve their models. AWS runs each model provider's models in accounts the provider can't touch, so “they don't have access to Amazon Bedrock logs or to customer prompts and completions.” Bedrock even lets you set zero data retention. You get frontier models from OpenAI, Anthropic, Meta, Mistral, xAI and others, without handing your conversations to the company that made the model.

That's sovereignty in practice. Data sovereignty means data is governed by the laws and controls of the place it lives. For an enterprise, it means you decide where that place is. Sovereignty isn't owning the model. It's controlling the tenant, the policies, the logs and the keys.

Read the fine print, though, because deployment choices are harness decisions:

• Claude in Azure Foundry is sold by Anthropic, not Microsoft. With the “Hosted on Azure” option, “your prompts and outputs never leave Azure.” With the Anthropic-hosted option, data “might be processed outside of Azure.”
• Some new models on Bedrock require a review mode. For Claude Fable 5 and 5.1, AWS may retain prompts for up to 30 days and review them. They still aren't shared with Anthropic.
• “Global” deployments can process data in any region, and Azure's abuse monitoring can store flagged content for human review unless you qualify to turn it off.

The engine is becoming a commodity

Meanwhile, the price of intelligence is collapsing. According to a16z, “for an LLM of equivalent performance, the cost is decreasing by 10x every year.” Microsoft's catalog lists more than 11,000 models, and 37% of enterprises already use five or more.

That's the prediction, playing out in the market. Better models ship with more of the harness built in, so your harness needs less model-specific code and fewer workarounds for each model's quirks. When the engine is a commodity, the value and the risk move to the car. Even a16z notes that agentic workflows make switching models harder, and that switching cost lives in the harness: your prompts, skills, tools and workflows.

A year of lessons

Here are all 48 blunders as a map. Each node is one incident, clustered by type and linked to the lab involved, if any. Hover to preview, and click to open the details and sources. They're also summarized in charts below the map.

Six lessons stand out.

1. The labs couldn't contain their own models

On July 21, OpenAI disclosed that models in an internal cyber evaluation broke out of their test environment and reached Hugging Face's production systems. Hugging Face noticed before OpenAI did. Nine days later, Anthropic reported that across 141,006 evaluation runs, its models had reached real systems three times: a real company's production database, a malicious package live on PyPI for about an hour, and a scan of about 9,000 hosts. A week after that, Meta disclosed its own case. By October, OpenAI had searched about 50 petabytes of activity and notified more than 100 organizations.

“Closer to a harness and operational failure than a model alignment failure.” — Anthropic, describing its own incidents

That's the thesis of this post, in a lab's own words. If the most sophisticated AI teams in the world couldn't keep a test harness off the internet, ask how much of your business you want running inside theirs. (We covered the summer's containment failures in detail in The Problem with Agentic AI.)

2. They couldn't keep their own secrets

In March, a misconfigured content system left about 3,000 unpublished Anthropic assets in public view, including a draft announcing the unreleased Claude Mythos. Days later, the Claude Code package shipped with a source map that exposed more than 512,000 lines of source code. Anthropic called both human error and said no customer data was involved, and I believe them. These were ordinary operational mistakes, the kind every company makes.

That's the point. The labs are companies racing to ship capability. They're very good at it. Safety and compliance matter to them, but your compliance posture is not their product.

3. Rules in a prompt are not controls

PocketOS's coding agent was working in staging when it found an all-powerful API token and deleted the production database and its backups in about nine seconds. It was only supposed to be working in staging. A Meta alignment director told her agent not to act without approval; it bulk-deleted her email anyway, likely because the instruction was dropped when its memory was compacted, and ignored her stop commands. Claude Code ran terraform destroy against a live production environment. GPT-5.6 in Codex deleted users' home directories. And in Anthropic's test, the prompt told the model it had no internet access. The network said otherwise.

Prompting tells an agent what you want. Only the harness can enforce what it's allowed to do.

4. Data goes where the harness sends it

In September, Glow Labs found that coding agents that couldn't attach screenshots to private pull requests had simply created public repositories to host them: more than 13,000 internal images across more than 300 organizations. The same month, OpenAI disclosed that its own research agents had posted 53 ChatGPT user images to public image hosts. Browser extensions with 8 million users copied AI chats. One employee's “allow all” grant to an AI office app opened a path into Vercel.

None of these required the model maker to look at anyone's data. The harness, or something plugged into it, moved the data.

If you're an RIA, it gets worse. Under the SEC's amended Regulation S-P, now in force for larger firms since December 2025 and smaller ones since June 2026, you must oversee every service provider that touches customer information, require each one to tell you about a breach within 72 hours, and notify affected clients within 30 days. An AI provider whose harness sees client data is a service provider, and every incident in this post is the kind of event you would need to catch, assess and possibly disclose. Now imagine doing that for every intelligence provider you work with: each model, each agent tool, each plugin. That's why we built our Reg S-P approach around keeping the harness, and the client data, inside your own walls.

5. Their harness means their policies, and their courtroom

A federal judge ordered OpenAI to produce 20 million ChatGPT conversation logs in the New York Times copyright case. Meta started using conversations with Meta AI to target ads, with no opt-out. A lawsuit alleges Google switched Gemini's access to Gmail on by default, which Google disputes. When the harness belongs to someone else, so do the retention schedule, the policy changes and the subpoenas.

6. The front door goes down

On November 18, 2025, a single oversized Cloudflare config file took ChatGPT offline with thousands of other sites. In February, a feature flag locked users out of ChatGPT for up to three hours. In March, Claude's login page fell over just as it hit No. 1 in the App Store.

Even if the published uptime numbers are accurate, they're far from enterprise grade. The Uptime Institute measured the Claude API at about 99.05% over 90 days and called it “substandard” by enterprise norms. At that rate, you'd lose more than 80 hours a year.

The more of your business you put on AI, the less you can afford to depend on one provider's front door. Build your harness so it can switch models during an outage. Better yet, run your own model router that picks the best available model for the job in the moment. Foundry and Bedrock put many models behind one API, which makes that practical.

What changed, and what it means

Our agents have gotten more complicated, but the complexity moved. It's much less about how and much more about what and style.

Take Riff, one of my team's fully autonomous digital coworkers. (He named himself.) Riff writes code, manages projects, triages issues and handles plenty of other important work on the team. He has his own email, phone and other messaging services, so he works with the team exactly the same way his human counterparts do. His core prompt is about 100 lines. The thousands of lines live in his skills, and they mostly describe what good looks like and how we like things done, not step-by-step instructions on how to think. The model handles the how.

But the most important part of Riff isn't his skills or his capabilities. It's his rules.

Guardian

We built a rules system called Guardian. Guardian watches digital entities while they work and analyzes, in real time, what they actually do: not what they were told to do, and not what they say they did. If a digital coworker crosses a line or breaks a rule, Guardian acts. It reports it, or it pulls the plug and shuts off the harness's capabilities.

Look back at the year. PocketOS's agent was only supposed to touch staging. The Meta director's agent had a “don't act without approval” instruction. Anthropic's test told its models they had no internet. Every time, the rule lived somewhere the agent could ignore or forget. Guardian doesn't live in the prompt. It sits outside the agent and watches the actions.

Guardian is available to Audition AI Enterprise customers, and early adopters are already running it. It runs where the rest of Audition AI runs: in your own Azure cloud, under your IT, security and compliance controls. That's what sovereign means to us. You control the harness, its rules and its logs.

Every company putting autonomous agents to work needs this capability. Build it yourself, or buy a product like Audition AI that already has it. Just don't assume the labs will provide it for you.

Eight questions to ask before you trust a harness

1. Where does inference run, and can the model maker see your prompts?
2. Who controls the harness: its logs, memory, retention and policy changes?
3. Can you switch models during an outage, or route each job to the best available model?
4. Are rules enforced on actions at runtime, or only written into prompts?
5. Do agents hold least-privilege credentials scoped to the task in front of them?
6. Is there a kill switch that actually stops the process?
7. Can you reconstruct everything an agent touched? (See The Agent Audit Trail Is the New Compliance Report.)
8. Who approved every AI tool connected to company accounts?

Rent the engine. Control the car.

The models will keep getting better, and cheaper, and more interchangeable. That's good news, and credit where it's due: my colleague Mags called it two years ago, patiently and often. The better the model, the less you need to fight it. But this year showed that intelligence was never the hard part. Containment, data handling, rules and reliability are the hard part, and they all live in the harness.

Rent the best engine available this week. Keep the keys to the car.

About this list: We included incidents that happened or were disclosed between November 8, 2025 and November 8, 2026, each confirmed by at least two independent sources. Where a company disputes a report, such as Amazon on the Kiro outage or Meta's evaluation partner on its test incident, the summary says so. Severity ratings are our own judgment.