I spent the better part of my career inside big tech companies, and the thing nobody tells you about the genuinely bad decisions is that they almost never announce themselves as bad in the moment. They show up looking sensible. Someone’s got a problem, someone else has a clean-looking answer, the slide is tidy, and everyone in the room nods along because the alternative is being the one person who slows things down and makes it weird. That’s more or less how I imagine Meta’s Model Capability Initiative got the green light, and I’ve sat through enough of those rooms to feel reasonably confident I’m not far off.
If you missed the story, here’s the gist. Meta had software running on its US employees’ computers that was quietly logging keystrokes, mouse movements, clicks, even grabbing the odd screenshot of whatever happened to be on the screen at the time. The point of all of it was to feed that into their AI so the models could learn how an actual human fumbles around a computer, the dropdown menus, the keyboard shortcuts, all the little stuff we do without thinking. And for most staff there was no opting out. You were in it whether you wanted to be or not.
Then it leaked, like properly leaked.
And I keep getting stuck on the shape of this because it’s almost too perfect. The security notice that went around flagged that data sitting across something like 45,000 of Meta’s internal hive tables had been exposed. We’re not talking anonymous activity logs here either. Private conversations, full transcripts and performance data. The sort of thing that lands on the wrong desk and suddenly someone’s career is hanging by a thread, or at the very least their next performance review is going to be a deeply uncomfortable conversation. Meta rated it a SEV 2 on their own internal scale, which is serious. Not the building is on fire, but the smoke alarm is definitely going.
So a tool built to watch everybody ended up showing everybody’s private business to everybody else. I genuinely don’t know what to call that other than the project failing precisely because it worked.
The bit that actually gets under my skin though isn’t even the leak. It’s the response to it. Meta said the program was carefully designed with privacy safeguards, that there’s no indication anyone improperly accessed the data, and that they’re pausing it while they look into things. Read that one more time and watch what’s going on. Nobody’s arguing about whether the data was exposed. It most definitely was. It was just sitting there. The only question they’re willing to put on the table however is whether someone improperly looked at the thing that was lying wide open for the whole company to see. Which is a much easier question for them to answer, and it conveniently steps right past the one that actually matters, which is how on earth sensitive staff data ended up unprotected in the first place after people were told it would be tightly controlled.
What really annoys me is how the leak has sort of swallowed the other half of the story, which is that people inside Meta saw this coming. Back in May more than 1,600 of them signed a petition against the whole thing, and one of the specific worries they put down on paper was, you guessed it, the risk of breaches and data getting out. So this wasn’t some freak event nobody could have predicted. Staff predicted it, in writing weeks ahead of it actually happening. And it all comes at a time when morale at the company is apparently somewhere near the floor anyway, which Meta’s own CTO has more or less admitted, with the layoffs and the endless churn of AI projects grinding people down. Picture being painted you’ll be monitored so you can help train the very technology half the building quietly suspects is coming for their jobs. I have no idea how you stand up in front of a team and pitch that with a straight face.
Anyway. Why should any of this matter to someone reading this in South Africa.
Because it’s Meta. WhatsApp is basically how this entire country communicates. A massive chunk of small business in South Africa runs on Facebook and Instagram, and I don’t mean as a nice extra, I mean it’s the actual shop, the actual storefront, the place the orders come through. So when a company that holds that much of your daily life shows you how it genuinely thinks about data, how quickly “tightly controlled” can rot into “accessible to the whole company,” you’d be foolish not to take the hint. Because the instinct on display here, this grab everything now and deal with the fallout later mentality, doesn’t politely stay locked inside some internal staff programme. It’s the same instinct shaping how these platforms treat the rest of us, the ones who never signed up to be training data but probably are anyway.
And the deeper lesson is one I think most of us already know and just choose not to look at too hard. Data doesn’t stay where you put it. It moves. It collects in places nobody planned. It turns up in front of people who were never meant to see it. Doesn’t matter if you’re a Meta engineer in California or someone over here who tapped agree on four screens of terms because reading them properly on a cracked phone screen was never realistically going to happen.
The thing that stays with me, after all of it, isn’t that Meta wanted to mine its own staff for training data. That’s grim, sure, but it’s the specific flavour of grim this company has spent years training us to expect. It’s that the whole thing didn’t fall apart because of the petition, or the ethics of it, or anyone up top having second thoughts. It fell apart because the surveillance machine couldn’t even keep its own catch from spilling everywhere. It did exactly what it was built to do and that was the failure.
And look, if a company with Meta’s money and Meta’s engineering bench can’t pull this off without leaking sensitive data across 45,000 hive tables, then I’d think very hard before trusting any smaller outfit currently being sold the same shiny idea by some vendor with a confident slide deck and a promise about productivity gains.
I’ll be keeping half an eye on this one. “Paused while we investigate” has a long and frankly unconvincing track record of turning into “quietly switched back on once everyone stopped looking.” And nothing Meta has actually said tells me they’ve dropped the idea. Feels more like they’re just waiting for a quieter week.
