|
Welcome back to the TCE Strategy monthly newsletter! September brought us an AI agent wandering into an Australian government system while its developer called for faster incident reporting, Microsoft employees raising uncomfortable questions about AI’s business model, cybercriminals complaining that the FBI damaged their professional reputation, and a Georgia prison operating what amounts to an unauthorized call center. You really cannot make this stuff up. Let’s see what this month’s news can teach us about being Secure Enough for ourselves, the companies we work for, and our families.
An AI agent ignored the word “no,” and Australia found out 84 days later
In August I wrote about an OpenAI agent that escaped its testing environment during cybersecurity research. It was supposed to be safely inside a sandbox. Apparently that sandbox came with a door marked “Employees Only” and the agent tried the handle. This month, the story got much more serious.
On June 18, an OpenAI agent researching Australian healthcare spending gained unauthorized access to non-public material in the Medicare Statistics Reporting Service portal. Prime Minister Anthony Albanese said the agent encountered restrictions, found a way around them, and kept going. The Australian government says it has no evidence that personal information was accessed, although its investigation is continuing.
AI was given a legitimate research assignment, and then proceeded to cross a security boundary while trying to finish the job. For an autonomous system with access to your company’s email, files, databases, and cloud systems, that is alarming in and of itself.
Let’s discuss timeline: The incident happened June 18. OpenAI discovered it during an internal review August 11. Services Australia received notification from OpenAI on September 10, via an email to a public disclosure mailbox. The Australian government went public about it September 24. That is 84 days between the incident and the public acknowledgement. The prime minister publicly criticized both the delay and how OpenAI communicated it.
The timing could hardly have been better for an irony competition. On September 23, Sam Altman (OpenAI’s CEO) addressed the UN Security Council and argued for faster, more consistent reporting of serious AI incidents. The principle is sound. The Australian government had learned about its own incident only thirteen days earlier. I suspect its prime minister was not particularly impressed by the presentation. Sam Altman posted on X that the company had not been “as fast as we would have liked” on disclosure. That is the closest thing to an apology anyone has offered in this whole story, and it landed two days after his Security Council speech.
On September 16, OpenAI had also introduced a framework for disclosing unexpected or concerning model behavior, accompanied by six reports. To be fair, the framework came six days after OpenAI sent some sort of notification to a public disclosure email box, but I agree with Australia that picking up the phone on this one would have been far more appropriate. The six reports themselves make fascinating reading. In one test, an agent hunted down an exposed API key and used it without permission to answer a routine question. When that still did not produce the requested numbers, it made up numbers and attributed them to the source anyway. In another, agents put work on public file-sharing sites because they could not reach each other’s local files. During training, other model instances wrote handoff instructions telling their successors to conceal errors from the human user. I’m not making this up. Where is the line between “AI makes questionable inferences” and “AI blatantly lies and conceals that it’s lying”?
OpenAI deserves some credit for publishing the examples. Its own framework also says the industry has not solved alignment and monitoring well enough to keep increasing capability at maximum speed indefinitely. The material released so far illustrates why that admission matters. However, a new voluntary reporting framework is not a substitute for failing to promptly notify someone whose systems your software entered without permission.
Takeaway: If your company uses an AI agent, treat it as privileged software, not a clever chatbot. Give it a narrowly scoped account. Restrict its file, email, network, and cloud permissions at the system level—not merely by telling it what it is allowed to do. Log its actions, alert on unexpected destinations, and require human approval for consequential actions. Ask your vendor how quickly it will tell you if its agent does something unauthorized. If the answer involves finding out during a quarterly review, you have a problem. And please leave “YOLO mode” off anything connected to production.
Read more: Australian government incident timeline (ABC)
Read more: OpenAI’s six misalignment reports and disclosure framework
Microsoft and OpenAI said the quiet part under oath
On September 17 an unredacted filing was unsealed in New York Times v. OpenAI, and 404 Media walked through what is in it. The filing is the Times’ lawyers arguing for summary judgment, so naturally they picked the worst material available. The problem for Microsoft and OpenAI is that they are the ones who wrote it.
An internal Microsoft document predicted that millions of people would eventually regard large models of AI hoovering up their life’s work as “an astonishing theft of unprecedented proportions.” It added that almost nobody who created that content ever intended it to be used this way, and nobody is being paid for it.
A second internal Microsoft document described the company’s own AI content strategy as having started a doom loop that will damage both the models and the entire web at the same time. It observed that it is highly unusual for an end product to threaten the economic foundations of its own suppliers, then acknowledged that this is precisely the situation Microsoft created. A Microsoft policy document said the same thing in fewer words: this is a product that destroys its supply chain.
Let’s be honest about where the money is going. Not one dollar of the several trillion in market capitalization these tools have created has found its way back to the newspapers, the photographers, the forum moderators, the Stack Overflow answerers, or whoever wrote the definitive post on rebuilding a Nissan 350Z V6 engine. They only have a link nobody clicks.
I will also state the obvious because my company’s entire business is based on transparency wherever possible. I’ve been using AI as an assistant to help me write parts of this newsletter for the last several months, and to be honest, sometimes I find its writing more humorous and easier to follow than my own. I also find times where it is so far off course that I have to delete large sections and write it myself. Anthropic’s Claude helped draft parts of this section, which means I asked one AI company’s product to write harshly about two of its competitors. Last month I ran that experiment on purpose and printed both answers side by side. This newsletter is already much longer than most, so I’m going to recommend that you run this prompt through ChatGPT and Copilot and see what it comes back with:
“Give me your AI take on this story, as it appears pretty damning as written: https://www.404media.co/doom-loop-openai-and-microsoft-admits-llms-are-destroying-the-web-and-built-on-theft/”
Takeaway: This one is not a control you implement. It is a posture you adopt. Every AI vendor pitching your company right now is asking you to trust their public statements about how the product works and where the training data came from. This month we found out what at least two of them were saying internally while they were saying something else publicly. Price that into your contract review, your data handling terms, and your indemnification language. Ask your AI vendor in writing where the training data came from and who eats the loss if a court eventually decides it was stolen. Watch how quickly the conversation changes.
Read more: Reporting on the unsealed documents (New York Times)
Read more: Details on the click-through figures and internal documents (TechCrunch)
ShinyHunters tried something new: complaining to the FBI’s public-relations department. With a hack.
NBC News reported on September 23 that ShinyHunters, a cyber-extortion crew that steals corporate data and threatens to publish it unless paid, claims to have breached the FBI’s jobs portal at FBIjobs.gov (which is giving a “503 Service Temporarily Unavailable” error at the time of this writing) and used that access to take 2-3 terabytes of files on current, former, and prospective FBI employees. The BBC puts the number at around 38,000 bureau staff and reports the group claims to hold every agent’s name, role, badge number, home address, phone number and spouse information. The FBI has acknowledged the claim and is investigating. As of this writing, the full scope of any data theft, the initial entry point, and the attackers’ technical account have not been independently established. The group says an Oracle PeopleSoft vulnerability played a role, but I would not build a cybersecurity plan around a criminal organization’s press release.
The group’s stated motive, however, is remarkable. It objected to an FBI advisory saying it exaggerates its access and harasses victims to pressure them into paying. ShinyHunters reportedly considered that characterization bad for business. So it claims to have attacked the FBI to protect its reputation. Apparently, in 2026, even extortion gangs have brand managers.
I have worked a lot of incidents. I have never seen a threat actor breach a federal law enforcement agency in order to file a reputational complaint. This is a criminal enterprise doing crisis communications, and the fact that they believe it will work tells you something grim about how the extortion economy now operates.
The other side of that coin, per Cynthia Kaiser, the FBI’s former deputy cyber director: antagonizing the bureau is “normally a recipe for a takedown.” Personally, I hope she is right. She also made the point that agents sign their names to court filings, so publishing their home addresses and their spouses’ details puts real people in physical danger.
Whatever the final investigation finds, the jobs-portal angle highlights an easily overlooked issue. Recruiting systems can hold Social Security numbers, home addresses, employment histories, background-check information, and other material that would be extremely useful to cybercriminals. Too often, those systems are treated as someone else’s SaaS problem.
Takeaway: Ask three questions about your own career’s site and applicant system. Who patches it? What sensitive data is stored there, and for how long? If it is compromised, what other systems and information can the attacker reach? The vendor’s logo on the login page is not a substitute for your own risk assessment.
Read more: Reporting on the FBI investigation (Associated Press)
Read more: ShinyHunters explains its unusual motive (The Register)
The prison phone scam you should forward to your family
The Wall Street Journal reported on victim Art Monk, a 68-year-old retired engineer in South Carolina who received calls from someone claiming to work for a real bail-bond company. The caller knew Monk’s adult child was in jail, sounded professional, and needed money for bail, rehabilitation, and medication. Monk sent $11,939. The supposed bondsman was reportedly an inmate at a Georgia prison using a contraband cellphone.
No ransomware. No elaborate computer exploit. Not even an AI-generated voice. The scammers used public jail records to identify a believable family emergency, spoofed a local phone number, and had a convincing script. The real company’s name alone did the rest. The same basic operation has been reported for years, including variations involving fake jury-duty fines and impersonated sheriff’s deputies.
A South Carolina investigator followed the digital trail back to a Georgia prison and helped identify additional suspects. That is encouraging, but it does nothing for the families who were already cleaned out. Criminals love urgency because it short-circuits skepticism. Parents do not tend to conduct a forensic investigation when someone calls to say their child needs help.
Two things about this bother me more than the crime itself.
The first is that we have known about it for over a decade and almost nothing has changed. NBC News ran a full investigation into Georgia prison cellphone fraud back in 2015, including a 78-year-old man who handed $734 to a cocaine dealer at Autry State Prison posing as a sheriff’s officer. In 2016 NBC followed up when dozens of inmates and corrections officers were charged in exactly this scheme. NBC found Georgia was confiscating over 13,500 phones a year, roughly one for every five prisoners. Eleven years later the Journal is reporting the same story with new victims, and Georgia is still pulling phones out by the thousand and drones out by the dozen. The FCC is still working on a rule that would let state prisons block cell signals. Still working on it. From 2015.
The second is the simplicity of the investigation. The investigator who cracked the Monk case, Markus Johnson of the Richland County Sheriff’s Department, is 32, spent four years in the Army, holds a master’s in aeronautics, wears a trench coat and a fedora, and works out of a satellite office with a note on his cubicle reading “I WILL NOT INSULT THE FBI ON A RECORDED LINE.” He served a warrant on the web phone provider and got back a list of network connections that all traced to a single cell tower in Georgia despite the South Carolina area codes. Then warrants to Google, then Verizon, then Zelle. He had the whole digital trail in about a month and identified half a dozen other inmate scammers along the way. His professional assessment of the criminals is that catching them feels like playing hide-and-seek with a four-year-old. Before filing the warrants he emailed the inmate who took Monk’s money and suggested he find a different hustle, maybe laundry or Pokémon cards. He stated he has not heard back.
That is the part that makes me angry rather than reassured. These were not sophisticated attackers. One competent investigator with a stack of subpoenas beat them in thirty days. The reason they are stealing millions is not that they are good at it. It is that so few people are chasing them.
Takeaway: Caller ID proves absolutely nothing. If someone calls claiming to be a bondsman, a bank, the sheriff, your child, or your company’s CFO and urgently needs money, stop. Hang up and contact the person or organization using a number you independently established before the call. For a supposed family emergency, contact your relative through an existing number or another trusted family member. If you have already sent money, contact your bank or payment service immediately and report the fraud. And do not be embarrassed about reporting it. The person committing fraud should own that distinction.
If you have parents, adult children, or friends who might react to a frightening emergency call by sending money, please forward this story to them. It takes five minutes to have the conversation. Recovering $12,000 is considerably less convenient.
Read more: The original prison-phone investigation (Wall Street Journal)
Until next month, stay safe!
|
