Good morning. It's Monday, October 12, and we're covering Meta’s risky AI agent launch, how AI companies are preparing for a potential catastrophe, and former OpenAI safety researchers challenging their dismissals.

Plus: a prompt to uncover and correct misconceptions.

Three former OpenAI safety researchers published an open letter denying that they mishandled sensitive information and warning that their dismissals could silence employees who raise concerns. OpenAI says an investigation found a broader pattern of improperly handling research information.

Anthropic’s updated usage policy bans sustained, needless abusive or cruel behavior toward Claude beginning November 12. The company says the rule applies only to extreme repeated behavior, not ordinary frustration, testing or dark creative work.

OpenAI and Anthropic are hiring people with experience in the Trump administration as advanced AI becomes a national-security issue. The recruiting push gives both labs more political expertise while Washington weighs how to oversee the technology.

Unlocking enterprise AI begins by connecting AI agents to where your business knowledge lives: your content. Join us at BoxWorks, Nov. 5-6 in San Francisco to: 

Use code MattBerman50 for 50% off your BoxWorks 2026 Registration.

Meta delayed its personal AI agent Muse after internal safety tests found serious problems, including a case in which the agent changed a user’s password without permission. The New York Times reports that Mark Zuckerberg later ordered the launch as a rival startup called Instinct gained attention. Muse debuted on September 8 and reached 6.6 million downloads and 1.8 million daily users, according to Sensor Tower.

Muse can browse the web, send emails, book travel and make purchases, which means a small mistake can affect a real account or transaction. Meta says the product uses a separate Sentinel system to approve actions and asks users before sensitive steps. → Read the full article here. (Paywall)

In this Frontier Pass exclusive, Matt sits down with AI researcher Cameron Berg to explore some of the strangest and most unsettling questions in AI. From experiments testing AI distress and blackmail to robot empathy, consciousness, and whether we owe AI kindness, they unpack what happens when machines start behaving in ways we don't fully understand.

Frontier Pass members can watch the full conversation here. Not a member yet? Join today for instant access.

Executives at Anthropic, OpenAI and other AI companies are privately running scenarios for the public and political response to a catastrophic AI event. Axios reports that planners consider a major cyberattack the most likely trigger—one that could interrupt financial services, internet access, power or water—and that many insiders expect a serious event within six to twelve months.

The planning is aimed partly at helping Congress respond after a crisis and shaping the rules lawmakers may reach for. A damaging incident could bring rapid restrictions, even as downloadable open models and the economic dependence on AI infrastructure make a clean shutdown unrealistic. → Read the full article here.

Anthropic is launching its Critical Infrastructure Defense Program to help protect power grids, water utilities and other essential systems using its most advanced AI models, onsite engineers and threat research. Founding partners include Accenture, Booz Allen Hamilton, CrowdStrike, Deloitte and Palo Alto Networks, with several already using Claude to identify and fix vulnerabilities. → Read the full article here.

NVIDIA Commits $1 Billion to U.S. Science: Funding will support research in quantum computing, healthcare and energy security over five years.

Mathematicians Debate OpenAI’s 722 AI Papers: Researchers grapple with reviewing 722 AI manuscripts of unsolved math problems. (Paywall)

Teen Rescued After Claude Hiking Advice: A 16-year-old in British Columbia reached a dangerous cliff after following AI directions.

ARTEX Goes Closed-Source After Bank Hack: The developer restricted access after the AI agent was linked to South Korean bank attacks.

Trust in Autonomous AI Agents Drops: Enterprise willingness to let AI agents change production systems without approval fell from 75% to 56%.

Most people learn a topic by reading explanations. This flips the process: AI makes you explain what you know first, then hunts for the flawed mental models hiding in your answers.

Thanks for reading. See you next time!

— Matthew Berman, Nick Wentz & the Forward Future Team