The Refusal Nobody Tests For

_{Guest contributor: Ben Pouladian is the CEO of BEP Holdings and publisher of BEP Research, where he covers AI, semiconductors, energy, and infrastructure.}

The safeguards "are intentionally broad right now and may flag safe and routine coding, cybersecurity, or biology work."

That's Anthropic's own notice, shown to me by Anthropic's own model. I published it with the screenshot on July 7.

The notice as it appeared on my screen. The second sentence is the one I should have read more carefully.

I've been putting together a biotech primer. Genomics, drug discovery, the applied-sciences turn I think is the next real inflection for AI. I just kept on getting shut down, which was kinda weird, and then getting locked out, and that feeling of being held hostage is kinda scary.

I finished the primer. The work got interrupted, and somebody I've never met decided when I could pick it up again.

On July 27 Anthropic published its position on open-weights models, under Dario Amodei's name. He asks the government for three things: keep powerful chips from China, crack down on industrial-scale distillation, and test every sufficiently capable model before release. All three are rules about which models get to exist, and none of them reaches the filter that locked me out mid-session. What I want isn't on his list or on the other side's: publish the thresholds, and publish how often they fire on people doing ordinary work. A market can't allocate around a policy nobody can read.

One of the three I'll leave alone. Chips are a national-security judgment I don't have standing on, and I'm long the company that sells them, which is a poor combination for a confident opinion.

I like Claude and I pay for it. Read this as a disagreement with a vendor I buy from.

When you send something to a hosted AI model, the model is rarely the only thing reading it. A layer above it scores your text first, and past some threshold you get a refusal, a hedge, or a quiet swap onto a more conservative model mid-session. That layer scores text and nothing else, as far as anyone outside can tell, so it can't see your job or which side of anything you're on. Where the threshold sits isn't published anywhere.

A daily cap is a different thing. You burn through your allowance and it says you're done for the day and you have to sign on tomorrow, which is kind of weird. Both were landing on me the same week. One is a company adjudicating what your work is. The other is a billing tier.

I saw people posting the same complaints on X, getting downgraded off Fable 5 while asking about mitochondria. I don't like being mid-task something and then getting shut down and not being able to work on it.

Two things get lost in the fight over his post. First: "Anthropic has never advocated for a ban on open-weights models." Second: "Open-weights models that don't have dangerous capabilities are a public good." He'd also exempt "less capable models, such as those from startups and academia" from testing, so on his own terms the mandate wouldn't touch most people reading this. My objection was never that it would land on a two-person team.

Where he splits from the July open-weights letter he otherwise agrees with: "But I don't agree with the letter's assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true."

I can't knock that down. If a filter can't tell a defender from an attacker, the open model you fall back on can't either.

What keeps me where I am is mostly just free markets. Whenever people force you to do certain things, it doesn't work out the right way, the way rent control reduces the supply of apartments. That's an argument I hold rather than a scar I picked up, and it's the wrong analogy for what Dario is asking, because a price ceiling isn't an export rule or a safety test. So none of this is an argument against testing. It's about the half that lands afterward.

Testing settles which models get to exist, and my problem started after that question was settled, when a filter decided a drug-discovery primer was close enough to something dangerous. His post has nothing on that half. No false positives, no accounting of what a shipped filter costs the people it stops by mistake. His own product concedes that possibility in writing.

The notice says something else, which I skipped past on July 7: "These measures let us bring you Mythos-level capabilities sooner, and we're working to refine them." That's the vendor telling me a blunt filter is the price of shipping fast, which argues for his third ask rather than against it. I'd still take the capability sooner with the settings published. But the sentence is on my own screenshot and it belongs to him.

The best case on record for my side is a security one, it's secondhand, and the man telling it put his name to the open-weights letter, ships one of the biggest open stacks going, and sells the chips both kinds run on. I called that the CUDA playbook in February, in The World Model Reckoning: "democratize the paradigm, own the infrastructure."

At the SF AI Summit on July 24, Ed Ludlow set it up as "Hugging Face trying to use an open model i- in its defense, where the guardrails were a factor." Huang: "And I think they, they used, uh, GLM 5.2 was my understanding. Mm. Um, they c- they couldn't get a proprietary model. They could not get a closed model, uh, to help them figure out what happened." He hedges the model, not the refusal, and Ludlow handed him the premise. I posted it flat on X on July 24 with none of those hedges, and having read the transcript I'd hedge it. It could still be wrong: those responders may have reached for an open model because it was faster, or because the data wasn't allowed to leave the building. What I do trust is the mechanism. Incident response means pasting real attack material into the model, which to a text-reading filter looks like a request to write one.

How the mechanism works, drawn generally. This is not a diagram of any real incident, and the open model you fall back on answers both senders.

Biology is where his case is strongest and mine is thinnest. He worries that biology has "a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case." A footnote carries the load-bearing part: what keeps us safe is "a negative correlation between intellectual capability and desire to commit catastrophic harm." The people who want to do it can't, the people who could don't want to, and AI might break that.

My gut answer was that if people wanted to create a bomb or some bioweapon, they would have done it by now. They don't need AI to tell them how to do that. Which doesn't touch his argument. He isn't claiming the information is missing; he thinks capability was the barrier and a model hands it over.

How much does a capable model really add on top of published literature and the physical work? Less than his case needs, in my read, because the hard part of that pipeline is tacit skill and the failures nobody writes up, not the reading. If I'm wrong about that, testing before release gets a lot more attractive, and this is where I'm weakest. The measuring has also started, which I'd rather say than have a reader catch: he cites a UK AI Security Institute study on the cyber version of the question, I haven't read it, and I won't pretend my instinct beats it. The same footnote carries the thing I can't argue away. Once open-weight models are released, it says, "these options are lost permanently: safeguards can be removed, and copies can be downloaded, redistributed, and run on private systems beyond monitoring." Ship-and-watch assumes you can un-ship. He'd hold the release until the measuring is done. I'd let it ship and watch, which is riskier and I know it, and it's most of why I want the tests published rather than mandated.

The version of his second ask you hear in public is theft: Chinese labs ran somebody's model, collected the answers, trained on the answers. That last step is distillation, how a cheap model picks up most of an expensive one's behavior without paying the training bill. His own version is narrower and better. Distillation is compute-efficient enough, he says, that it "allows China to build much better models than its number of chips would ordinarily enable, and thus partially evade chip bans."

That's the argument worth answering, and his own footnote answers it. Anthropic already bans accounts it catches distilling. Those accounts, he writes, "can often only be identified after substantial distillation has occurred," and the work "often involves creating large numbers of fake accounts that form a moving target." Then: "The practices of any individual company cannot entirely solve the problem." He reads that as the reason to hand the job to policy. I read it as a rule that arrives late by construction, aimed at accounts that were disposable the day they were made. That's a thin return for what it costs on the other side, where openness is the thing being traded away.

The likeliest way I'm wrong is that Dario agrees with me. He can say a shipped filter's false-positive rate is a product problem, and the remedy is the one I just named: let the customer decide. That concedes the whole observation and costs him none of his asks. And I'm the proof. I was the customer. I complained, I kept paying, I finished the primer. The exit my own argument prescribes was right there and I didn't take it, because this was annoying enough to write about and not annoying enough to leave over. Which is the size of the problem, and why I want a number published rather than a rule passed.

I don't have a named team. I've been asked twice for a pharma or climate team on the record that a refusal changed what they built. I don't have one. What I do know is that a lot of startups are choking under the API token pricing of Anthropic and OpenAI, and they need open-source models to survive and be profitable. That's a cost argument. My own best instance is still friction.

The closed vendors could ship this themselves. Private deployment already exists. I haven't found one that lets the customer set the refusal thresholds and read them, but that's a product decision, not a law, and any of them could ship it next quarter. It's the outcome I'd bet on first.

Three things would move me: a published uplift study showing a big gain over what's already in the literature; a vendor shipping that readable, configurable refusal policy; and a documented case of a defensive team blocked at scale.

Renting an open-weight model from somebody else's API is cheaper, and running one yourself is a different bill. I've blurred those two myself. The numbers are in my July 12 note: "Databricks' board landed GLM 5.2 statistically tied with Opus 4.8 on quality at $1.28 a task against $1.94." Huang, in the same interview and against his own book: "Frankly, I think closed models are cheaper," and then "So there's nothing cheap about doing that."

Three options, not two. The middle column is the cheap one, and only the right-hand column answers the last two rows.

If you build: don't self-host. It's expensive and most small teams shouldn't. If you're a small company, you shouldn't be dependent on one model because that's a bad business decision. You should have multiple models. That way, if something happens, like the situation I've been describing, you can quickly switch, and you won't have business interruption. What that buys is continuity, and continuity isn't control. Switch to an open model you rent from somebody else and the host decides what your work is. The only column that answers the last row is the one I told you not to buy, which is why what I'd ask a vendor for is a published threshold and not a rule out of Washington.

If you invest: the demand worth watching is for models a company runs inside its own walls, with somebody accountable for them. That's real whether or not I'm right, and one of the leading Western open stacks comes from the company selling the hardware under both options. I'm long NVDA and have been since 2016; full disclosure at the bottom.

For everyone else: the argument in public is about which models get to exist. What actually reaches you is a policy you never see, sitting above a model you rent, deciding what your doctor's software is allowed to help with. Nobody is testing for that. Just that feeling of someone else being in control was kinda scary.

Where to Go Deeper: The fuller, more technical version of these arguments lives at BEP Research.