10 Comments
User's avatar
David Ulrich's avatar

Working on a post, based on the data one might conclude that US labs are against Open-weights because they can't or won't compete in the space.

Tempus Fugit's avatar

Excellent piece. Your point about people hiding behind classified information is especially salient. More often than not it comes from a good-faith inclination of career national security officials to try to plug every possible hole, address any possible threat. But it is also not infrequently used to discredit an opposing view, especially in any remotely partisan debate. In both instances, there is rarely a dominant consensus. I spent two decades in the intelligence community, and this happened all the time.

That perspective, even when offered in good faith, also relies on a narrow definition of national security. X bad actor uses Y tool, thus Y tool must be banned. What it ignores is the remaining scope of what constitutes national security, including economic competitiveness, innovation, and an overall bias toward growth and development.

Chris Hughes's avatar

This is correct, and stifling access to leading open weight models would harm defenders far more than attackers, especially given the challenges of refusals practitioners run into when trying to use the leading models for cyber-oriented defensive work.

Nathan Lambert's avatar

Appreciate you chiming in, I feel like the comment section here could get interesting in a few hours...

Aidan Parisian's avatar

The prosumer/hobbyist ecosystem around open weight models should have a positive affect on general AI safety. More people doing more testing in more ways always results in more findings. I’d feel safer in a world with 100+ models running on 100+ harnesses than 10 models running on 10 APIs owned by 10 people.

Jillian Berk's avatar

The structural feature here is the classified briefing. Evidence that cannot be shown cannot be contested, and it reliably argues in one direction. Notice who sits on both sides of that channel: the parties whose product becomes more valuable if the alternative is banned are also the parties briefing on the risk. That is not an accusation of bad faith. It is a description of a process with no adversarial step in it.

Nan Deng's avatar

The air-gapped point is the part I wish got more attention. A defender running a model on a private network can log every prompt and tool call and audit what it actually touched, which is a kind of control you don't really get from a hosted API either. I'd be curious whether anyone has tried measuring defensive uplift in those deployments the way the offensive evals measure attacker uplift.

Hans W. A. Hanley's avatar

I feel like the discourse often misses the need for LLMs’ defensive capabilities to be wide spread as well

Josh's avatar

Do you you believe the prominence and persistence of this argument in the discourse has any relation to power concentration?

praxis22's avatar

GLM 5.3 & 5.3-Flash are amazing for RP, great prose.