Discussion about this post

User's avatar
Chris Hughes's avatar

Appreciate the shout out and willingness to revisit your perspective.

Another of my frustrations with the recent discourse around this topic is all of the research on these topics, both from frontier corps and external teams such as Epoch and Irregular is a heavy emphasis and focus on offensive tasks and capabilities. (e.g. GLM-5.3 can do x, y and z for offensive tasks).

We're seeing very little by comparison in terms of research efforts to demonstrate the defensive capabilities of these models, which I think is tied to various factors, such as:

- Frontier corps incentivized to demonize open weight models due to competitive factors

- The fact that offense is far more verifiable than defense, especially as each environment is unique with different architectures, compensating controls etc.

- The reality that it would highlight the level of refusals legitimate defensive users/use cases encounter with closed models compared to open.

Almost all the hype and attention is on offensive use cases, and I suspect that is also influencing training and hill climbing, which inevitably and painfully ironically makes each model subsequently better at wreaking havoc vs. mitigating it.

1 more comment...

No posts

Ready for more?