Discussion about this post

User's avatar
Denmark Vesey's avatar

The whole earth and all of humanity should be the home for open-source and even further ALL software and other goods should be FREE and Open-source.

Humans are on the brink of extinction literally and billions of people suffer everyday.

Nation States are old tech. Time to move forward and work together all people to solve the problems we face now and the coming worsening effects of climate change.

Direct Particatory Democracy and a Degrowth framework is a big part of the solution.

Has anyone seen a movie that came out a few years ago, i think it was called Don't look up?

Expand full comment
T Stands For's avatar

The American open-source AI ecosystem should differentiate itself through a stronger commitment to safety (i.e. Antrhopic’s approach). If the world is to build atop America’s open-source models, labs should try to earn users’ and developers’ trust in the models’ underlying safety features.

As stated in the essay, this stands in sharp contrast to Chinese labs like DeepSeek, which have favored rapid deployment over safety considerations. Their performance on many safety benchmarks is not competitive with leading American models (attached below). The threat of malicious backdoors only heightens these existing safety concerns. Yet, these concerns extend beyond misalignment. DeepSeek put forward little effort to reinforce their external security posture, resulting in massive leaks. This disregard could undermine trust in their ability to defend key elements of the development pipeline. Overall, a lot of doubt is brewing.

In contrast, open-weight models warrant particularly rigorous safety standards. They should ultimately face a higher bar. In the wild, open-weight models can be deployed without moderation filters or classifier safeguards. At the same time, they must also be hardened against weight tampering (harmful fine-tuning attacks). While innovations like TAR and Tamper-Resistant Safeguards show promise, they lead to drop-offs in capabilities. This illustrates the central tension, balancing safeguards and model performance. If American open-source AI can better strike this balance while maintaining an enduring focus on safety, I suspect the market will reward them. Do not underestimate the power of trustworthiness.

https://www.enkryptai.com/blog/deepseek-r1-ai-model-11x-more-likely-to-generate-harmful-content-security-research-finds

https://blogs.cisco.com/security/evaluating-security-risk-in-deepseek-and-other-frontier-reasoning-models

Expand full comment
17 more comments...

No posts