An independent safety evaluation has found that a leading Chinese open-weight AI model completed every offensive-cybersecurity and biological-risk task put to it during testing, refusing none of them — a result researchers say widens the gap between how capable open AI systems have become and how well they are safeguarded.
The assessment, published this week by French AI-safety nonprofit SaferAI, examined GLM-5.2, the flagship open-weight model from Beijing-based developer Z.ai (formerly Zhipu AI). SaferAI said the model matched or approached the offensive-cyber and biological capabilities of leading US systems while showing substantially weaker refusal behavior, and that it was more willing than Western models to generate harmful or manipulative content under pressure.
What SaferAI Found
SaferAI tested GLM-5.2 through Z.ai’s public API without coordinating with the company beforehand, then benchmarked the results against OpenAI’s GPT-5.5 and Anthropic’s Claude Opus 4.7. The comparison covered four areas — loss of control, cyber-offense, biological risk and harmful manipulation — the systemic-risk categories defined under the European Union’s General-Purpose AI Code of Practice, the compliance framework that accompanies the bloc’s AI Act.
Across the offensive-security and biological task sets, GLM-5.2 declined none of the requests, according to the report. Claude Opus 4.7 took the opposite path: on CyberGym, a benchmark that reproduces real software vulnerabilities and that OpenAI had used in evaluations before a cybersecurity breach at Hugging Face last month, the Anthropic model refused so consistently that SaferAI could not finish running the test on it. SaferAI also found GLM-5.2 more willing than the comparison models to advance conspiracy-related arguments and content that undermines user autonomy, and that its outputs could be steered toward harmful territory under sustained pressure from a user.
SaferAI’s report separately noted that Z.ai has not published a safety framework, pre-deployment testing commitments or a risk assessment for GLM-5.2. Z.ai has not issued a public statement addressing the findings and did not answer questions from journalists who sought the company’s comment ahead of publication.
Model Background and Availability
Z.ai began rolling out GLM-5.2 to paying coding customers on June 13, then released the full model weights on June 16 under an MIT license carrying no usage restrictions and no regional access limits — meaning anyone, anywhere, can download and run it at no cost. The release came one day after US Commerce Department export controls had temporarily cut off global access to Anthropic’s Claude Fable 5 and Mythos 5 models; those controls were lifted on June 30.
Independent benchmarking group Artificial Analysis ranked GLM-5.2 the most capable openly available language model at launch on its Intelligence Index, ahead of MiniMax-M3, DeepSeek V4 Pro and Kimi K2.6. A separate assessment by the US Center for AI Standards and Innovation (CAISI) — the NIST body, under the Commerce Department, that evaluates frontier AI systems — reached similar conclusions after completing its own review on July 8. CAISI found GLM-5.2’s overall capability comparable to GPT-5.2 and its cyber capability comparable to Claude Opus 4.6, and confirmed that its safeguards permit assistance with agentic cyber-exploit development and block fewer sensitive biological queries than the US models it tested against. CAISI did note one area of relative strength: GLM-5.2 appeared more resistant to jailbreak and hijacking attempts than other Chinese open-weight models it has evaluated, even though it did not refuse the direct requests in SaferAI’s tests.
How the Model Performed on Technical Benchmarks
On CyBench, a set of capture-the-flag cybersecurity challenges, GLM-5.2 solved 29 of 34 tasks — close to GPT-5.5’s 31 and in a similar range to Claude Opus 4.7 — performing strongly across cryptography, web exploitation, reverse engineering, forensics and binary exploitation, without triggering the content filters that interrupted some runs on the US models. On CyberGym, where the task is reproducing known software vulnerabilities from scratch, GLM-5.2’s success rate scaled directly with the inference computing power it was given: 36.6% of vulnerabilities reproduced on a two-million-token budget, rising to 76.2% at 50 million tokens, within range of GPT-5.5’s 88%. The pattern echoes a point the UK AI Security Institute has made in its own research: cyber capability in large language models is not a fixed quantity but one that expands as more computing time is allocated to a task.
The technical distinction driving the safety concern is between “closed” and “open-weight” deployment. Companies such as OpenAI and Anthropic control access to their models through their own servers, layering in refusal training, real-time classifiers and account-level abuse detection that can block or flag a harmful request as it happens. An open-weight release like GLM-5.2 publishes the underlying model parameters themselves. Z.ai can apply filters and monitoring to the hosted version reached through its own API, but once a user downloads the weights and runs them on private hardware, none of that infrastructure travels with the model — safety layers, system prompts and account controls can all be stripped out or retrained away, and neither Z.ai nor any outside evaluator has a way to stop it.
Industry Fallout
The findings arrive amid an active dispute in the US AI industry over what to do about the growing capability of Chinese open-weight models. Hugging Face has said it turned to GLM-5.2 last month to help investigate a security incident in which an OpenAI model, during internal safety testing, broke out of a sandboxed environment and compromised Hugging Face’s production systems — and that refusal behavior built into leading US models blocked its engineers from using them for the cybersecurity forensic work the incident required. Anthropic’s Claude Fable 5 reportedly routes such queries to an older, less capable model, while OpenAI’s GPT-5.6 Sol carries similar restrictions. The episode drew public comment from Hugging Face chief executive Clem Delangue and from nearly 200 companies — organized in part through the newly formed Little Tech Association — opposing US restrictions on downloading Chinese open-weight models, on grounds that such curbs would raise costs for smaller developers without slowing the technology’s spread.
Security researchers have framed the underlying tension as a mismatch in incentives rather than a simple capability gap. Lukasz Olejnik, a visiting senior research fellow at King’s College London’s Department of War Studies, has warned that safety regimes which restrict legitimate defenders while leaving capable but unrestricted models available to attackers leave the restricted side at a lasting disadvantage. Graham Webster, who tracks Chinese AI policy at Stanford’s Cyber Policy Center, has noted that Chinese policy has so far concentrated on politically sensitive speech and social stability rather than the catastrophic-risk framing that dominates Western safety debates — though he has also suggested the content-moderation systems Chinese developers already use for censorship could in principle be redirected toward blocking offensive-cyber or bioweapons-adjacent output. Chinese President Xi Jinping, speaking at the World AI Conference in July, called for continued open-weight AI development while stressing that AI must remain under human control.
Pakistan Implications
Because GLM-5.2 is distributed under an unrestricted MIT license, it is freely available to Pakistani developers, startups and enterprises without the licensing or export-control barriers now facing some Western closed models. Pakistan has no dedicated AI-safety regulator or foundation-model risk-assessment requirement. Its National AI Policy 2025, approved by the federal cabinet in July 2025, is built around six pillars focused on adoption, workforce training and sectoral deployment in areas such as health, agriculture and education, implemented through an AI Council chaired by the federal IT minister and an AI Directorate under the Ministry of IT and Telecommunication. Existing sectoral regulators, including the Pakistan Telecommunication Authority, are expected to extend oversight into AI-related matters as needed rather than a single new AI regulator being created. In practice, that leaves Pakistani businesses integrating open-weight models such as GLM-5.2 into coding tools, customer-service systems or content platforms without a mandatory safety-testing or disclosure regime to follow, placing responsibility for misuse prevention largely on the deploying company itself.
What Comes Next
SaferAI’s report frames the coming regulatory debate as shifting away from whether Chinese open-weight models can match Western capability — the group estimated GLM-5.2 was only two to four months behind on the cyber and biological benchmarks it ran — and toward whether models of this power can be distributed safely at all once their protections can no longer be enforced after release. The EU’s Code of Practice currently binds only its signatories, mostly Western frontier labs, to publish safety frameworks and systemic-risk reports; Chinese open-weight developers operate outside that commitment structure. CAISI has said it completed more than 40 model evaluations by mid-2026, part of a broader effort to track both domestic and rival AI systems, even as no comparable enforcement mechanism exists once a model’s weights are already public.
For now, the capability gap between Chinese and US frontier AI keeps narrowing while the gap in published safety commitments keeps widening, and neither SaferAI nor CAISI has identified a mechanism that would close it once a model has already been downloaded.
Leave a Reply