On Thursday, weeks after launching the yet-most powerful AI model, Google released a technical report showing the results of its internal safety assessment on the Gemini 2.5 Pro. However, the report is underestimated in detail, experts say, making it difficult to determine which risk the model poses.
Technical reports provide information that companies don’t always promote AI widely, and sometimes tedious. Overall, the AI community views these reports as good efforts to support independent research and safety assessments.
Google uses a different safety reporting approach to some of its AI rivals, releasing technical reports when considered a model graduated from the “experimental” stage. The company also does not include findings from all of the “dangerous ability” ratings in these articles. They reserve them for separate audits.
Several experts TechCrunch spoke to were still disappointed with the sparse of the Gemini 2.5 Pro report, but note that Google’s Frontier Safety Framework (FSF) has not mentioned it. Last year, Google introduced FSF in an effort to identify future AI features that could cause “severe harm.”
“This (report) is very sparse and contains minimal information and came out a few weeks after the model was already publicly available,” Peter Wildford, co-founder of AI’s Institute of Policy and Strategies, told TechCrunch. “It is impossible to see if Google contributes to public commitments and it is impossible to assess the safety and security of the model.”
Thomas Woodside, co-founder of Secure AI Project, said he is pleased that Google has released a report for the Gemini 2.5 Pro, but is not convinced of the company’s commitment to providing a timely supplementary safety rating. When Google last published the results of its dangerous capabilities test, Woodside noted that it had been announced in June 2024 for the model released in February of the same year.
Not too confident, Google hasn’t made its reporting available for Gemini 2.5 Flash, a smaller, more efficient model that it announced last week. A spokesperson told TechCrunch that Flash reports are “coming soon.”
“We hope this will promise to begin publishing more frequent updates from Google,” Woodside told TechCrunch. “These updates should include results from evaluations of models that have not yet been published, as those models can pose serious risks as well.”
While Google may have been one of the first AI labs to propose a standardized report on the model, it is not the only one that has been accused of underestimating transparency these days. Meta has released an equally revealing safety rating for the new Llama 4 open model, and Openai has chosen not to publish reports for the GPT-4.1 series.
Hanging above Google’s head ensures that the tech giant has done it to regulatory authorities and maintains a high standard of AI safety testing and reporting. Two years ago, Google told the US government it would publish a safety report for all “important” public AI models “within scope.” The company pledged to “provide public transparency” around AI products, and tracked its promises with a similar commitment to other countries.
Kevin Bankston, senior advisor to AI Governance at the Center for Democracy Technology, reports the sporadic and ambiguous trends as a “race to the bottom” on AI safety.
“Combined with reports that competing labs like Openai shaved safety test times before releases from months to days, this slight document on Google’s top AI models tells the troubling story of AI safety and transparency as companies bring their models to the market,” he told TechCrunch.
In a statement, Google said it was conducting safety testing of the model and a “adversity red team” before its release, although it is not detailed in the technical report.