Rahul Gupta

AI companies make safety calls based in large part on internal tests they develop themselves. It’s rare for there to be independent assessment of the quality of those tests. Earlier this year, @Amazon and @AVERIorg agreed to a pilot project in which AVERI assessed the quality of Amazon’s internal safety evaluations. Today we’re releasing the result of that independent review. As a result of this project, Amazon is changing how it tests future models. Our review of Amazon’s internal AI safety evaluations confirmed several strengths such as topical diversity, and we found the evaluations to be consistent with Amazon’s public statements on them. At the same time, we found areas for improvement: On attack sophistication, some prompts represented relatively straightforward attack vectors that would test safety filters rather than deeper model reasoning. On evaluation signal quality, we identified a concern that superficial cues, rather than safety reasoning, may drive some model refusals, so passing these evaluations might not indicate robust safety behavior in novel situations. On linguistic and stylistic diversity, some multilingual prompts were included, but prompts across risk categories were predominantly in English, and some prompt structures were reused across scenarios. On evaluation freshness, a small number of prompts contained temporal references that could become outdated. In response, Amazon reports four changes for future internal evaluations: Refreshing its test sets to remove redundant prompts and add more challenging adversarial scenarios; Introducing a tiered difficulty calibration framework; Expanding multilingual coverage across all risk categories; and Establishing a quarterly review schedule to identify staleness and incorporate emerging attack patterns. Why does this matter? A passing grade on an easy test doesn’t tell us much on its own. As the stakes of AI development and deployment grow, we think that this kind of independent evaluation of internal evaluations (“metaevaluation”) will grow in importance. We appreciate Amazon opening their internal evaluations up to outside review. We hope more companies take this step to help make their models safer. While no clear standards exist for metaevaluation yet, @AVERIorg is working to change that. We look forward to sharing more pilot results in the coming months and will continue our work developing standards with institutions like @aievalforum. Read AVERI’s full blog post here: averi.org/ourwork/averi-pilo… Read Amazon’s blog post on the collaboration here: amazon.science/blog/strength…