AI Evaluation Initiative Could Boost Commercial Adoption

Anthropic’s new funding program for advanced artificial intelligence (AI) evaluations could accelerate the adoption of AI across various commercial sectors, industry experts say.  The AI company announced Tuesday it will fund third-party organizations to develop new methods for assessing AI capabilities and risks, addressing a critical gap in the rapidly evolving field. The initiative seeks to create more robust benchmarks for complex AI applications, potentially unlocking billions in commercial value. As businesses look to deploy AI solutions, the lack of comprehensive evaluation tools has been a barrier to widespread adoption. “We’re seeking evaluations that help us measure the AI Safety Levels (ASLs) defined in our Responsible Scaling Policy,” Anthropic stated in its announcement. These levels determine safety and security requirements for models with specific capabilities.

Key focus areas include assessments of AI models’ potential cybersecurity capabilities, such as vulnerability discovery and exploit development. The company also seeks “evaluations that assess two critical capabilities: a) the potential for models to significantly enhance the abilities of non-experts or experts in creating CBRN [chemical, biological, radiological and nuclear] threats, and b) the capacity to design novel, more harmful CBRN threats.”  Improved benchmarks could address critical challenges in AI adoption for businesses. “The main problems of adapting generative AI at the moment are cost, hallucinations and safety,” Ilia Badeev, head of data science at Trevolution Group, told PYMNTS. 

Read the complete article here