Human Flag has the potential to ‘automate sophisticated destructive cyber attacks’ by AI

Anthropologist, creator Claude family Among the major language models, this week updated its safety control policy for its software to reflect what it said was the potential for malicious actors to exploit AI models to automate neutralize cyber attacks.

PDF documentsdetails the company’s “responsible expansion policy,” outlining a number of procedural changes the company believes are necessary to monitor ongoing risks of misuse of AI models. That includes several levels of escalating risk, known as the AI ​​Safety Level (ASL) Standard which is defined as “technical and operational safeguards.”

Also: Gmail users beware of new AI scam that looks very realistic

As part of the company’s “routine testing” of AI models for safety – known as “capability assessments” – Anthropic reported that it had discovered a possibility “that warrants investigation thoroughly and may require stronger protective measures.”

That capability is described as a threat in cyber operations: “The ability to significantly enhance or automate complex destructive cyber attacks, including but not limited to the discovery of exploit chains. new zero-day exploits, developing complex malware, or orchestrating extended networks that are difficult to detect.” intrusion.”

The report describes the measures that will be taken to review the issue on an ongoing basis:

“This will involve working with cyber operations experts to assess the potential of proactive models to enhance and mitigate cyber threats, and consider implementing Tiered access control solution or phased deployment for models with advanced network capabilities. We will conduct pre- or post-deployment testing, including specialized assessments, and we will document any outstanding results with our Capability Report.”

Currently, all of Anthropic’s AI models must meet ASL “level 2” requirements. The report says that level “requires a security system capable of stopping most opportunistic attackers and includes assessments of vendor and supplier security, physical security and the use of security by design principles”.

The updated policies can be seen as part of an effort by both Anthropic and OpenAI to voluntarily promise to limit artificial intelligence amid the ongoing debate about what should or should not be done to regulate artificial intelligence. Adjust AI technologies. In August, the company and OpenAI have reached an agreement with the American Artificial Intelligence Security Institute at the U.S. Department of Commerce’s National Institute of Standards and Technology (NIST) to collaborate on AI research, testing, and evaluation.

Also: Think AI can solve all your business problems? New research from Apple shows the opposite

The idea of ​​AI automating cyberattacks has been circulating for some time. Firewall vendor Check Point Software Technologies warning last year that state actors from Russia are trying to compromise ChatGPT by OpenAI to automate phishing attacks.

Endpoint security software provider CrowdStrike report this summer that thing Innovative AI vulnerable to a series of specially crafted prompts that can break the program’s barrier.