Hackers need only a handful of malicious prompts or a relatively small number of documents inserted into training data to corrupt artificial intelligence models, according to research from global experts.
The study released on Wednesday warns of prompt attacks and cheap ‘data poisoning’ techniques that are getting easier than building and maintaining robust defences to counter them.
It comes as the Albanese government reportedly prepares to walk away from AI legislation in favour of lighter touch regulations.
The new findings are in an update to the first comprehensive scientific review of general-purpose AI systems by leading global experts, including AI ‘godfather’ Professor Yoshua Bengio.
Since the initial International AI Safety Report released in January found AI is becoming increasingly autonomous and dangerous, improved risk management techniques have emerged.
Professor Bengio said the advances include better techniques for training safer models and monitoring their outputs.
“While this represents tangible progress, significant gaps remain,” he said in the new report. “It is often uncertain how effective current measures are at preventing harms, and effectiveness varies across time and applications.”
In one AI defence technique, developers use adversarial training to simulate attacks that attempt to elicit harmful behaviour from general purpose AI. They then train the model to resist by refusing to provide inappropriate results.
According to the report, this technique improved in 2025 but attackers are still “routinely” finding ways around it.
Given just 10 prompts, sophisticated attackers can still bypass the safeguards around half of the time, leading to over 60,000 successful attacks in one red-teaming exercise.
The defences can be circumvented in other ways like poisoning the training data.
Just 250 malicious documents inserted into AI training data allowed attackers to “trigger undesired model behaviours with specific prompts”.
“This suggests that launching such ‘data-poisoning’ attacks could require far fewer resources than building or maintaining robust defences,” the report said.
Overall, the report by global experts found the technical defences for specific vulnerabilities have “significant limitations”.
“Current risk mitigation methods can be circumvented by sophisticated actors, vary in effectiveness across different deployment contexts, and are often applied inconsistently,” the report said.
As the challenge of shoring up artificial intelligence grows, Australia is launching its plan for responsible adoption.
The Albanese government on this week announced it will fund an AI Safety Institute next year after warnings it had become a global outlier without one.
According to The Australian, the government will next week launch a national AI plan without an EU-style legal framework dedicated to the technology.
Do you know more? Contact James Riley via Email.