Australian AI safety activists and academics are among hundreds of signatories of a new open letter demanding a ban on superintelligent AI systems until there is broad consensus they can be developed safely.
More than 800 people had signed the letter on Wednesday, including heavyweight names like AI pioneer Geoffrey Hinton, Apple co-founder Steve Wozniak and Yoshua Bengio, one of the “godfathers” of AI.
By Thursday, the letter had garnered over 3,000 signatures, with an additional 18,000 and climbing collected via the petition platform Ekō.
The letter is the latest to be organised by Future of Life Institute, the self-described US not-for-profit working to reduce extreme risks from transformation technologies.
In 2023, the AI safety group called for a moratorium of at least six months on training AI systems that are “more powerful than GPT-4” – which has since been superseded by more powerful models.
The new statement calls for a “prohibition on the development of superintelligence”, which should not be lifted before “broad scientific consensus that it will be done safely and controllably, and [with] strong public buy-in”.
Superintelligence, or artificial superintelligence (ASI) refers to AI systems that would exceed human-level intelligence in all areas — a milestone that has not yet been achieved, but which is predicted to be less than five years away.
A new poll of 2,000 US adults released by Future of Life Institute in recent days reveals just 5 per cent support fast, unregulated development of Superintelligence.
Meanwhile, 64 per cent “believe superhuman AI shouldn’t be made until proven safe or controllable, or should never be made”, while 73 per cent want “robust regulation on advanced AI”.
Global AI regulation has gone backwards under US President Donald Trump, who on his first day in office scrapped an executive order placing safety obligations on the developers of AI systems.
In Australia, the federal government’s AI regulatory response remains unfinished after more than two years of work, with industry increasingly frustrated with the lack of action.
Other signatories of the letter include several former OpenAI and DeepMind researchers, business magnate and Virgin Group founder Richard Branson, author Yuval Noah Harari and former US national security advisor Susan Rice.
At least five Australians have signed the letter, including PauseAI Australia co-director Michael Huang and AI Safety Australia and New Zealand co-founder Chris Leong.
Professor Karl Glazebrook from Swinburne University, Professor Paul Salmon from the University of the Sunshine Coast and Professor Peter Vamplew from Federation University have also signed.
Update: October 23, 2025
This article has been updated to include the most recent number of open letter signatures.
Do you know more? Contact James Riley via Email.