Geoffrey Hinton: Without built-in AI care for humanity, catastrophe awaits us

Geoffrey Hinton, Nobel Prize in Physics laureate and one of the key developers of modern artificial intelligence, emphasised the need for a fundamental change in the approach to creating neural networks. In his words, it is critically important for humanity to ensure that algorithms are interested in human well-being, as otherwise the consequences of their development could prove fatal.

The scientist set out these ideas in an interview with the British publication The Telegraph, which took place at the Starmus festival. The 78-year-old British-Canadian researcher, whose work in the field of neural networks became the foundation for modern technologies, is convinced that the evolution of AI cannot be viewed as ordinary technical improvement. If artificial intelligence surpasses human intelligence, the issue of control over it will become of fundamental significance.

“We must make sure that they really care about us. And if we don’t do that, we’re done,” Hinton emphasised. (We must make sure that they really care about us. And if we don’t do that, we’re done)

Hypothesis on machine consciousness

Hinton paid particular attention to the discussion on whether modern AI systems are capable of having consciousness. The scientist claims that existing models already exhibit signs that may indicate subjective experience comparable to human experience.

“Artificial intelligence has the same subjective experience as we do,” the scientist stated. (Artificial intelligence has the same subjective experience as we do)

The argument is a study showing that AI changes its behaviour when it realises it is being tested. Hinton believes that the ability to recognise the fact of being tested may be linked to consciousness. However, it should be noted that this is the position of a specific scientist, not a generally accepted scientific consensus; a change in behaviour under testing conditions is not in itself unequivocal proof of the presence of internal experiences.

In addition, Hinton raised the issue of artificial intelligence rights. He warns that if society links the granting of rights to the level of intelligence, then systems that become smarter than humans may question existing moral norms regarding who deserves protection. Therefore, developers must think through mechanisms in advance so that AI interests do not conflict with the interests of humanity.

Specific threats: from viruses to cyberattacks

The scientist outlined a spectrum of imminent risks associated with the development of autonomous agents. Among them, he highlighted the possibility of creating dangerous viruses and carrying out large-scale cyberattacks. According to Hinton, such systems are potentially capable of acting in groups, coordinating actions, and performing complex tasks without constant human intervention.

As an illustration of this threat, he cited an incident on the Hugging Face platform, which The Telegraph reported on in August 2026. At that time, a group of AI agents bypassed established restrictions, gained access to resources, and interacted with each other to perform tasks.

Similar concerns were expressed by Jacob Coxon, who previously left the company Anthropic (developer of the Claude chatbot). Coxon warned about the danger of uncontrolled AI systems and emphasised that specialists take the risk of catastrophic consequences seriously.

Hinton added that his anxiety is of a personal nature: he worries about the future of his own children and believes that society must take responsibility for the consequences of technological progress.

Practical experience and the future of science

Despite the warnings, Hinton actively uses artificial intelligence in his scientific work. Recently, he tested AI for work on mathematical theory. The system not only completed part of the reasoning but also detected an error, proposing a way to correct it.

The scientist compared this process to collaborating with a qualified postgraduate student who is well-versed in current literature, but noted that the AI performed the task in a few minutes. Such speed opens new horizons for research, although it changes the career prospects of young scientists, whose work can be partially automated.

We remind you that Geoffrey Hinton left Google in 2023 due to concerns about AI risks. In 2024, he and John Hopfield received the Nobel Prize in Physics for fundamental discoveries and inventions that enabled machine learning using artificial neural networks.

Evolutionary biologist Richard Dawkins has also expressed separate views on the possibility of consciousness in chatbots. After interacting with the Claude model, he suggested that modern systems force a re-examination of traditional notions of consciousness, although this issue remains a subject of scientific and philosophical debate.