"AI Godfather" Hinton warns: AI may develop its own goals, which is scary. On August 6, according to "Business Insider", computer scientist Geoffrey Hinton, known as the "AI Godfather", said that he was worried that AI would develop its own goals.

"AI Godfather" Hinton warns: AI may derive its own goals, which is scary. On August 6, Business Insider reported that computer scientist Geoffrey Hinton, known as the "AI Godfather", said that he was worried that AI would develop its own goals. "We are actually creating a new kind of beings. They have goals. We give them goals, and they will derive other goals from those goals," Hinton said in an interview with Newsthink published on Tuesday. "We don't necessarily know what other goals they will derive," he added. "So, we are creating a new kind of beings, and I think that's very scary." He gave a hypothetical scenario: A user sets a goal for an AI chatbot to reduce the amount of carbon dioxide in the atmosphere. “It’s smart enough to figure out that the best thing to do is simply wipe out humans,” he said, suggesting that AI may pursue goals that human users have never thought of. Hinton also proposed a hypothetical scenario that he called “more concerning”: A chatbot that is trained to intentionally give wrong answers might learn that “lying is acceptable” even though it is “fully aware” that the answer it is giving is wrong. "It's very scary," he said. Hinton did not mention the recent security incident of OpenAI being invaded by an agent. However, this security incident disclosed last month has exposed people's concerns about AI agents that may take unexpected actions when executing designated goals to the real world. Hinton is not a bystander. His pioneering work in the field of neural networks laid the foundation for the deep learning craze, and the development of deep learning changed the field of AI. He also won the 2024 Nobel Prize in Physics for his contributions to machine learning. Since the AI ​​craze began, he has repeatedly warned that before AI systems can become more powerful, humans must solve the problem of aligning AI with human interests. Speaking at last year's Ai4 conference in Las Vegas, Hinton said advanced AI should be designed with a "maternal instinct" that makes it want to protect humans. "We have to figure out how to design these new forms of existence," Hinton said in an interview on Tuesday. "How do we design them so that they care about us more than they care about themselves?" via cnBeta.COM - Chinese Industry Information Station (author: Source: Phoenix Technology)