ChatGPT can output detailed tutorials on the production of biological weapons. The company only blocks the account that asks questions. Last summer, after OpenAI upgraded the performance of its large chat model, hundreds of users around the world asked it about the production and delivery methods of biological weapons and highly toxic agents.

ChatGPT can output detailed tutorials on how to make biological weapons. Companies only ban accounts that ask questions. After OpenAI upgraded the performance of its large chat model last summer, hundreds of users around the world asked it about the production and delivery methods of biological weapons and highly toxic agents. In the absence of rigid control at the federal level in the United States, the model will provide detailed answers, and its step-by-step instructions are simple enough for high school biology students to understand. According to people familiar with the matter, biosecurity and counter-terrorism experts reviewed the relevant conversation records of ChatGPT afterwards and determined that some of the answers were accurate enough to be fatal. OpenAI said that most of these questions revolve around the preparation of highly toxic agents. OpenAI banned the accounts of these users who inquired about drug production and biological weapons development, but did not report clues to law enforcement authorities. There is currently no federal law in the United States that forces AI companies to restrict such questions or report search records involving weapons manufacturing or deadly attack plans. The rapid improvement of AI technology capabilities, especially the ability to accurately design vicious attack plans at the biological and network levels, has made executives from major AI companies and senior White House officials deeply vigilant. The world's leading AI R&D institutions are caught in a dilemma: they must prevent malicious users from using technology to cause disasters, but they must not hinder the normal research and consulting needs of compliant scientific researchers. AI companies such as OpenAI and Anthropic stated that they are working closely with the White House to promote model release control and take account restrictions and bans on users with suspicious behavior. via cnBeta.COM - Chinese industry information station (author: Source: Global Market Report)