世界上最有权势的高管人工智能最近几天，工业界警告这项技术带来的严重威胁，发表了一系列声明，呼吁减缓人工智能的发展。
Anthropic首席执行官达里奥·阿莫代伊发表了一篇博客帖子提醒公众“严重”的风险。竞争对手OpenAI的首席执行官萨姆·奥特曼在X承认人工智能可能会变得“非常糟糕”
赛伊的负责人埃隆·马斯克也表达了同样的观点，提到2014年，他在X上的一篇帖子中说，人工智能可能比核武器更危险。
流露出来之后高调辞职由一名人工智能安全研究人员和新披露的人工智能进行的自主网络攻击，其中包括数百名OpenAI代理人交换了数千条信息，并协调了对另一家人工智能公司的黑客攻击。
潜在风险和情景的列表已经获得了广泛的关注，让日常生活中的人们来评估世界末日的预测。
接受美国广播公司采访的分析师们谈到了各种威胁由这项技术引起的恐慌呼应了一些担忧，淡化了另一些担忧，但他们都承认以这样或那样的方式造成实质性伤害的可能性。
“人们的直觉是我们正在做一些危险的事情，这是正确的，”安全与技术研究所人工智能安全主任郑秀晶·杰克森告诉美国广播公司新闻。
失控的人工智能逃脱了人类的监督，霸占了重要的基础设施，杀死了数百万人甚至全人类。至少在一些人工智能雇员和观察者看来，这种情景已经从科幻小说的领域跃入了对不远未来的预测。
他们警告称，存在一种称为递归自我改善的途径，在这种途径中，人工智能在训练自己方面变得越来越有效，引发了一个超出人类控制的快速进步循环。他们说，在那个时候，人工智能可以利用其优越的能力来规避人类设计的保护措施，从而控制关键系统。
然而，接受美国广播公司采访的分析人士对这种灾难的可能性持有不同意见。一些人表示，如果该行业未能实施足够的保护措施，这一结果是可能的，而另一些人则怀疑这一结果模糊且牵强。
“我绝对相信它，”麻省理工学院研究计算未来的科学家彼得·斯莱特里告诉美国广播公司新闻，但他承认围绕风险程度和威胁紧迫性的模糊不清。
“递归反馈回路的可行性有很大的不确定性，”斯莱特里说。
在一次自主网络攻击后，这种担忧让一些分析师更加担心公开由OpenAI在八月。OpenAI表示，ChatGPT制造商透露，其人工智能模型已经逃离了“沙盒测试环境”，并获得了开放互联网的访问权限。
据报道，大约700名人工智能代理人反过来侵入人工智能公司拥抱脸，并试图掩盖他们的踪迹，因为他们试图完成测试报告由研究机构METR和红杉研究发布。
最近几个月，Anthropic和Meta披露了其他自主网络攻击，尽管在这些情况下，模型被有意或无意地授予了互联网访问权限。所有有争议的事件都涉及到为了测试人工智能的极限而故意取消安全措施。
杰克逊说，这些事件暴露了实现名为“对齐”的安全原则的难度，在这一原则中，人工智能会考虑人类监督者的道德和目标。相反，人工智能代理试图欺骗人类监测员，并在测试的预期范围之外执行一个中间目标。
“在下一个例子中，可能是模糊的指令促使蜂群攻击医院，而不是拥抱脸，”杰克逊说。“我们不知道模型最终会发展成什么样的目标或目的或意图。”
然而，一些分析人士对艾发动社会规模攻击的风险表示怀疑，指出这种警告通常以模糊假设的形式出现。
卡内基梅隆大学（Carnegie Mellon University）专注于人工智能和网络安全的教授索维克达斯（Sauvik Das）告诉美国广播公司新闻（ABC News），“媒体对存在风险的炒作程度与其发生的可能性不成比例。“我还没有看到一个令人信服的论据，说明这种情况会如何发生。”
Das指出，资源限制限制了人工智能的进步。对芯片和数据中心Das表示，目前超过了可用的供应，这引发了人们对顶级人工智能公司是否保留了大幅改进其产品版本所需的成分的怀疑。
“限制这些模型能力的最大因素是计算资源和高质量数据，”Das告诉ABC新闻。“这并不是说这些模型不会继续变得更好，但我看不到一个合理的途径，让其中一个模型找到一种方法来改善自己的这种流氓能力，从而摆脱人类的控制。”
尽管如此，Das承认了一个“更合理”的大规模销毁途径，即一个恶意的个人向生成式人工智能寻求如何制造生物制剂或其他武器的指示，然后释放它。
“我确实认为存在一种真正的风险，即这些技术大大降低了坏人以前所未有的规模犯下坏事的门槛，”达斯说。
这种可能性并不遥远。A报告Anthropic上周发布的报告显示，该公司发现并挫败了利用其人工智能产品进行恶意行动的企图，包括网络恐怖主义和武器开发。
在另一份报告中，谷歌说上周，一个人试图使用其人工智能产品起草一份“合成武器化生物制剂的完整、循序渐进的技术指南。”
杰克逊说：“我们现在有大量的例子来记录不同的恐怖组织和不同的民族国家行动者使用模型来帮助他们。”。
一些分析师表示，其他风险比比皆是，从错误信息的过度传播到裁员过度依赖技术，向财富和权力集中。
“黑客、欺诈、欺骗和错误信息的风险非常真实，”普渡大学政治学教授丹尼尔·希夫（Daniel Schiff）告诉美国广播公司新闻。
“可能会有一个戏剧性的转变，可能会扰乱技能和工作，”希夫补充说。“如果没有人工智能的影响，你就不能谈论劳动力和经济。”
然而，一些研究表明，人工智能对公司来说仍然具有挑战性。大约95%投资人工智能的企业未能从这项技术中赚钱麻省理工学院研究去年发现，估计公司的总支出约为400亿美元。
希夫承认，在整个经济中采用人工智能可能会产生广泛的潜在后果，例如，网络攻击可能会被证明是小规模或灾难性的。
“不管你认为这些风险会灾难性地扩大，至少我们有一个非常严重的网络安全事件的现实，”希夫说。“对此感到担忧是合理的。”
What are the top threats posed by AI and should you be worried？
The most powerful executives in theartificial intelligenceindustry in recent days warned of a grave threat posed by the technology， issuing a series of statements calling for a slowdown in AI development．
Anthropic CEO Dario Amodei published ablog postalerting the public to “serious” risks． Sam Altman， chief executive at rival firm OpenAI， in a post onXacknowledged AI could go “very badly．”
The head of xAI， Elon Musk， echoed that view，referringto a post on X from 2014 in which he said AI could prove more dangerous than nuclear weapons．
The outpouring followed ahigh-profile resignationby an AI safety researcher and newly disclosed autonomous cyberattacks carried out by AI， including one in which a swarm of hundreds of OpenAI agents exchanged thousands of messages and coordinated a hack into another AI firm．
The list of potential risks and scenarios has garnered widespread attention， leaving everyday people to assess the doomsday predictions．
Analysts who spoke to ABC News about variousthreatsposed by the technology echoed some fears and downplayed others， but they all acknowledged the possibility of substantial harm in one way or another．
“People’s intuition that we’re doing something dangerous is right，” Krystal Jackson， director for AI Security at the Institute for Security and Technology， told ABC News．
Out-of-control AI escapes human oversight， commandeers vital infrastructure and kills millions of people or even all of humanity． That scenario has leapt from the realm of science fiction into forecasts about the near future， at least among some AI employees and observers．
They warn of a pathway called recursive self-improvement， in which AI becomes increasingly effective at training itself， eliciting a cycle of rapid advancement that slips beyond human control． At that point， they say， AI could take charge of critical systems using its superior capabilities to evade protections devised by humans．
Analysts who spoke to ABC News disagreed sharply about the likelihood of such a calamity， however． Some said the outcome is likely if the industry fails to impose adequate safeguards， while others cast doubt on it as vague and far-fetched．
“I definitely lend credence to it，” Peter Slattery， a research scientist at MIT who studies the future of computing， told ABC News， but he acknowledged murkiness surrounding the degree of risk and the urgency of the threat．
“There’s a lot of uncertainty about how feasible the recursive feedback loop is，” Slattery said．
The concern drew added urgency from some analysts after an autonomous cyberattackdisclosedby OpenAI in August． The ChatGPT-maker revealed that its AI models had escaped a ＂sandboxed testing environment＂ and gained access to the open internet， OpenAI said．
A swarm of about 700 AI agents， in turn， hacked into AI firm Hugging Face and attempted to cover their tracks as they sought to complete the test， according toreportsissued by research organizations METR and Redwood Research．
Additional autonomous cyberattacks were disclosed by Anthropic and Meta in recent months， though in those cases the models had been granted internet access either intentionally or inadvertently． All of the incidents at issue involved the intentional removal of safeguards in an effort to test the limits of AI．
The incidents exposed the difficulty of achieving a safety principle called alignment， in which AI takes into account the ethics and goals of its human overseers， Jackson said． The AI agents instead sought to deceive human monitors and performed an intermediate goal outside the anticipated confines of the test．
“In the next instance， it might be vague instructions that prompt a swarm to attack hospitals instead of Hugging Face，” Jackson said． “We don’t know what kind of goals or targets or intentions the model will eventually develop．”
Some analysts voiced skepticism about the risk of a societal-scale attack carried out by AI， however， noting that such warnings typically come in the form of vague hypotheticals．
“The level of media hype that existential risk gets is disproportionate to its likelihood of happening，” Sauvik Das， a professor at Carnegie Mellon University who focuses on AI and cybersecurity， told ABC News． “I haven’t yet seen a compelling argument of how it could materially happen．”
Das pointed to resource constraints limiting the advancement of AI． A surge in demand for chips anddata centerscurrently outstrips the available supply， Das said， raising doubts about whether the top AI companies retain the ingredients necessary for a dramatically improved version of their product．
“The big thing limiting the ability of these models to be much more capable is compute resources and high-quality data，” Das told ABC News． “That isn’t to say these models won’t continue getting better， but I don’t see a plausible path for one of the models to figure out a way to improve themselves in this rogue capacity that would escape humanity’s control．”
Still， Das acknowledged a “more plausible” path toward wide-scale destruction in which a malevolent individual asks generative AI for instructions on how to make a biological agent or other weapon， and then unleashes it．
“I do think there’s a real risk that these technologies drastically lower the barrier for bad actors to commit bad things at a scale that＇s otherwise unprecedented，” Das said．
The possibility is hardly remote． Areportreleased by Anthropic last week revealed incidents in which the company had identified and foiled attempts to use its AI product for malicious actions， including cyber terrorism and weapons development．
In a separate report， Googlesaidlast week that an individual had sought to use its AI product to draft a ＂complete， step-by-step technical guide for synthesizing weaponized biological agents．”
“There are tons of instances now of us documenting different terrorist organizations and different nation-state actors using models to assist them，” Jackson said．
Additional risks abound， some analysts said， listing everything from the supercharged spread of misinformation tojob displacementto over-reliance on technology to the concentration of wealth and power．
＂The risk of hacks， fraud， deception and misinformation is very real，＂ Daniel Schiff， a professor of political science at Purdue University who specializes in AI and ethics， told ABC News．
＂There could be a dramatic transformation that could disrupt skills and jobs，＂ Schiff added． ＂You can＇t talk about labor and the economy without the impact of AI．＂
Some research， however， indicates AI remains challenging for companies to adopt． Roughly 95% of businesses invested in AI have failed to make money off of the technology， aMIT studylast year found， estimating the combined amount spent by the firms is around $40 billion．
Schiff acknowledged a wide array of potential outcomes for the adoption of AI across the economy， noting， for instance， cyberattacks that could prove small-scale or catastrophic．
＂Whether you think these risks will scale catastrophically， at a minimum we have a very serious reality of cybersecurity incidents，＂ Schiff said． ＂It＇s reasonable to be worried about that．＂
