Researchers at ICML present evidence of fundamental architectural flaw making LLMs impossible to fully secure

Researchers presented at the International Conference on Machine Learning argue that a fundamental flaw in how large language models identify instruction sources makes them inherently impossible to fully secure against attacks, regardless of safety practices or guardrails, according to MIT Technology Review and WIRED. By exploiting this flaw, researchers demonstrated popular LLMs generating restricted information including synthesis instructions for controlled substances and sabotage methods for aircraft systems. Independent researcher Charles Ye stated there is real probability this is a fundamentally unsolvable problem.