Anthropic Researchers Discover Many-Shot Jailbreaking Vulnerability

Anthropic researchers have documented a new technique called 'many-shot jailbreaking' that exploits the large context windows of modern large language models (LLMs) to bypass ethical safeguards. By embedding a harmful question among dozens of benign but contextually similar queries, the model's in-context learning improves its responses to inappropriate content, potentially providing dangerous information like bomb-building instructions. The research, published in a report, highlights a fundamental trade-off between model capability and safety, as limiting context windows would degrade performance. Anthropic has shared the findings with the wider AI community, including competitors like OpenAI and Google DeepMind, to encourage collective mitigation of these vulnerabilities.