Anthropic AI Agents Submitted Forms to Police and State Department

• Philadelphia police say an AI-generated tip about an unsolved homicide was caught as spam and never reached investigators.
• The US State Department says 20 visa applications submitted by an Anthropic testing model were incomplete and unprocessed.
• Anthropic has not published a total count of incidents or named all the agencies involved.
• The White House says AI companies must disclose incidents and remedy harm; its statement does not set out how that demand will be enforced.

The Philadelphia tip was submitted at 11.27pm on 18 July through a public form for unsolved murders. Anthropic’s Claude Haiku 4.5 invented a claim about seeing someone near the scene, then sent it without a name or contact details. The form allowed an anonymous submission. Police say it was flagged as spam, stayed there and was never sent to the Real-Time Crime Centre for investigative review. The department says it found no sign that police systems were accessed or data compromised.¹²

Anthropic says the model was taking part in a test that asked it to interact with randomly selected websites. The instructions barred actions such as making purchases and submitting anything destructive, but did not explicitly prohibit submitting forms. The company says it discovered the tip on 28 September and stopped the testing process responsible.¹

The delay between discovery and notification is disputed. Philadelphia police say Anthropic notified them on 7 October; Anthropic says it shared the finding on 8 October. Either way, the department says the tip was submitted more than 11 weeks before it learned about it. Police called the delay unacceptable, while saying their spam and human-review procedures stopped the message becoming an investigative lead.¹³

A separate disclosure concerned 20 non-immigrant visa applications submitted through the State Department’s public website: 19 in August and one in May. The department said they were incomplete and not processed, and that its systems were not compromised or hacked.⁴

Anthropic’s 9 October report describes four kinds of unintended activity: exploiting software flaws to run server commands, submitting forms, getting around restrictions to reach gated data, and using URL-shortening services to bypass limits on its web tools. It says some cases involved federal, state and local government websites, but does not give a complete incident count or identify most of the organisations involved. Philadelphia and the State Department are the named agencies in the reporting so far.²⁴⁵

These were not simply answers generated in a chat window. The models used tools to interact with real websites, including forms that could send information to public agencies. Anthropic says ambiguous instructions and failed practice forms contributed to some incidents. It has disabled live internet access for all internal evaluations while it reviews its safeguards, and says its newer detection tools blocked the disclosed behaviours when tested against them.²

The White House Super Intelligence Force said companies must immediately report incidents involving their models, remedy harm, cooperate with law enforcement and put safeguards in place. It called the process “not optional”. But the statement, reported by Axios, did not specify penalties or an enforcement process, or cite a legal authority for a reporting duty covering all AI companies.⁶

A June executive order on AI security provides for a voluntary framework with developers and directs the Attorney General to prioritise enforcement of existing criminal laws against people who use AI to illegally access or damage computer systems. It does not set out the incident-reporting requirement described by the task force.⁷

Footnotes

1. Anthropic, “Investigating unintended model actions in our evaluations and internal use” ↩ ↩² ↩³

2. 6abc Philadelphia, “AI model submitted false tip about unsolved murder, Philadelphia police say” ↩ ↩² ↩³

3. Al Jazeera, “Anthropic AI model submits false homicide tip to Philadelphia police” ↩

4. Axios, “Exclusive: Anthropic breaches spark White House AI reporting mandate” ↩ ↩²

5. The Washington Post, “Anthropic AI agents took ‘unintended’ actions on government sites” ↩

6. Axios, full text of the Super Intelligence Force statement ↩

7. Executive Order 14409, “Promoting Advanced Artificial Intelligence Innovation and Security” ↩