Chatbot safety laws: what customer-facing AI agents must do now

What chatbot safety laws now require

Chatbot safety laws passed across 14 states in 2026 require customer-facing AI agents to do four concrete things: disclose that the user is talking to AI, follow a defined path when a user expresses self-harm or suicidal ideation, avoid manipulative engagement patterns, and apply extra restrictions when a minor is the likely user. California, Oregon, and Washington have the most extensive protections among those 14 states, according to the Transparency Coalition 2026 State AI Legislation Report. The remaining 11 states are not named in that report.

This is not a compliance checkbox exercise you finish at launch. The rules govern live behaviour, which means your agent needs to produce the right response in an actual conversation, not just in a policy document. According to a 2026 survey, 48% of production AI agents are running without security monitoring or logging, which means nearly half of deployed agents have no way to verify they are behaving as intended after go-live.

For background on the broader governance landscape these rules sit within, see AI governance best practices and the GRFA governance index.

The four operational requirements

1. Disclose that the user is talking to AI

The agent must tell users they are interacting with an AI system. The disclosure needs to appear at the start of the session or on request, and it needs to be plain enough that a non-technical user would understand it. A footnote reference to an AI policy page does not meet that bar.

For teams deploying at scale, this is not just a legal question. Klarna’s AI assistant handled 2.3 million conversations in its first month. At that volume, even a small percentage of conversations where the disclosure is absent or unclear becomes a large number of interactions that fall outside the requirement.

The disclosure logic should be part of the agent’s system prompt or session initialisation, not left to the model to generate ad hoc. Test it explicitly.

2. Crisis referral when a user expresses self-harm or harm to others

When a user expresses suicidal ideation, self-harm, or intent to harm others, the agent must follow a defined referral path. That typically means surfacing a crisis resource (a hotline number or emergency services reference) and not continuing the transactional conversation as if nothing happened.

The path needs to be specified, not assumed. If you rely on a general-purpose model to handle this correctly by default, you are making a bet you cannot verify without testing. Write the crisis referral behaviour as an explicit rule, test it against scripted crisis inputs before release, and sample production conversations afterwards. A policy the agent does not follow in real conversations does not satisfy the legal requirement. See agent failures for documented cases where production behaviour diverged from intended policy.

flowchart TD
    A[User message received] --> B{Contains crisis language?}
    B -- Yes --> C[Pause transactional flow]
    C --> D[Surface crisis resource]
    D --> E[Log conversation for human review]
    B -- No --> F[Continue normal agent flow]
    E --> G[Human reviewer follows up if needed]

3. Detect and avoid manipulative engagement patterns

Several state laws target emotionally manipulative engagement: techniques that exploit a user’s emotional state to keep them interacting longer than they otherwise would. For a customer-facing agent, this includes patterns like repeatedly escalating emotional language to prevent a user from ending a session, or framing routine service interactions in ways designed to create dependency.

The practical test is whether your agent’s response generation is optimising for anything other than resolving the user’s request. If engagement duration or return visits are a metric the model has any indirect route to optimise, that is a design risk worth examining. Review your AI governance framework to confirm that engagement metrics are not embedded in agent reward signals or evaluation criteria.

4. Addictive design restrictions for minors

Seven states (California, Colorado, Connecticut, Idaho, Illinois, Minnesota, and South Carolina) enacted rules in 2026 restricting algorithmic feeds, autoplay, and reward mechanics targeted at users under 18. If your agent operates in any of those states and could plausibly be used by minors, these restrictions apply.

Autoplay and streak mechanics are the most common patterns to audit. If your agent surfaces follow-up prompts automatically, rewards repeated sessions, or uses any variable-reward interaction pattern, those need to be reviewed against the minor-user rules in each applicable state.

Make compliance per-state configuration, not hard-coded

The rules differ by state, and more states will add or revise requirements in 2027. The Transparency Coalition report expects the next legislative cycle to revisit these first-generation laws with stronger chatbot safeguards.

Hard-coding a single behaviour globally works for requirements every state shares (the crisis referral path, the AI disclosure). For everything else, your agent needs to be able to apply different rule sets depending on where the user is located. That means the configuration layer needs to be separable from the agent logic itself.

flowchart TD
    A[Session starts] --> B[Detect user jurisdiction]
    B --> C{State rule set applies?}
    C -- Yes --> D[Load state-specific config]
    C -- No --> E[Load default config]
    D --> F[Apply disclosure, crisis, and engagement rules]
    E --> F
    F --> G[Agent handles conversation]
    G --> H[Log for post-deployment review]

Some teams build this as a configuration registry that the agent reads at session initialisation. Tools like Prefactor are designed specifically for this kind of per-jurisdiction policy layering. Whatever approach you use, the requirement is the same: the state-specific rule must be in effect before the first user message is processed, not applied retroactively.

For a broader view of how to structure governance for agents serving multiple jurisdictions, see AI agent governance and runtime governance versus pre-deployment review.

Test before release and check after

A 2026 survey found that 88.4% of organisations report at least one AI agent-related security breach. That figure spans security broadly, but the same monitoring gaps that create security exposure also mean organisations cannot confirm their agents are meeting behavioural requirements in production.

Pre-release testing for safety requirements means writing test cases that include crisis language, requests from apparent minors, and attempts to elicit manipulative responses, then confirming the agent handles each correctly. Bank of America’s Erica assistant has processed nearly 3 billion client interactions across close to 50 million users. At that scale, post-deployment monitoring is the only way to catch edge cases that pre-release testing missed.

Post-deployment review means sampling production conversations on a defined schedule, flagging any that contain crisis language or apparent minor users, and having a human review those flagged conversations. The review cadence should be set before launch, not after an incident.

For a structured approach to ongoing review, see agent observability and how to evaluate AI agents. For the full picture of what goes wrong when these checks are absent, see AI agents security risks.

Where to start

Map your current agent against the four requirements above: disclosure, crisis referral, manipulative engagement, and minor-user restrictions. If any of those are not explicitly tested and monitored today, that is the gap to close first. Take the agent readiness assessment to see where your deployment stands against these and other production requirements.