California Seeks to Implement AI Guardrails for Mental Health Treatment
A bill has been unanimously passed by the California Senate that seeks to establish common-sense guardrails for artificial intelligence (AI) use in mental health treatment. The bill, SB-903 Mental health professionals: artificial intelligence, was authored by Senator Steve Padilla and seeks to protect individuals seeking therapy or psychotherapy services in the state of California. The bill was passed by the Assembly with a vote of 71-4, and now awaits the signature of the state governor.
AI tools are being rapidly adopted by healthcare providers to automate routine clinical and administrative tasks and improve operational efficiency. The Trump administration is keen to expand its use further and is pressing ahead with plans to integrate AI more fully into medical care, including deploying AI agents for conducting therapy sessions and prescribing medications to patients under Medicare.
While the capabilities of AI have advanced considerably in recent years, mental health experts have voiced concerns about patient safety. AI algorithms may be able to accurately diagnose a mental health condition, but they are still capable of error, and in mental health, errors can have grave consequences. The bill seeks to ensure patient safety by requiring oversight by licensed mental health professionals and requiring patients to consent to the use of AI tools.
“Thousands of the professionals caring for patients across California are all urging Governor Newsom with one voice, “Sign this bill’,” said Senator Padilla. “The Trump administration is listening to tech companies who want to put unlicensed chatbots in front of vulnerable patients to make a profit. California should be listening to the licensed clinicians who spent years training to do this work safely.”
If signed into law by Governor Gavin Newsom, individuals, corporations, or entities that provide or facilitate psychotherapy services may only use AI tools to assist in providing administrative support services or supplementary support for psychotherapy services. AI tools or systems may not be used to conduct therapy sessions. If AI tools are used for making therapeutic decisions, therapeutic recommendations, assessment results, diagnoses, or treatment plans; detecting emotional states or mental stress; performing triage or screening; or directly interacting with patients or clients in any form of psychotherapeutic communication (unless FDA-approved and compliant with HIPAA), their output must be reviewed and approved by a licensed professional.
The bill prohibits advertising or otherwise purporting to offer psychotherapy services when they are provided through the use of companion chatbots, including claiming that a companion chatbot is a therapist or provides therapy. If AI tools are used to record or transcribe psychotherapeutic communications, psychotherapy sessions, or triage or screening, then the patient or their legally authorized representative must be informed about the specific purpose for which AI is being used, and must provide (revocable) consent.
