HIT Consultant – Read More

What You Should Know: 

– The use of artificial intelligence (AI) in mental health is a rapidly growing field, but a new national survey from Iris Telehealth reveals that consumer trust hinges on one critical factor: human oversight. 

– The 2025 AI & Mental Health Emergencies Survey suggest that while respondents recognize AI’s potential to expedite the detection of behavioral health crises, they overwhelmingly oppose any model in which AI makes final care decisions without human involvement.

Human Oversight Is Non-Negotiable

According to the survey, 73% of consumers believe that human providers should make the final decisions in AI-flagged mental health emergencies, while only 8% would trust an AI system to act independently. This preference for human control is a recurring theme. The biggest concerns cited by respondents are false positives (30%) and overreliance on technology at the expense of human connection (23%).

When a mental health risk is detected by an AI system, consumers’ preferred immediate response is human-centered. The top two choices for immediate support were having a pre-selected family member or friend notified (28%) or receiving a phone call from a trained counselor within 30 minutes (27%). Only 22% of respondents trust AI to immediately connect them to a professional without their explicit permission when a risk is detected.

A Generational and Gender Gap in AI Acceptance

The survey also uncovered significant differences in attitudes toward AI in mental health based on demographics.

• Generation: Younger generations are more accepting of AI. Only 5% of Baby Boomers are “very comfortable” with AI identifying mental health crises, compared to 29% of Millennials and 24% of Gen Z. A striking 63% of Millennials would use automatic AI monitoring tools, compared to just 26% of Baby Boomers.
• Gender: Men are more willing to embrace AI for crisis detection, while women prioritize human oversight. For example, 23% of men are “very comfortable” with AI identifying mental health crises, compared to only 13% of women. Furthermore, 78% of women want human providers to make the final decisions in AI-flagged emergencies, versus 68% of men.
• Income & Education: The survey identified an “income paradox,” showing that higher-income consumers are more skeptical of AI mental health tools despite likely having better access to technology. The lowest income bracket showed greater willingness to use automatic AI monitoring (61% acceptance) compared to the highest earners (44%). Similarly, more educated consumers were found to be more cautious. PhD holders were the least comfortable with automatic AI monitoring (31% acceptance).

For more information about the survey, visit https://iristelehealth.com/blog/ai-mental-health-crisis-detection-what-1000-americans-really-think-about-ai-powered-emergency-response/