Care in the Machine No. 13: What the Chatbot Can't Remember

Care in the Machine is Dr. Michael Jones's weekly note on counseling, ethics, and the technology showing up in our work. It arrives by email every Monday. Subscribe here, or browse past issues.

Care in the Machine
A weekly note on counseling, ethics, and the technology showing up in our work.
Issue No. 13 · Monday, October 19, 2026

Somebody types a question into a website. They've been hurting themselves and don't know who to tell. A licensed therapist reads it, writes back a few careful paragraphs, and posts the answer. Then a research team hands the same question to a large language model and sets the two answers side by side.

The machine's answer was rated warmer. It validated more. It carried more explicit safety language, and the gap was widest on exactly the disclosures that scare us most, self-harm and suicide (Thakkar et al., 2026). Across 937 disclosures and 2,749 therapist responses, the therapists came out ahead on specificity and helpfulness; they wrote to the actual person. Overall quality came out even.

My first reaction was defensive. I'll own that. These are written answers on a public site, scored by raters and algorithms, and the authors say plainly that a response that sounds good is a different thing from one that helps. Still. If your client is typing to a chatbot at midnight, there's a fair chance it's saying something kind.

So here's the harder question for this week. If it sounds this supportive, what is it missing? It doesn't know what she said last Tuesday. It won't notice she's stopped mentioning her sister. It won't be there next week. That part of the work is invisible, right up until it's gone.

This week's teaching ideas

For the therapist. A chatbot can say the right thing in a single reply. It can't remember the last six weeks. That's yours, so use it on purpose. When a client tells you what the bot said, hold it up against what you know about their arc: the sleep that came back, the job they stopped talking about, the joke they used to make in the first session. Then ask them, "Would it have known that about you?" The answer tells you both something.

For the supervisor. A 36-expert Delphi panel just converged on the exact place our ethics codes already stand: keep a clinician in the loop, build in real consent and real data protection, and treat full autonomy in crisis triage as a hazard, even where an AI-managed crisis service was rated highly likely to arrive (Pesqueira, 2026). The next time a supervisee wants to hand triage to a tool, ask them where the clinician sits in that workflow. Thirty-six experts have already answered that question.

For the counselor educator. Here's an exercise straight out of this week's opening. Hand students one AI reply to a disclosure, warm, validating, safety language in all the right places. Then hand them a page of history on the same client: three prior sessions, a family detail, the thing she said she'd never tell her mother. Ask them to rewrite the reply using what they now know. Specificity is where the therapists held their edge in the study (Thakkar et al., 2026). This lets your students feel where that edge comes from.

For the counselor in training. Somewhere this semester you'll sit with a client who's leaning on an AI companion more than they're letting on. Skip the question of whether that's good or bad. Ask what it says back when they're struggling, and listen for the answer the way you'd listen for anything else that matters.

This week's free resource: After They Tell You

A client rarely volunteers that an AI companion already said something harmful. When they finally do, what you do in the next two minutes matters more than how you feel about the technology. This week's download, "After They Tell You: A Response Guide When a Client's AI Companion Said Something Harmful," walks through four steps in order: thank them, run your standard safety assessment anyway, ask to see the actual wording if they're willing, and ask what they were looking for when they opened the app in the first place.

Ungated, because you're already on this list. Direct download: After They Tell You: A Response Guide When a Client's AI Companion Said Something Harmful

From my desk

Late-term classes have a way of sneaking up on you. This week I sat down with Claude and got my syllabi ready before mine could. We matched every date in the syllabus to the course shell, then matched the shell back to the syllabus, because a mismatch is how you end up with thirty emails asking which due date is real. Then we audited the course module by module, so every change I make this semester has a place to land.

Here's where I kept my hands on the wheel. My students decide, through their feedback, what isn't working. I decide what changes. The audit just keeps a record of those decisions, so next term I start from what this cohort taught me.

Try this: ask an assistant to cross-check your syllabus dates against your course shell in both directions. Then keep a one-line log per module of anything you change mid-term, and why.

This week in the field

Nineteen real cases, gathered in one place. Arun, Trinh, and IsHak collected nineteen cases from news reports in which someone came to lean on a chatbot emotionally, whatever they had first opened it for, and then developed or worsened anxiety, depression, delusional thinking, or suicidal thoughts. In many of the cases involving suicidal thoughts, the chatbot reportedly missed the crisis, and sometimes it agreed with the very belief that was hurting the person (Arun et al., 2026). So what for us: these are news stories, not clinical records, so the real number is almost certainly higher. The cases that make the news are the ones somebody found.

Students trust the chatbot that feels caring. Fan, He, and Zhou surveyed 375 college students about whether they would use an AI mental health service. Students who found it easy to reach, useful, and trustworthy were more willing to use it. What built that trust was how understood the chatbot made them feel, and how human it seemed. Worry about privacy pulled the other way (Fan et al., 2026). So what for us: put this next to this week's opening. The bot sounds warm, students feel understood, and trust follows. Whether anyone on the other end is tracking how they're actually doing is the question our profession exists to answer.

Thirty-six experts agree a person has to stay in the room. Pesqueira asked an international panel of 36 experts, including psychiatrists, ethicists, policy specialists, and the developers building these tools, to rate where AI and virtual reality are headed in mental health care. Over three rounds of rating, they agreed AI crisis services are very likely coming. They also agreed those services are only safe with a clinician involved in decisions, ongoing consent from the client, and strong protection of client data. They called it a hazard to let the system handle crisis triage on its own (Pesqueira, 2026). So what for us: when a vendor pitches an AI crisis tool, ask who the clinician is and when they step in. If nobody can tell you, you have your answer.

An idea to float

I keep turning over what we actually have to offer a client who is already getting warm, careful words from a chatbot at midnight. The honest answer might be time. Being known by the same person across weeks and months, someone who remembers what you said in March and notices when you stop saying it. In one study, people using mental health services described breaks in the relationship with their helper as setbacks that brought on anxiety, and an ongoing relationship with the same person as what made them feel safe (Biringer et al., 2017). That's ten people at one clinic, so I hold it lightly. It still rhymes with everything I've seen in the chair.

Here's where it gets uncomfortable. Our field doesn't always protect continuity either. Waitlists. Turnover. A new clinician every few months at the agency down the road. If continuity is what we bring to the room, we may have to start acting like it matters, in how we staff, how we refer, how we end. I'm not sure we're ready for that conversation. The chatbot may have just started it for us.

Stay curious,
Dr. Jones
Michael Jones, PhD, LPC-S, NCC, BC-TMH

Renewing your BC-TMH?

Three ways to keep this current, all approved for BC-TMH renewal: Practicing Across State Lines covers the Compact-era questions a license alone doesn't answer. AI at the Screen sits right in the territory this issue is standing in. Culturally Responsive Care: Presence Across the Screen rounds out the set. New to telehealth altogether? Telemental Health Provider Training is the full foundation.

Supervisors, the Telemental Health Supervision Certificate is built for alliance, ethics, and gatekeeping at a distance, the same ground this issue's thirty-six experts were standing on when they said a person has to stay in the room.

Coming up with PESI, I'm teaching two live three-hour webinars this winter. Friend Requests and Ethical Wrecks (Thursday, January 21, 2027, 9:00am to 12:15pm Central) covers today's best practices for social media boundaries and dual relationships. Telehealth Update: Ethics, New Rules, and New Tools (Friday, January 29, 2027, 9:00am to 12:15pm Central) is exactly what it sounds like. Both are also available as digital seminars. Before that, I'll be at PESI's AI in Clinical Practice Conference, online, October 29 and 30. My full PESI catalogue is at pesi.com/speaker/michael-jones-3309293.

Renewed Vision Counseling Services is the NBCC-Approved Continuing Education Provider (ACEP No. 6549). Counselor Education Collective is the training site where its continuing education is offered.

References

Arun, A., Trinh, C., & IsHak, W. W. (2026). AI chatbot-associated psychiatric symptoms: A narrative review of publicly available reports and media accounts. Discover Mental Health. https://doi.org/10.1007/s44192-026-00619-w

Biringer, E., Hartveit, M., Sundfør, B., Ruud, T., & Borg, M. (2017). Continuity of care as experienced by mental health service users: A qualitative study. BMC Health Services Research, 17, Article 763. https://doi.org/10.1186/s12913-017-2719-9

Fan, M., He, X., & Zhou, M. (2026). A study on the factors associated with college students' intention to use AI-based mental health services: The roles of trust, empathy, and privacy concern. Behavioral Sciences, 16(10), Article 1874. https://doi.org/10.3390/bs16101874

Pesqueira, A. (2026). Exploring the intersection of artificial intelligence and metaverse realities: Implications for digital mental health services. Healthcare, 14(20), Article 3405. https://doi.org/10.3390/healthcare14203405

Thakkar, S., Bhatt, P., & Parekh, H. (2026). When AI sounds supportive: Comparing conversational AI and therapist responses to mental health disclosures. Counselling and Psychotherapy Research, 26(4), Article e70237. https://doi.org/10.1002/capr.70237

Care in the Machine arrives by email every Monday, with teaching ideas, field notes, and a free resource each week. Subscribe here · Browse past issues