AI in Mental Health Practice: Opportunities, Risks & Responsible Use

Disclaimer: The following transcript has been autogenerated and may contain occasional errors or inaccuracies resulting from the automated transcription process.

Host (00:01):

Hi there. Welcome to Mental Health Professionals Network podcast series. MHPN’s aim is to promote and celebrate interdisciplinary collaborative mental healthcare.

Angela Mariani (00:18):

Welcome to Mental Health in Practice, a podcast from the Mental Health Professionals Network. In this episode, we’re focusing on exploring AI and mental health, what it is, where it’s already showing up, the opportunities it creates, and how clinicians can engage with it thoughtfully and responsibly. My name is Angela Batra-Mariani. I’m an occupational therapist and co-founder of Everbility. I’m joined today by Mani Batra, a software engineer and co-founder of Everbility and David Lopis, a psychologist who works teaching clinicians how to use AI ethically and responsibly. Why is this an important conversation to be having right now? David, let’s start with you.

David Lopis (00:54):

Thanks so much and great to be here. So I think there’s a lot of excitement around AI and a lot of confusion and uncertainty as well. And I think that’s where we as clinicians need to be asking the question, what is the space all about? Why are clinicians finding this to be helpful and what are the ways to be using it responsibly? So I think it’s a conversation to become informed, to be able to think, how can I use it myself personally, but how are also our clients and the wider society making good use out of AI, I think. Over to you, Mani.

Mani Batra (01:25):

Cool. So I’ll give sort of a broader view from the technological standpoint. I believe, and I think a lot of us are already aware that this is one of the biggest technological revolutions of our lifetime, if not the biggest. The last was the internet and now it’s AI. It is more than just hype. It is actually providing a lot of value. I think the models or the AI models that are used behind these AI products have become sufficiently intelligent where the difference between people who use it now versus people who try and use these model two years into the future, they’ll be vast. So it is a great time to start playing with these tools and employing them in your daily life and your work and your personal lives and start reaping the benefits and also start learning because the curve of this technological revolution is very fast.

(02:13):

So there is a little bit of sense of like, “Oh, if you don’t use it right now, the gap between people who use it and don’t use it will just keep on widening and not trying to scare anyone. It’s also very exciting. So I’d definitely get on this train.

Angela Mariani (02:26):

I already agree with both David and Mani on everything they’ve said. It’s really important that we’re educating ourselves because we have a responsibility as health professionals to use it ethically ourselves, but also make sure, and David already touched on it, but I think it’s just a really important time that we’re all learning about it so that we can both use it responsibly, but also help others use it that way too. And I think we should also touch on what we mean by AI. AI is something that’s existed for decades and decades, but a few years ago everyone started to learn about ChatGPT. So large language models, LLMs is what most people are using synonymous with the word AI today. And we’re mostly going to be focusing on the use of large language models. AI is probably being used in so many products you’re not already aware of.

(03:13):

Does anyone else want to touch on what we mean by AI?

Mani Batra (03:16):

Yeah. So some products that already use AI, like Facebook, Google, these are software products used by millions and millions of people who have had an AI base for a very long time. Siri, as bad as it is using a form of AI. And I think the difference from traditional software or tools that you might have used for a lot of years is that these models continue to evolve with you. They learn from you. They learn from all the data that the humanity has ever produced so they can solve a multitude of problems rather than just a subset. So people are using them in very creative ways and people just keep finding new and new problems to throw at these models and they keep on solving them. So very exciting.

David Lopis (04:00):

I might just add as well, it’s quite life changing. In terms of what AI can do, I think once you start playing around with it, you are going to see your personal life and work life completely differently. And I say that without exaggeration. Once you start playing around with them, they are not requiring significant digital skills, technological savvy to be able to make a huge amount of difference to your day. So by the end of our discussion today, I would really like the listeners to be able to have a really good sense of what are the important tools for clinicians, how are they going to be making such a big impact? Some of the key considerations around ethical use, have a really good sense of the space. That’s what I’m really hoping for from today.

Mani Batra (04:41):

Completely agree with that. And just adding a couple of things. So a lot of people have already started talking about billion dollar businesses run by a single person with the aid of AI. So that just gives you an idea of the impact you can have if you start using these tools. I think one of the best compliments we have ever received as co-founders of Everbility has been that using the product gave us time with my family back. I think that just goes to show the far and wide impact this technology is having already.

Angela Mariani (05:09):

I think as well, we’re going to talk about risks. There’s obviously some really important considerations. It is easy to use AI. It’s also easy to use it unsafely. It’s also easy to make mistakes. At work, we have some pretty clear policies with our employees at a health tech company about how they can use AI for work because it could also put our company at risk. It could put our customers at risk if it’s used inappropriately. So it’s really great to go and play and we totally encourage everyone to go and start understanding how to use it. And I think it’s incredibly exciting and we’re going to talk more about those risks as well.

Mani Batra (05:44):

And David, I know you are teaching people about how to use AI. Are there some common patterns of mistakes that you see like new users doing that they shouldn’t be doing, especially in a health setting?

David Lopis (05:55):

Yeah, absolutely. And some of those mistakes are quite worrying from putting in client information into ChatGPT. It’s a very common concern I hear over and over again. And if you are listening to this and wondering how that’s a problem, that’s all right. We will discuss and explain the big issues associated with that. I hear of people who are, I guess having clinics where staff are coming in not being taught of what is responsible use, who are letting their staff fall through the cracks, beginning to use the tools without, I guess, a sense of consent, for example, of having those good explanations of here’s what the tool is, here’s what you are agreeing to, here’s what will happen with your data. And that is a major concern because clinicians are obligated to have that kind of discussion and ensure it’s agreed to. I see AI as being a huge knowledge base of information, number one, in that there is a great deal of past information from the internet, from research, from videos, from all forms of modality as one side of things.

(07:01):

There’s a massive amount of information you can access. But on the other side of things, I see it as a really capable analytic brain where it can understand and summarise past information, access past information strategically, put that information into what form you would like it to be in and also be able to even, and this is what really excites me with AI, can even extrapolate and take further ideas from what we know about already from the past due to what we were wanting to know and explore in a more maybe nuanced way. I see AI as having those different kinds of capabilities that we’re tapping into.

Angela Mariani (07:42):

I think the fact that it can extrapolate and go further is both an opportunity and a risk, especially when we start to think about clinical work. We need to be careful that we’re not relying on AI to do clinical reasoning for us and make sure that we’re giving it that part of the equation. Because if you are asking it to do a task in a clinical setting, it may go, “Okay, well, you also need to come up with a plan and all these other things.” And it can be very helpful in a not great helpful way sometimes. So I think it’s kind of a double-edged sword always with AI and we’re going to talk a lot about that. I did also want to talk about organisations, maybe not giving any guidance, but also I’ve seen this other end of the spectrum where they’re completely banning AI use and people are still using AI and just not telling their employees, which is actually, I think, a huge risk.

(08:29):

So I think it’s really important that organisations are educating their staff about safe use, which tools are approved, which are not understanding their responsibilities as an organisation and what their clinician’s responsibilities are for their clients so that that’s actually being done in a safe way.

Mani Batra (08:47):

For some reason, the song who let the dogs out just kind of rang in my head. They’re like, “AI is a dog that has been let out and it’s not coming back for sure.” So yeah, you can try and stop people for a while, but people are using it like Angela pointed out in incorrect ways and people will use it because the impact and the positive effect that it can have on a person’s life and time and wellbeing and just the quality of work they can do is massive. It is not hype. I’ll say it again. So if you are in an organisation, if you are a leader and your team is not using AI or saying that they’re not because I’m sure they are, I would encourage building some practices and I think David can touch on this around how to use AI responsibly.

(09:28):

And if you are an individual working in an organisation, I would definitely urge you to just become an advocate for it because it can benefit not only you, but like your peers and your entire organisation.

Angela Mariani (09:39):

And your clients too. Exactly. There’s such an amazing opportunity that we can provide more accessible resources, better communication with other care providers in that person’s life. There’s so many more things we can do now because of AI as well. So I’m sure, David, you’d love to add to this.

David Lopis (09:57):

Yeah, look, absolutely. A few things I’ll mention from what we were discussing earlier. So when we have staff members using AI without permission and without the organisation being aware, I would call that shadow AI use. I’ve heard it talked about in that way. It’s maybe not the most exciting of topics because it’s not as fun as and exciting to talk about as all the great uses, but from a compliance perspective and from risk management organisations really need to be thinking about this, as I said, in terms of induction, in terms of their policies that they need to be having, in terms of ongoing PD and keeping up with the regulations and all of that as time goes on and as regulations and uses evolve over time. But in terms of the uses of AI, let’s sort of nosedive into that. As of course Mani and Angela will talk about in terms of the AI scribes, which is probably the most popular use amongst clinicians.

(10:44):

What that involves is where there is often a recording of the session that’s taking place whereby the client has given consent and they should be knowing about what is happening with their data, as I’ve alluded to, and be using a tool which is compliant with Australian privacy principles. That’s quite critical if you’re in Australia. Worldwide, there are other compliance systems like HIPAA and GDPR to be having in mind. And the idea is that once you have completed that session, because you don’t really have to do anything once the session starts, you are just hitting record at the beginning and then the session just happens with your client as you would ordinarily have your session and then your audio of your session then becomes a transcript which might then become your preferred chosen note form. You would’ve selected a template which best represents how you work, which could be something you’ve put together from the past from how you’ve worked previously, or it could be using one of the ones that the AI company has provided and then the note that gets produced, you would then be checking, making sure you’re happy with it and then putting into your practice management software would typically be what you’re doing with your note.

(11:53):

And it would be very similar if you have a report, which could be a short letter or it could be very long report that has a similar sort of workflow associated. And so the AI scribe use is probably the most common one, what most clinicians are aware of and the most sort of time saving use. But there are plenty of other uses that maybe we’ll go into as well, but I thought maybe it’s just worth reflecting on your perspective on the scribes in general and that kind of use.

Angela Mariani (12:17):

Definitely. Scribe was kind of a logical add-on to healthcare. The company that we created, we actually focused on lengthy reports to begin with. And I think that that was due to my experience of working as an occupational therapist and we added scribing in later. And I think there’s so many opportunities for using AI for admin-based work. I think for clinicians listening to this, it’s important to know how to evaluate different software. So ensuring that it’s meeting your compliances and not just a checkbox, is it meeting a compliance? I would actually find out what they’re doing with your data, how they’re using it. You can meet Australian privacy principles and still do some pretty … I don’t think they’re ethical. We don’t do them at Everbility. So I know that there are companies that do things with your data that they’re allowed to do under those guidelines, but I still don’t think I would be comfortable with them doing them if it was my data.

(13:06):

So I think really actually reading privacy policies of these tools, understanding how long they’ve been around now because it’s so easy for people that are non-technical to create software, it’s also really important to understand who’s behind the software. There are security implications that someone who isn’t technical might put themselves in. So there’s a level of evaluation that you need to do and I’m sure money can actually probably add in a more rich way to this.

Mani Batra (13:33):

Yeah. I think the easiest thing to ask or inquire about the tool you’re using is like, how are they using your data? Are they using it for training their AI? If they are using it for training their AI, are they removing all the personal identifiable information as it might be in the data because you’re dealing with clients and all of their personal stuff? Do you have control on the data? When you delete the data, is it deleted from the systems? So these are the very simple questions you can ask. And obviously having compliances like GDPR, HIPAA is another very big one, and SOC2, these just add a little bit of extra trust. Going back to use of AI, there’s obviously use of AI with clinical data, like David mentioned, you get a recording, then it can be made into a note or you can write reports.

(14:16):

There’s also use of AI for other admin stuff. In healthcare, I feel they’re not enough tools right now that take care of all your admin needs, but we definitely envision a future where a clinician is just doing what they’re best at, which is talking to clients and all the other burden is taken care of by AI if not completely with the human in the loop. And a good example of this is we obviously tinker a lot with AI. So we have an email assistant running for us twenty-four seven at our home, which gives us summaries of emails. There are security implications. That is why we do not have it live in Everbility right now because you’re handing sensitive data when it comes to email, but this is one of those things that’ll just start pervading different areas of your life, your work life and just keep on becoming more useful and start saving you more and more time.

(15:06):

And I think David said a very important thing where whatever is being generated by these models needs to be verified. A lot of AI companies come in with a lot of hype where they say like, “Yeah, this model is more intelligent.” And the fact is it is, but even the best and best of models are not at a stage where you could just blindly trust their judgement. They’re very driven by the questions you ask. Their output is very much shaved by the data that they have been trained on or have access to. So never ever, especially in a healthcare setting, I would say make a mistake of just trusting whatever’s been generated by AI, double check it, make sure if they are mentioning any references, they are actually real references because these models have tendency to hallucinate, but definitely try them out.

Angela Mariani (15:54):

Wonderful. David, over to you.

David Lopis (15:56):

So in terms of other uses that are not just the scribes, and these would be far less known by many clinicians, for example, uses of AI for research. I think this is very exciting, very underrated. And what it allows for you to be able to do is to look at masses of information in research journals and ask questions that you can try and make them more nuanced, but they might just be less research that’s able to answer those questions. But you can try all sorts of questions you’re wondering something about and have AI tools like, for example, elicit or consensus, be able to do a very extensive analysis for you and give you sources along the way. Again, got to obviously double check and verify everything, but that’s something which I think is going to only get better and will allow us to be better science practitioners and really has a lot of potential.

(16:44):

And also AI tools allow you to create great visuals, which for your clients, if you’ve got information that might be better presented in an infographic that looks more fun and engaging and I think that a lot of people are better learners when they can see information that’s visual. And so really what we’re talking about with AI I think is a whole new exciting way of working with clients that’s more accessible, that’s more individualised, that’s better supporting of them and better able to create a relatability and I think is going to lead to far better outcomes. And that to me is really the bottom line. But at the same time, being able to assist clinicians to go about their day without the burnout, being able to look forward to their client work and focus much more on that sort of face-to-face relationship, I think that really is everything.

Angela Mariani (17:34):

I totally agree. I think as well a couple of other interesting things that are happening in terms of how AI is showing up, people are starting to use AI to answer phone calls. I will also add to the research actually in another consideration to make around using research tools for AI is actually knowing where they’re searching because if can just search anything, you may be getting a very big range of quality information. So you need to understand how to drive these tools to really get the most out of them. People are starting to use AI chatbots as a mental health tool that obviously has risks and some potential benefits as well. So understanding that people are going to go and talk to ChatGPT, even if it’s not a mental health specific tool, your clients are going to be doing that as well. And so educating people about how that might happen, understanding there’s new actual conditions coming out, mental health conditions based on AI use like AI psychosis.

(18:32):

So it’s important to actually educate the people we work with about the use as well. It’s a really great opportunity right now. These tools are in such early development that professionals that are experts in this field get involved with people that are building software to help build that future that we’ve all alluded to where clinicians are actually being able to do the work that they’re amazing at without all the burden of understanding every single condition in that depth. Obviously we have to understand them, but I mean you have a partner in the process that’s handling admin, that’s helping you do research, that’s helping you stay on top of tasks, that’s making sure you follow up with clients. There is a future where this is going to happen and I think that if clinicians are actually helping create that, this is the time to start doing it.

(19:20):

So I think that’s another really exciting opportunity that I wanted to mention.

Mani Batra (19:24):

And I would just say just start, try and generate an image and ChatGPT. It doesn’t have to involve any client data. Just start getting a feel of how to talk to these tools because that’s a skill in itself and you’ll just start coming up with new and new ideas. So yes, there is an aspect of ethics and responsibility, especially when dealing with sensitive client data, but feel free to use it just as a sounding board.

Angela Mariani (19:48):

If you go to the website as well, there’s going to be some resources below the podcast and you can find more information about how to make sure that you’ve covered your different responsibilities. So like the app or guidelines around AI use, we can provide a link with some simple questions to answer for yourself when you’re evaluating different software. I’m sure David has got plenty of resources as he teaches clinicians there. So there’ll be plenty of resources that he can share.

David Lopis (20:14):

Yeah. I mean, I guess with regards to the risks and responsibilities, I want to make explicit that we have to manage the possibilities of mistakes. We have to be prepared for anything that AI does as being potentially sounding confident, but that doesn’t necessarily mean it’s more likely to be right. And we have to always be very skeptical to use it in ways that are going to be helpful and meaningful, but always critical. There’s also the possibility of hallucinations, which is where it’s making things up that have not occurred, thinking there are sources but they don’t exist. We also need to be mindful around having all of this assistance with AI, there’s the possibility that we can become complacent. We can find our work to be too easy and start forgetting our traditional skillset. And I think we therefore need to be deliberate in continuing the learning and you can even use AI to help you stay fresh to keep challenging you and/or having other strategies, for example, having some non-AI days.

(21:13):

So all sorts of different ways that you can promote your growth and continue to use AI but also manage many of these concerns. There’s also a lot of debate around to what degree clinicians take ownership of what is produced with AI. And I think best practice is to assume that the clinician does have to be the human in the loop, meaning that they are taking ownership of what they are using AI to complete by having the right kind of prompt and putting in the information that is appropriate to feed in, but also at the same time, always double checking before you send something off before you are finalising any particular AI task. I guess as an AI educator in this space, I think it’s absolutely critical that clinicians, before you get started, you go and have AI training that is not just a demo with an AI company where you are learning about ethical use of AI.

(22:09):

You’re learning about what does responsible use mean, consent, what kind of consent discussions do you need to have and how can you stay on top of good uses using the tools appropriately and being, I guess, an informed user of AI before you’re starting to use AI, before you have your admin team or peers in your clinics start to use AI. Obviously I’m a little bit biased than someone who runs this kind of thing, but I just think that responsible use and the only way to stop people from dumping a whole lot of information into ChatGPT is to be knowing from the get- go what is acceptable, what is not acceptable, what sort of things can you do with some tools and what sort of things can’t you?

Angela Mariani (22:47):

100%. I agree that clinicians need to be learning generally how to use AI safely. A lot of people have the misconception that simply taking people’s names and date of birth and address out of information is enough to make it de- identified. And when it comes to these models, there’s a lot of debate around this, but I don’t believe that that’s enough to be putting that into ChatGPT and having that being trained. They’re incredible at pattern matching, that’s still a problem. So understanding how the systems are actually working as a company, we also provide this kind of training too. So outside of demos, I think that there is a lot of opportunity to learn and to make sure that your team is using a software that is along the process with them and helping staff really understand what they’re doing.

Mani Batra (23:34):

Go use it, but just use it responsibly and learn how to use it responsibly.

David Lopis (23:38):

I think AI is already enormously helpful for accessibility, which I mentioned earlier, but I’d like to elaborate a little bit on what that looks like because if you have a client who for some reason might need a letter or might even just want information explained, you can with a prompt make it more suitable given their literacy skills or suitable given their cognitive abilities. Or you could even make it related to their interest in Batman so that they might be able to better understand it through ideas and concepts that come easier. So I really see AI as being able to better assist you as a clinician to connect with your client and to be able to use tools that make your key messages and strategies better understood and have them be able to walk away from the session getting so much more and having a bit of a better sense as to how your support can actually lead to meaningful change and maybe better understood, maybe also having a better rapport with the client.

(24:40):

I think all of these are really going to be exciting new steps to come from many clinicians’ uses of AI.

Angela Mariani (24:45):

When you’re talking about this, these things are already possible, which is so exciting. And when I start to think of the things I think can be possible in the future, I also get really excited. I can imagine a AI system that is being used alongside a clinician. The clinician just sits down with their client, they’re present, they don’t need to sit and type and refer to things in their notes and it handles that whole part of it and it helps with all of that, but also helps with creating new research, helps with understanding how people all around the world are being helped and treated and what’s working and what isn’t working and, oh wow, this person in a tiny little country town in the United States had this situation and these things really helped them. And oh my gosh, this other person on the other side of the world also had this thing that helped them.

(25:37):

And there might be a pattern forming here that it would’ve taken so much longer and this is not happening now, I’ll just make that clear, but I think there is a future where we can have those kinds of insights and learnings and those sort of grainfield ideas, that’s what’s really exciting me about the future of this stuff, something I didn’t mention, but I have a sibling on the NDIS in Australia and a lot of what we do at Everbility is also being driven by the passion I have about making healthcare better for her and for people like her. So I think there are so many opportunities for how AI can be used as professionals that is going to make our clients’ lives better and that’s why I get up every day and do this.

Mani Batra (26:20):

I think that’s great. And I think we are kind of entering the era of personalised tools and personalised software, whether it’s you learning new things, becoming a better clinician, you will start encountering tools that help you do that in a very personalised approach that works just for you, like a tool for one. And similarly, like David touched on, when you’re disseminating all the information or all the experience you have and helping your clients, you can do that in a very personalised manner and you don’t have to worry about like, how will I do this for 100 clients? AI does give you that ability where you can personalise information for each of those clients and help them get the best out of it.

David Lopis (26:58):

One of the real challenges with AI for anyone who is following the space is trying to find the sausage and not just the sizzle because there’s a lot of hype, there are a lot of tools with a lot of promise and there are a lot of tools that are very, very capable and being able to distinguish between the two can take some time and can take, I think really having a good community of peers who are discussing the AI space, and this is something that I’m busy building myself as well, but I really think to be able to know what are good tools to be able to make use on your own personal use as well as your professional use and what’s going to be meaningful in your life means continuing those discussions with others to find tools that are going to be actually helpful.

(27:39):

I might also just add in terms of tools that change my practice, I think many clinicians will be really excited to hear there are tools more advanced, I would say, than like ChatGPT. I’ve been really excited by Manus where you could put in masses of documents and put in a whole lot of very detailed prompts about what you’re wanting and get workshops developed in great detail and put them for whatever format you’re wanting or being able to have multiple documents or different kinds of files produced within seconds that might’ve taken you hours to put together yourself. So really I think the AI space is going to change how you see your workday, change your headspace, allow for you to do more in your day that excites you, that energises you, that makes you see your day differently. Finding the tools that work for you is key and I really hope that today’s chat helps make clear some of the directions to get that information.

Angela Mariani (28:33):

Thanks, David. Obviously we could talk about this stuff for days. So if anyone wants to contact us, feel free. I’m sure David doesn’t mind either being reached out to. AI has opportunities, it has risks and you can find out more about those. Really give it a go, go and start learning. It’s not scary. David mentioned you don’t need these heavy technical experience or skills to really start using it. You can just talk to it. It has some knowledge, but you still need to guide it and make sure that it knows what it’s doing.

(29:06):

Thanks for listening to Mental Health in Practice, a podcast from the Mental Health Professionals Network. If you’d like to learn more about today’s guests or access related resources, visit this episode’s landing page. We’d also love your feedback. You’ll find a short survey on the landing page to share what was useful and what you’d like to hear more of.

(29:22):

Thank you for your commitment to multidisciplinary care and lifelong learning.

Host (29:27):

Visit mhpn.org.au to find out more about our online professional program, including podcasts, webinars, as well as our face-to-face interdisciplinary mental health networks across Australia.