This article is sponsored by SAIVA AI. This article is based on a discussion with Liz Borer, CNO at Millennial Health, Gill Bejerano, Chief Data Scientist at SAIVA AI, and Tim Tarpey, SVP at SAIVA AI. This discussion took place on September 14, 2023 during the SNN RETHINK Conference. The article below has been edited for length and clarity.
Tim Tarpey: What is AI? What is machine learning?
Gill Bejerano: Artificial intelligence is anything that a program does that seems intelligent to you. It is defined as the science and engineering of making intelligent machines.
Machine learning is a branch of AI which is about learning from examples. You don’t have to define the rules. You don’t have to say, “Well, if they have CHF and they have this and they have that, I want this.” You don’t specify the rules. You give a nice set of examples and the machine learns from that about the past and predicts the future for you.
Tarpey: How do you develop a machine learning solution for a provider of Skilled Nursing? What is the process, how does that work?
Bejerano: When we come to a new provider, we take their last several years of EHR history. We learn from that history which patient trajectories have gone well and which patient trajectories require the relapse into a hospital. We teach an algorithm to predict the future, which of these patients are going to go into a hospital and which of these patients are going to be stabilized and possibly even thrive at the nursing home.
Tarpey It took me a long time to understand the concept of machine learning. Can you give us an example of machine learning other than what SAIVA AI does?
Bejerano: Think about your spam filter when you read email. You get a lot of spam. There’s email that you like and there is email that you don’t like. Your spam filter is based on machine learning. Every day you say, I’m going to read this email. I’m going to mark this email as junk. Your spam filter sits there, learns what you like and what you don’t like. It only provides you the emails that you think are good for you. It also gives you an option to look at your spam box in case it makes mistakes. That’s a great example of machine learning used in everyday life. It focuses on you.
Tarpey: Liz, You’ve been with us for almost a couple of years. Tell us your story. How did it get started? Any skepticism, challenges, successes, anything?
Liz Borer: I think that it was that the staff, as we all know, our directors of nursing and our administrators in the facilities are just more and more challenged every day, from a regulatory perspective, from a staffing perspective, post-COVID, kind of going through that. We know the most critical part of our day is our morning clinical meeting. It’s really where we capture everything. A lot of hesitancy, because again, at first it was okay, something else, something new, something else we have to do.
Once the directors of nursing started to really wrap their arms around the AI and SAIVA, it’s really only as good as the documentation. Not only does it really pull from data that’s specific to PCC or whatever system that you’re utilizing, but the ranking of it really, really has put us in a great place. I think that once they start to see the feedback and they start to see that, “Okay, I did save that patient.” Going into the hospital transfer log, doing the QI tools is great with INTERACT, but having this push report every day for myself as a director of nursing, if I could go back in time and have this, I think I would keep every resident possible in the building every day. At least I would hope so.
Now they’re really wrapping their arms around it. It’s part of our daily clinical meeting. They were very apprehensive. It really took me getting on calls with them to make sure they’re looking at the push report and then the mobile app while doing the rounding. It’s very critical and I can’t see our day-to-day without it now.
Tarpey: Take us through a day especially right when an individual gets to the building, how it starts, and how machine learning is a part of that day.
Borer: Prior to having computer systems or electronic health records, obviously we all remember those days where your 24-hour report would just be on a big sheet of paper you would pull off. The director of nursing came in, and they were really just scrubbing data. They didn’t go back out to the floor very often. Today it’s much, much different. Now, we get our nurses back to the bedside to provide the care and services. Now, they come in, round with their nurses, see how the night went, move into a clinical morning meeting, hopefully, prior to their normal standup meeting with the executive director.
There are times now where our directors of nursing are on med carts. We are agency-free with our organization, so we do everything in our power to be able to continue that. At times, our directors are pulled to the floor, and I think that that’s why this tool comes in handy, across the board, in the absence of a unit manager, or a director of nursing, or the clinical educator.
In a high-skilled building, one of our centers took 80 admissions last month.
It’s a lot of admissions and discharges. Reviewing the 24-hour report for those admissions and transfers takes a lot of your day. Going through that 24-hour report to drill down on areas of change of condition that maybe the nurse didn’t capture, but it’s on the SAIVA report. Now I see it in front of me.
It saves so much time for the director and it gives her the ability to really go back. It cuts down the morning clinical meeting tremendously by eliminating the need to look through the electronic health record for changes of condition, and then they’re able to actually get to the floor prior to noon, let’s say, in those high-skilled buildings.
Tarpey: You are provided a ranked list of 1% through 15% of people in the building most at risk with one being the most likely to re-hospitalize or go back in the next 72 hours. Have you found that useful? How is that useful? Do you use the rankings?
Borer: Absolutely we do. It’s part of our census call every day when we get on with the teams. Did you review your SAIVA report? Were there interventions placed for somebody that was ranking? Maybe they were ranking one three days ago, but now they’re ranking three. We’ve put interventions in place, they’ve stabilized a little bit. Sometimes they stay on the report. We can take a look at that in their monitoring. Either A, when they return to the hospital, it’s really premature discharge as we all know. The length of stay at the hospital setting is much shorter and we’re getting more critical patients. Then add on your behavioral patients that we have to manage between the two from a skilled perspective and then along with some altered mental status that typically occurs. Then the ones that we’re able to put interventions in to start to see the ranking go down has been absolutely tremendous.
Tarpey: Why should nurses trust machine learning?
Borer: I think that no matter what they should trust machine learning because it’s always verified. If I see something that’s pulling into the ranking or I see data, I’m going to go and I’m going to verify. It gives me the ability to actually go back 72 hours prior to where they may be triggering now. Their BUN and creatinine might have skyrocketed and now they’re on the list and they’re at the top three. Then I’m drilling down. Was it something that we missed that was trending up as a clinical team prior to that? Does that now place me in an uncomfortable position from a regulatory perspective because now we missed capturing that three days prior?
We should trust it because it’s giving us the data there to really say, “Okay, this is somebody I have to drill down on further.” If you’re just looking at it and you’re not drilling down and looking at the whole, encompassing the whole patient as a whole, just as we would if someone wrote it down on a 24-hour report, I think that we should trust it all day long.
Tarpey: Yes, thanks. Liz, I’m going to read a quote from one of your DONs. I don’t know whether you’ve seen this. “SAIVA reports have provided the necessary information for us to have the best return to hospital rates in the county.” That’s from Dawn Brooks, one of your DONs.
Question from the audience: How do you know AI is working effectively to prioritize your nurse’s time? How do you measure that it’s working? How do you know it’s working effectively to prioritize their time?
Tarpey: We actually have a report that we provide, monthly and quarterly. It lists everybody who transferred out and where they were ranked for the 72 hours prior to their going out. In that report, we list everybody, and we show the rankings. We have a customer that I provided this report to yesterday, and I should have looked at yours, but 67% of the people who did go out in their first 30 days, were ranked 1 through 5 on our report. All their buildings had over 100 residents. 67% of people who returned were in the top 5 in the 72 hours prior to going out.
We help caregivers to prioritize care. We’re discussing SAIVA AI today, but there will be other machine learning tools that do the same thing.
Question from the audience: How do you know if the report is being used?
Open rates are measured, so we know how often people in a building open the report. Until just recently Liz, your buildings were at 100% for about 3 months in a row. Every building was open every day. Was that something you specifically enforced, or challenged your team to do?
Borer: Yes, we made it part of our daily clinical meeting where our regional nurse consultants or our regional vice presidents are in there and kind of a checkoff box these things have to be happening. When they’re not, and it’s communicated back, we go back through it in a very nice, gentle way. Their day-to-day life is very hard. Again, it’s to help them and to get them to understand why we do it and why we’re doing it.
That readmission, when the patient comes back in, the caseload that it takes on the nursing team is far and wide. Especially once our referrals, we’re taking patients and new clients, so super important, especially having the challenge with the staffing on the 3:00 to 11:00 shift and when most of our admissions do come in. It’s just that reeducation with them. Once they get back on it, whether it’s kindly or forced, gently, they understand its importance. They understand how much it’s actually saving from a regulatory perspective.
It’s not just re-hospitalization, it’s truly helping us. It’s helping the nurses understand more how to document. Again, these things are pulling up on the report and I’m thinking, oh, what was she thinking writing that in there, when we talk about defensive documentation and what really to stick to the facts when we’re writing nurses notes. We know that high scope and severity for immediate jeopardies and the potential for is just on a rise across the nation.
A lot of those things I’m able to capture, for example, on the report not too long ago, we were holding long-acting insulin. What that tells me is it was held, insulin not required, or per parameter, where we all know that from the clinical side, long-acting insulin should not be held without a physician’s order and/or it allowed me to be able to do a full company-wide audit to identify how many young new nurses are not getting that in orientation. It was pervasive, so I urge you to go back and look at yours for the new nurses that are holding long-acting insulin.
That is something that the report brought to my attention in reviewing it that could have been absolutely just detrimental to the organization and to the patient’s well-being of course.
Tarpey: Getting back to that thought of trust, we named this talk, “It works when you use it.” We have a customer, Care Spring, Chris Chirumbolo, who was here this morning. He came to our booth at the Ohio Health Care Association meeting where we had a couple of customer quotes on a pop-up banner. He said, “I’d like to have a quote on the banner.” I responded, “We’d love to have your quote on the banner. What would you say?” He told us, “It works when you use it.” That’s perfect, it works when you use it. That’s why you encourage your staff to use it.
Borer:Exactly.
Question from the audience: What is the measurement of Machine Learning efficacy?
Tarpey: It’s called recall rate. At the end of a given time, at the end of the month, we’ll calculate how many people transferred out compared to how many of those people were on the list for the 72 hours prior. Essentially, across all the buildings we work with, 80% of the people in the 30-day window will be on our list and about 60% for the general population, all hospitalizations.
Let’s go back to you, Gill. What has your team learned that we’ll see coming in the future? We have a new version coming out. What have you learned? What can we expect?
Bejerano: Maybe before I even mention that, I’ll just say, your staff works every day to provide us with beautiful data. Tim mentioned the ability of the models to learn every day. Everything that gets tagged, everything that gets noted at the SNF, whether somebody finished their meal, whether they were able to get out of bed by themselves, all of these data are just beautiful examples. I tell my students, for example, that this other hat I wear, that AI and healthcare are a match made in heaven. You play the same game over again with different faces, these are complex patients but you’ve seen previous complex patients like them and the trajectories are similar. The data that the staff collects is just beautiful. That’s first. As we started wrapping our heads around, we can actually predict the future, how could we better serve the SNF population? We came up with the following advances
First of all, we’ve been focusing on hospitalizations. We are now making that canvas broader. We would like for no unplanned transfer to happen. It could be a hospitalization overnight, it could just be an emergency ED visit. If you know your patients, you know that anytime you call 911, that’s an experience that is traumatic and could lead to further deterioration. Now, we are looking at hospitalizations, any unplanned ED visit, and any death on premise. Any unplanned transfer, any unplanned drastic change to the patient’s status is now going to be monitored and predicted for you. That’s one.
Beyond that, when we look at the scope of other things we could be predicting, again, based on the history of the SNF itself, we picked up on falls and we picked up on pressure wounds. As long as your facilities have documented in the past when somebody has had a fall, for example, in a risk incidents are portal, as long as you have that good documentation, and certainly with pressure wounds, we can actually predict for you who’s at greatest risk of having either a fall or a pressure wound in the coming 72 hours allowing you that horizon of actually proactively going in and preventing these events from happening. That’s I think is where we’re going. We’re very excited about that.
Tarpey: Liz, What are you excited about for the future? I know you use ChatGPT yourself. What are you excited about for the future?
Borer: I think AI as a whole is phenomenal. As we were speaking just before, I have the ChatGPT app on my phone, so everything I use, I use in there to break it down. Anything that’s going to bring the nurses and the nurse leadership and the team as a whole back to the bedside, our industry is shifting in a way where, again, from nurses, they are so fixated on making sure that their med passes in the green. They’re losing track of the patient and why we’re nurses, why we do what we do.
I believe that AI as a whole is hope for the future, that it will bring the nurse back to the bedside and remember why we’re all in long-term care and why we want to make sure for my mom, for everyone’s, for ourselves, being in long-term care, it’s super critical and important. I’m hopeful that this will be the thing.
Questions from the audience: One, I don’t think that it can make predictions as well as our nurses, and, two, our staff doesn’t document well enough. That’s not the exact same thing but it’s similar. It’s a resistance to change. The first one is “I don’t think it can make predictions as well as our nurses.” It can. It makes them better. We have numbers to show that.
We had bake-offs with a couple different DONs, a couple different companies. Our predictions are so much better than a DON can make. Each of those DONs were spending three hours per night before they went home generating those predictions. Both became customers and showed up in the morning and said that something better than their three hours of work is actually waiting for them.
The second part of that is our staff doesn’t document well enough. I guess both of you could have an opinion on that. Gill, what’s your opinion on that? Do we notice a difference in documentation?
Bejerano: Yes. Those two sides to this coin. Let me start with the positive. First of all, We plug into your entire EHR system. As long as there’s good enough documentation in there, we’re going to do the job for you. We are non-intrusive to your work habits. There’s just so much data being collected that if some quadrants of it are not as good as others, we typically can make do with that and just compensate from the other ones. We look at everything. The flip side of that is we can give you advice on how to improve some of these practices, point to the ones that would make the algorithm even better for you, and that would improve performance.
It’s an extremely robust tool that looks at thousands of variables per patient, per day. Every single day we collect thousands of data points for each of your patients. It’s robust and if you choose to, we can help you make it even better for you with some documentation skills, but we can come in exactly where you are right now, and because we look at the full spectrum, the robustness is just built into there.
Tarpey: Let me rephrase it for you, Liz. We often hear that documentation improves when people get our report, and they see the rankings, and then they see the second part of the report which shows their documentation. They either think this could be helpful, or wow, I don’t document well. Did you have that same experience?
Borer: Absolutely. I think our documentation has gotten far greater. I think another critical piece of it is the reports provide us with key words or key points that I’ve personally selected and got with my team on. Things like elope, strike, fall, and those are an additional feature for me that I find very beneficial that doesn’t even have anything to do with the ranking that highlights that. Whereas for me, in my position I typically pull a 24-hour report and do a key search and type those words in so you could see and understand how much time that would take me specifically to do that. I think that is huge and beneficial as well, but definitely, definitely has improved the documentation of our nurses.
Tarpey: Governor Parkinson said, “We’re doing God’s work.” Dr. Tariq this morning said, “AI can’t replace love.” Wesley Rogers yesterday said, “AI can help us provide meaningful info to lead us to provide better care.” How have you found that machine learning has helped your team to show your love by providing better care?
Borer: I think you just said it. I think that machine learning allows us the ability to be able to provide them with a safe psychosocial, physical wellbeing in our facilities when we number one, prevent that rehospitalization and that trauma that could occur from that, as well as giving our nurses the time to be able to go back to the bedside. It gives us the ability to share that love to be able to make sure they have a quick recovery if they’re there just from a short-term care perspective, and/or capture those things to prevent them, especially our long-term care residents that just that transfer out and back could be so detrimental to their health.
SAIVA AI reads your patients’ EHR for subtle clinical indications. Delivering reliable predictions, insights and notifications, modeled by your documentation, SAIVA AI gives your teams the tools they need to remain alert and aware of patients at greatest risk for clinical decline and hospitalization. To learn more, visit: https://saiva.ai/.
