AI Transforming Health Care: Full Event | Kaiser Permanente

Rebecca Flournoy: Welcome to today's forum on AI in healthcare. My name is Rebecca Flournoy, and  I'm a senior health policy leader at the Kaiser Permanente Institute for Health Policy. And I will  be your emcee for the day. So, at the institute, we host events and publish information to shape  policy and practice on important health topics, exploring ways to promote high-quality, equitable  healthcare and healthy communities. And we are so excited to see all of you here today. I  think many of

us are here because we see the really exciting potential that AI in  healthcare could bring. The way that AI in healthcare could help us promote patient care  and population health, reduce burnout among healthcare professionals, and make healthcare  more efficient, accessible, and affordable. And I think at the same time, we also recognize  how critical it is to be really thoughtful about how AI is designed and implemented and monitored  from the start. What needs to happen to ensure that AI and

healthcare is well designed and  well-governed? How can we ensure that AI is safe, useful, trustworthy, and equitable? We are going  to dig into these questions together today. We will start the day out with a fireside chat to  hear about current uses of AI in healthcare, bringing in some examples from Kaiser Permanente.  We'll then shift to a session on the future of AI in healthcare, bringing in some additional  current examples, but also looking to what's on the horizon, and some important c

onsiderations  to keep in mind. We will then break for lunch. And then in the afternoon, we'll have a session on  practice and policy and ways that we might address risks while also harnessing benefits. And then  we'll hear some closing reflections at the end of the day. I know some of you submitted questions  in advance. And I want to thank you for those and let you know that the moderators and speakers  have seen those, and we will do everything we can to incorporate those into the presentatio

ns  and the discussion. And then we'll also have a little time at the end of each panel for a  couple of live audience questions as well. Before we begin a few housekeeping items. We are  recording the event today, so please silence your devices. And if you need to make a call, please  either step outside or you can see one of our staff members to see if a room might be available  for use. To access information on the event today, you've probably noticed that there are cards at  your tables that

have information on connecting to Wi-Fi, and also a QR code that will take you  to a website with the agenda and speaker bios. If you're using social media to share posts  on today's event, you can mention us using the handle @KPIHP, and hashtags like AI and  healthcare AI. And then a final note here in the event space. Restrooms are located behind this  screen, down the hall, and to your left. And then emergency exits are located behind you through  the doors where a lot of you came in this mo

rning. So, now I want to transition to our first  conversation. I want to welcome up to the stage Tony Barrueta and Vivian Tan for our  opening fireside chat. Tony will introduce Vivian in just a moment. And Tony, as you come  up, I will tell the audience a little about you. Tony Barrueta is senior vice president of  government relations for Kaiser Foundation Health Plan and Hospitals. And in this role,  he oversees our legislative and policy efforts, leading a team of legislative advocates and 

policy professionals. He dedicates -- excuse me. He directs the development of her public  policy positions in support of the organization, our members, and the communities we serve. And  I will turn it over to you, Tony. Thank you. Tony Barrueta: Great. Rebecca, thank you. And let me add my welcome to all of  you. It's really wonderful to see so many people come out, you know, here at the Center for Total  Health. This has been a wonderful space, really, for the Institute for Health Policy to

put on a  number of different health policy forums. I think, you know, we've gotten restarted again  now after the pandemic. And prior to that, we just had this amazing run of really interesting  sessions. And I think we're back on track again, Rebecca, I think it's fair to say, with this  as well. So, I just want to welcome all of you. So, I am really happy to be able to have  this role today interviewing Vivian Tan, my good friend. Vivian has been with  Kaiser Permanente for 16 years now. So,

she's just barely not new  anymore at Kaiser Permanente. [laughter] You know, and she is really one of our key leaders  within the organization who is delving deep into issues of AI, machine learning, and all of the  others. And she has been -- and she has a really unique perspective within the Kaiser Permanente  organization. Because in addition to her current role as vice president of Strategic Information  Management and Global Relationships for Kaiser Permanente, the Global Relationships has

her  really representing Kaiser Permanente at the World Economic Forum, where there's a tremendous  amount of this work going on and a lot of policy thinking and development that's going on  there. She's had a number of roles within the organization that have her really right  at the intersection of senior leadership, engaging with the most interesting things  that are going on at Kaiser Permanente. She has been, you know, a key person in  the organization. She and I kind of grew up together in

some respects, providing support to  senior leadership on key initiatives and public policy. And still, she works very closely  with our wonderful chief financial officer, Kathy Lancaster, on a variety of issues, and  our CEO, Greg Adams, who takes a tremendous interest in this subject as well, and I know  has all kinds of thoughts on this subject. So, she's currently leading a lot of the  work that has been transformational in terms of what we're doing. And what  anybody who knows anything abo

ut Kaiser Permanente would know is it is a very  interesting and complex organization, and there's a tremendous amount of creativity  that goes on in a whole range of different places. And it's really -- I think Vivian's role is in  part to keep track of all of that stuff that's going along and figure out how to get it all  rowing in a common direction for the benefit of the people who we serve, our members,  and the communities in which we operate. So, Vivian, I'm delighted to be  able to be he

re with you. Vivian Tan: Thank you for having me. Tony Barrueta: I think it's fantastic because, you know, a key part -- I was just chatting with  Marilyn from BPC. And one of the things that's really necessary in this area is just kind  of explainer stuff and one-on-one stuff. And you know, can we give a basic overview of what's  going on? So, I'll ask -- I'll pose this question the way I pose all of my questions. As you've  heard me say many times, I'm not a healthcare person. I'm a History ma

jor. So, what do I need  to know? What can you tell me about the basics of AI and what's really happening right now, in  a way that a History major would understand it? Vivian Tan: Well, thank you, Tony, for that question to start off things. And welcome  everybody. Tony calls himself a History major, but he's much more and far from that. Maybe  we'll start with a primer. Some of it might be very rudimentary for you. And then for  others, hopefully, it's helpful. So, Tony, I always like to try t

o think of an analog or a  parallel. And if you think about AI like human intelligence, right, and you think of the phases  of human understanding and then sense-making and then actually being able to respond to it,  AI has, like, has -- you can think about the same components. So, in the understanding, you  know, like human beings, we have five senses, right? There's a lot of AI technology to help us  process a lot of structured and unstructured data. So, when you think about text and images an

d video  and you hear things like NLP or Computer Vision, that actually helps. You can use AI techniques to  ingest a lot of information. The second, you know, kind of again, likened to human intelligence is  the sense-making part. And you're trying to use the data that you've collected to make sense  of it and find patterns in the data. And you apply machine learning techniques, deep learning  techniques, reinforcement learning techniques. They're supervised and unsupervised. All these  are buz

zwords, but at the end of the day, you're really trying to figure out if the  information has patterns to predict outcomes, to diagnose things, to actually say that  this happened or did this not happen. And for those kinds of things, for a while, they  were very narrow in nature, right? You build a model, it could play chess, but it cannot do  anything else, or it can play the game of Go as many of you know with the Google Brain. And  now, we're in this phase where it's maybe the evolution of w

here it's going to self-generate  information and content. And this is the rise of, you know, foundational models that are  based on texts. If they are based on text, they're called LLMs. They are basically  transformer-based models using, again, mimicking the brain in terms of how you think  about -- you have layers and layers of processing. And you hear buzzwords like prompt engineering  or rag or fine-tuning, and all of these things actually just help us make sense of a lot of  data. Basicall

y, if you look at OpenAI, you know, ChatGPT, it's been trained on every piece of  information out there in the internet. And it's --people are speculating. And actually, we  have a Microsoft expert here. Maybe he can verify this [laughs]. The people are speculating that  it's actually trained on 100 trillion, you know, tokens of information. This is humanly not  possible to process. And it's very -- it's become very sophisticated in terms of predicting the next  word, but it essentially is a nex

t-word predictor. Tony Barrueta: So, it's sort of a term -- I mean, AI is sort of a term that  covers actually a lot of different things. Vivian Tan: A lot of -- yes, yes. Tony Barrueta: So, what types of things are we really focusing on? What types of things should  people be aware of that -- where this is moving? Vivian Tan: Yeah. You know, I have a little chat up there and I know it's  very busy. It's got a lot of things on it. And maybe I want to tie AI back to healthcare. And  you know, AI

is an enabler at the end of the day. It's a tool. You don't do AI for AI's sake. So,  I want to maybe tie a feature of healthcare with the application of AI. So, the first feature  of healthcare is that at the end of the day, people are heterogeneous. They are  not the same. And we are -- you know, I think one of our key leaders always says  healthcare is about people caring for people. So, on one hand, people are heterogeneous  that we're caring for, and the people that are providing that care

are experts,  right, and there's a lot of shortages, right? We don't have infinite resources. So,  you really want to match. You want to segment your populations and really match them with  the right care at the right time at the right place. And that takes -- that can be powered by  AI and that drives value. So, that's, you know, an example where we're applying AI to  healthcare in a way that drives value. Tony Barrueta: So, that's a staffing support issue, basically. Vivian Tan: It can be. Yes

, it can optimize staffing. It can optimize  supply chain. So, I'll give you another example. Healthcare is a dichotomy in a way, right?  We have long lead times. If you want a nurse, you can't get a nurse, you know, on the spot,  on-demand. If you want some drug that needs to be shipped from another part of the world. So,  there are long lead times. And yet it's also a real-time business where if you don't use that  bed, you don't use that exam room, you know, you use it or lose it, right? So,

that dichotomy  of long lead times and also the need to actually optimize things real time really, again, is  another place where AI would -- using prediction and prescriptive analysis, you can actually  match and optimize for your health system. So, we have a lot of examples of that in Kaiser,  where we actually have a real-time app that's running in all 40 of our hospitals that give  all our clinicians and operators, you know, up to the minute -- okay, no, it's up to five  minutes [laughs] whe

re it's refreshed. They have it at their fingertips on the go. They  can see everything that's happening. And we predict sensors. We predict staffing both for the  current shift and two shifts out, and beyond that. Tony Barrueta: So, just so I -- because you told me about this the other day and I'm like blown away  by this. So, this is really real-time examination of the population currently in the hospital  or I guess, scheduled to go to the hospital. Vivian Tan: Yes. Tony Barrueta: And AI is d

oing what with that? Vivian Tan: So, AI is helping with the throughput management. And you know, if you actually have  good throughput management, you also improve the access and also the experience. So, it's kind of  like it makes us more efficient, which has a cost impact. And it also improves the experience.  You know, no patient, no member wants to be waiting for something to happen. So, we predict  and highlight the bottlenecks and constraints in the system so that we can act on it, you  kn

ow, in the moment and get things resolved. Tony Barrueta: So, it's sort of -- I was thinking of this in terms  of is it -- it's kind of predicting what this current patient population is  going to need a couple of days from now. Vivian Tan: Yes, in the same shift and two shifts out. Tony Barrueta: Oh, wow. Okay. What else is there? [laughs] Vivian Tan: And we refresh that prediction on the hour every hour  with new information that we have. Tony Barrueta: That's remarkable. Okay. What else? Vivi

an Tan: So, you know, there are so many things. So,  we have about 300 deep learning models in production and 3,000 machine  learning models in production at KP. Tony Barrueta: Okay. Tell me what this means. Vivian Tan: Segmenting everything from -- with breast cancer screening and doing  segmentation there, predicting high risk for suicide in certain populations. We have things  that predict COVID for a long time. We have things that are really looking at -- we also  are using ambient AI and th

ings to support documentation and reducing clinical workload. So,  there are so many different experiments going on in different regions. I feel like Dr. Ainsley,  is she in the audience? She is leading a lot of amazing work in the area of imaging. And I  know she's going to come on in the afternoon. And hopefully, she can highlight some of the  innovations going on in that space as well. Tony Barrueta: So, I think in trying -- a lot of this is trying to figure out, okay,  we've been using algor

ithmic processing for a long time. What actually -- what's different  now? So, I mean, we've -- for many years, we've been sort of the early flu warning  system for the state of California and through that data and we do reporting. What's  probably advanced now beyond what it used to be? What's the difference between what was  happening before and what's happening now? Vivian Tan: Yeah. So, I think there are some things that haven't changed.  We're still doing value-based care. That is, you know

, kind of enabled by data and  analytics. I think with generative AI, the difference is the speed and the scale. And  I think it's both a good thing. And there are probably things that come with it. And I  know in the afternoon, Dr. Yang is going to -- where's Dr. Yang? He's going to lead us in  a conversation about benefits and risks of AI. And you know, I think with generative AI,  there's like half glass full and half glass empty. I think the big difference is the  speed and the scale. Becaus

e suddenly, I have not seen this industry and  this space move this quickly -- Tony Barrueta: Right. Vivian Tan: -- within such a short period of time. And the scale in which things can be  done actually is, you know, sort of in the order of magnitude. And the amount of data that's  being processed is mind-boggling, to be frank. Tony Barrueta: Okay. So, when I first started reading about AI six or seven years ago, it was like  the impending apocalypse and all of that stuff. [laughter] So, there

are things to worry  about. I don't think that is the thing that's first to worry about  in this context, in healthcare. So, what do you worry about that's in  the kind of shorter period of time? Vivian Tan: Yeah, I think for healthcare, you know, the bar is higher for healthcare, and actually, rightly so.  The bar is higher from a data privacy perspective. The bar is higher from a data security  perspective. The bar is higher from an algorithmic perspective. So, we have to actually, you know, 

both govern and regulate ourselves in, you know, to the highest standard. And then we also have  to work with regulators and policymakers, all of you in this room, to ensure that we have the right  public, private, you know, collaboration as well. So, you know, I don't know if it's a worry,  but I do hold it as a key responsibility and obligation. Because you know, healthcare  data is sensitive. The application of that data is sensitive too. And actually, with  the executive orders and stuff, yo

u know, it's also very clear where regulation is going,  where we have to ensure there's transparency. And we have to show that our data and models  are not biased. And we have to explain it. So, they are, I think, a lot of things that are  actually exciting as well about this space. Tony Barrueta: So, and I know we'll have a lot of discussion of this later  today. But you know, policymakers and people who are observing this are really focusing  on the equity issue. And I think the concern, you

know, the immediate concern that we see,  whether it's with the Federal Trade Commission or others looking at it, is will the use  of AI exacerbate existing equity issues, or -- which is certainly possible. Is there an  opportunity to actually make progress on some of the problems with health equity that we haven't  been able to make as much progress on them? Vivian Tan: Yes, I'm really excited about that. And I see KP being a leader  because we've taken really very intentional steps to create t

he right data that is not biased  and is very inclusive and equitable. So, I'll share an example from World War II because  you're a history buff here. So, [laughs] I'll do an example. So, in World War II, they had all  these planes. And they were trying to figure out what part of the plane they had to reinforce,  you know, to prevent it from being shot down, right? So, they started collecting data on  the planes that were coming back, and they tried to figure out where the holes were, right?  B

ut there was a huge flaw in that data because the plane -- some planes weren't coming back at  all, right? And the places where they were most vulnerable was actually the engine, not the wings,  which, you know, which the data was showing. Tony Barrueta: Right. Vivian Tan: So, one of the key things we really have to do with health equity data is to  make sure that the data is correct. The data is robust. It's not biased. It's inclusive. It's  equitable. Because with AI, the data actually drives

the model. So, if the data is wrong  or not complete, the model is not good. So, one of the things that we're really doing is when  we do any work now, we actually look through the lens of health equity. We actually check if the  model performs well when we look at subsegments by race and ethnicity. And we're very thoughtful,  you know, in terms of what models we use if it doesn't, you know, and we actually don't use  those models if it doesn't prove to be equitable. Tony Barrueta: Wow. So, you'

re fortunately -- oh, yes. We need to get --  getting a few questions from the audience. So, if there are questions -- please help me ask these  questions because as you can tell, mine are not as sophisticated as yours are likely to be. So,  please, think about some questions. I was -- you know, I think there's so many different types  of opportunities that are in front of us. First, I'm really pleased that you're in the spot  where you are to help us think this through. Where do you think other

organizations will  be learning from what we're doing up front? What opportunities do others have that maybe  they've a greater opportunity than we do in terms of the data they may have access  to or their models differing? I mean, maybe the other way to ask is, is  there something about our model that makes this a particular opportunity  or a particular hindrance to doing? Vivian Tan: Yeah. So, I'll say a few things. So, the first  is because KP is an integrated system, our data sources and wh

at we call data  domains are very rich and deep. So, we have our health plan data. We have clinical  data. We have consumer data. We have real-time data. We have retrospective data. So, one of the  things is the breadth and the depth of our data. And actually, we've got data because we've been in  existence for how long now, 77, 78 years. We have longitudinal data. And in the area of genetics,  we have family history from cradle to grave, right? And we have it for generations. So, our  data is -

- oh, and then we have a tumor bank with, I don't know, now like 60,000 different samples.  So, we have such rich data that it's really amazing to be able to look at all those assets and  figure out how to leverage it for the, you know, benefit of our members and patients. So, that's  one, I think, unique differentiator for us. The other is that actually we have done  a lot in partnership with other systems around interoperability and data exchange. And  we've also -- I wouldn't say we're unique

here, but we've learned a lot. And we  are hoping that we continue to learn and share and collaborate with  many health systems across the US. Tony Barrueta: Any questions? Yes. Kathy Curran: First, thanks for putting the session on. I don't know if people  can hear me. Thank you for putting this session on. I'm Kathy Curran with the Catholic Health  Association. It's a really important topic and really important work. And I wanted to  follow up on your comment about, you know, making sure we g

et the AI usage right from an  equity perspective. But how do you do that? Because the image that comes in my mind is if  you don't know that you don't know how to spell, then the dictionary is no good to you because  you don't know to look in the dictionary. So, how do you -- like what are some practical  examples of how we can know what we don't know when these applications are being developed  and the data sources are being chosen? Vivian Tan: Yeah, that is such a good question. And at KP, we

have a very practical seven-step approach which  starts with the framing of the question. You know, there's a lot that you can do to ensure that  your question is not biased, right, and you frame it in a way that doesn't introduce bias. The  second which I talked about was the data piece. It's so important to step back and investigate if  there's something critical that's missing in your data set. Whether it's, you know, whether it's an  actual, you know, segment of the population, or whether i

t's a dimension of feature or labeling of  the data that you need to really concentrate on. And this is where we have taken a very  multidisciplinary approach where -- and we really go and work with the subject matter experts.  The people that are closest to the population, the people that are doing the work day to day,  the people that understand the data, you know, in a great depth, right? So -- and we look at  it from multiple kind of places. And finally, when the model is built, there are a 

lot of techniques to actually check if the model is working well. And  you can actually explain it. So, whether it's looking at counterfactuals,  [unintelligible], attention models, there are ways to actually investigate the  model to make sure that the model is working. And the final thing I would say  is actually, most of the time, [laughs] when you first build your model with the  data, the model is actually pretty good. But what happens over time is there is this thing  called model drift.

As your data changes, as you get new data in, your model doesn't  get as good at predicting whatever outcome or, you know, prescribing or making  a recommendation on something. So, managing model drift with  continuous model management is critical. I could not stress  this over and over again because, you know, people spend all their time doing the  upfront work. And then once it's in production, I think there's a little bit of a culture of,  you know, forget, right? So, you put it in and then y

ou forget about it, where that  continuous management of it is critical. Tony Barrueta: So, we got to keep -- so, there's actually a critical human role in  making sure that this doesn't go off the rails. Vivian Tan: Absolutely. Actually, maybe if there's a chance to put the next slide on. I  try to keep things very simple. So, this is my ABCD chart of the ingredients it takes to actually  have AI at scale probably in any organization. So, first, you -- and I'm not going to go in any  sequence n

ecessarily. But you have to first have, you know, infrastructure, right, in terms of cloud  infrastructure. And I know we use Microsoft Azure for a lot of our data and analytics. And we have a  leader from that organization here with us today. But the infrastructure without the talent is also  not helpful. So, you know, you have to actually hire and retain very good talent. And then with  the data that we have curated, which we have a lot of in this organization. And then applying  the AI techni

ques that I talked about. You know, those are the ingredients of success, if you will.  There are probably other things. But you know, these are sort of the basic four ingredients  that go in to help an organization scale. Tony Barrueta: Awesome. Well, Vivian, as always, I learned a ton whenever  I talked to you. And fortunately, you're here all day. So, people have an opportunity  to buttonhole her as the day goes on. So, let me ask you all to join me in thanking  Vivian for starting out this m

orning. [applause] Vivian Tan: Thank you. Thanks for having me. Rebecca Flournoy: All right. Thank you so much, Tony and Vivian, for all those great examples and  making it very real. I really love that session. And now, I am very excited to welcome our first  full panel to the stage. This next conversation on the future of AI and healthcare will be  moderated by Dr. Ainsley MacLean, who is chief medical information officer and chief AI officer  for the Mid-Atlantic Permanente Medical Group. Dr.

MacLean oversees a team of software  engineers, physician informaticists, and clinical analysts in the design  and implementation of technology, AI, and telehealth enhancements. She is the  founder and co-executive sponsor of the Permanente Medicine Artificial Intelligence  Center of Excellence. And she is certified by and on the board for the American Board of AI  and Medicine. She is also an associate medical director overseeing radiology teams and an  ultrasound program that spans 20 differe

nt specialties. Dr. MacLean, thank you so much  for joining us, and I will turn it over to you. [applause] Ainsley MacLean: Yeah. Thank you so much. Thank you. Well, good morning, everyone.  It's so wonderful to be here. I really want to thank the KP Institute for Policy for putting  on this conversation. At this moment in time, we're at such a point of excitement looking  forward, and conversations like this are critical in ensuring that everything is done  well for our patients. So, thank you

so much. So, I'm going to introduce our first  panelist. And I'll let you know a little bit about how this is going to work.  So, each of our panelists who are amazing, are going to actually give some brief comments and  an overview of their opinions on what the future of AI and healthcare holds. And then we'll  sit up here together and we'll have a little conversation. And then we'll have an opportunity  for some of you to ask questions as well. So, I hope it's really engaging. And again, we're

going  to be now moving from the present to the future. So, our first panelist is Dr. Junaid Bajwa.  He's the chief medical scientist for Microsoft Research, and he's going to be talking about an  exercise of our imagination. Our next panelist is Dr. Julia Adler- Milstein. She's the chief of  the Division of Clinical Informatics and Digital Transformation and director for the Center for  Clinical Informatics and Improvement Research at UCSF. And she's going to be talking about AI  at UCSF Healt

h. And our final panelist is Dr. Judy Gichoya, who is an associate professor of  radiology and informatics and an interventional radiologist at Emory. She's going to be talking  about the future of AI in healthcare, capacity building for an AI-enabled future. So, please  join me in welcoming our panelists to the stage. [applause] Junaid Bajwa: I might stand here if that's okay. I'm tall enough as it is. Good morning, everybody.  My name is Junaid Bajwa. I'm the chief medical scientist at Microso

ft Research. I'm a practicing  physician in the U.K.'s National Health Service. And I'm delighted to be here. My sincere thanks to  the KP team for the kind introduction. Feel free to connect with me on your social poison of choice  if that could be helpful. Very happy to engage in conversations. Before I start, we've spoken a  little bit about these new tools. And without making any judgments, how many of you have used  any of these tools in the last year? Just a show of hands. Who hasn't ever

used any of these? Okay,  so a few. How many of you have used them for work? Okay. Interesting. Okay, fine. That's a  nice baseline for me to start with. So, one of my challenges was to think about actually,  how do we just ground everybody in what we potentially will begin to discuss a little bit  about today. So, I spent most of the last 20-odd years across the payer-provider, regulatory, life  sciences, and now big tech side of healthcare. So, one might argue that I just never stop doing  rot

ations. And the other argument is I just wanted to make my mom really proud and make sure  that I do lots of different things with my life. So, I sit on the board of the MHRA, which is the  U.K.'s equivalent of the FDA. I'm on the board of a large academic medical center in the U.K. known  as UCLH. And the thing that really drives me is impact at scale. And really thinking through  not just health equity but population health, reducing the cost of healthcare, but also  improving the experience o

f care for those on the receiving side of healthcare, but also  on the delivery side of healthcare, too. And Satya Nadella a few years ago, who's the  CEO of Microsoft, said that AI is technology's most important priority and healthcare its  most urgent application. I think the context for this is really some of the supply and  demand challenges that we have globally. So, we have an increasing aging population  with increasing complex comorbidities, and we have a massive workforce  crisis in mos

t parts of the world. The World Health Organization, pre-pandemic,  estimated that the world will have 14 million less doctors, nurses, and pharmacists on the  planet than what society would need. I would argue that post-pandemic, we're closer to about 25  million less doctors, nurses, and pharmacists by 2030 than what society will need. So, within six  years, we have a massive challenge around who will be delivering care for this aging population with  complex co-morbid needs. And I teach at a

range of institutions. So, in the Boston ecosystem in  the West Coast as well. And this kind of role of technology, if you kind of take this graph on  the left-hand side -- and I apologize for the font size being very tiny -- just shows the rise  of technology across all industries over time. And on the right-hand side, something that Vivian  was referencing around the size of models that now exist and the scale that exists to inform these  models. So, thinking about data, compute power, and the

parameters. And many of you would have  heard of GPT or other similar models. And we'll talk a little bit about what they might mean in  a moment. But what's really interesting is how they're now democratized. Everybody has access to  them. Many of you in this room have raised your hand, may have played with them to say, write  me a poem, or help me make a travel itinerary. And it's almost quite magical when you  start looking at these and you start to interact with these tools in novel ways. B

ut  what we're seeing is language understanding, reading comprehension, image recognition,  speech recognition, handwriting recognition, surpassing human performance now  around kind of the 2020 mark. And it's probably best if we demonstrate  this through a series of some examples. So, if you were to go to Copilot today and type in  something like this. So, what is metformin? Many of you will probably know what metformin is.  But the response it will give you is this. And it will tell you that m

etformin is a prescription  medication to treat type 2 diabetes and provide you some additional data. If you carry on the  conversation underneath to say, can anyone with type 2 diabetes take it? It will start to  tell you about who can and who can't take it. So again, this is just something that's freely  available today. Any citizen in the U.K., in the U.S., or otherwise can begin to play  with these things. So, it's actually getting quite complicated in terms of the information it's  providin

g. If you carry on the conversation even further to say, well, actually, are there  alternatives to metformin because actually, I'm one of those people that have a liver disease  or kidney disease? It'll give you even more information around what the possible alternatives  might be for you. And many people in this room who treat patients with diabetes will probably  understand that. Actually, that's relatively reasonable in terms of what it's providing.  But all of that can be massively complica

ted. And you might say, actually, well, that  seems massively complicated. I just don't understand what to do. And not only  will it try and empathize with you, but it will recognize that you feel overwhelmed  in your language. It will recognize that you feel overwhelmed and then ask you to  think about a holistic way and ensure that you engage with your holistic, healthcare  professional team in tailoring the need to you. So, it's safety nets, if you will. It will  reason behind your words as y

ou carry on the conversation. Out of curiosity, because  many of you are kind of on the policy side, what's the average health literacy,  the average health reading age within the U.S. or the U.K.? Give me a  number. Shout as loud as you can. Female Speaker: Sixth grade. Junaid Bajwa: Sixth grade. So, how old is that? Female Speaker: 12. Junaid Bajwa: 12. When you think about your medical communication, how much  of it is targeted to a 12-year-old? It's an interesting question, right?  And I ask

that because actually, the first time I ever played with this  was when my 12-year-old came home with the geography homework about 22 months ago. So,  I had an early access to some of these tools. And his homework was, write a 500-word  essay on the microclimate of the school, comparing the field to the playground and the car  park of the school in London. And being the heroic father that I am, I could have chosen to go to  a search engine of choice, type in microclimate, then understand actual

ly is a microclimate  different for grass versus mud versus tarmac. Being very lazy, I wrote down my prompt which was  write me a 500-word essay on the microclimate of a school in northwest London, breaking it down  into introduction, methodology, and conclusion, but put it in the voice of a 12-year-old. And  when it put it in the voice of a 12-year-old, not only could I understand it  and explain it back to my son, but actually I could have a better conversation  with my son. I never gave him t

he answer because that would be very bad parenting, but  I could explain it to him on his terms. [laughter] And so, what I've now done in my own clinic  is to challenge myself to say if I'm having conversations with patients about diabetes or  heart failure, how do I meet them on their terms and have better conversations? And when I'm  at the MHRA, and we look at the risk alerts that we get for drugs, often 15 scrolls worth  of information, might we be able to actually communicate that better? A

nd these tools are  fantastic at summarization and communication. Many of us who have done medical school  or nursing school, we often learn through clinical vignettes. So, if you take a clinical  vignette like this, which is from the USMLE, talking about a 12-year-old girl with a  set of symptoms and some early diagnostics, we're often given a multiple-choice  question. You could take this entire prompt, put it into Copilot today, and it will give you  the answer. It'll tell you what the answer

is, and it will rationalize to you  why it is that answer as well. So, it's educating. Not only giving you the answer,  but you can prompt it to then educate you as to why you think that answer may well be true,  and why it possibly can't be any of the others. But remember, that's just a clinical vignette  talking about a 12-year-old girl with a set of symptoms. What if we push the model a little  bit and remind ourselves that actually, each of these models have not been trained on  any medical

knowledge. They've been trained on the digital exhaust of the internet, and they  have begun to reason in the background. So, the same model is passing the bar  exam, passing USMLE examinations, passing MBA exams in the background,  and no specialized training. Imagine if we said to it, what do you think  the girl in this problem might be thinking and feeling? That requires a significant  level of reasoning underneath. And these have not typically been the types of models  that we would have be

en able to have access to in the past. But it will give you a very  reasonable answer. As an AI model, I cannot directly assess a person's thoughts and feelings,  but I'll make an educated guess. The 12-year-old girl in this scenario might be feeling worried and  scared. She may be concerned about missing school, and it's essential for the healthcare team to  address her concerns and provide reassurance. There's been some papers that have been published  over the last year around the empathic re

sponse that these models are giving, arguably more  empathic than some of the practitioners that practice medicine today. Let's go a little bit  further even than that. So, the girl's name is Meg. If you were Meg's doctor, what would you  say in order to provide comfort and support? And it will give you a very, very reasonable  script to go in. And when in this -- certainly, in the U.K., and I'm conscious in the U.S.  too, we have lots of role play that takes place. And that role play can be ver

bal, but  you could end up having chat-based roleplay. And if you extend this out and you play with this  role play, you could ask the model to assess how well you have done versus other benchmarks. And  you could say, "I am training to be a doctor. Please role-play the role of Meg. I will be  the doctor, and let's have a conversation on it." And so, you'll see how these models are  very, very different to any types of technology that we've used in the past and before. And  again, freely accessi

ble, and available. What I would say -- and there was one little  bubble in Vivian's chart, which is really, really important, is thinking about how we  deploy these tools in a responsible way. So, the exam question that I focused on at Microsoft  is how might we transform the practice of medicine with trusted, reliable, human-centered AI? And I  say that very deliberately. It's not just about AI. In healthcare, it must be trusted. It has  to be reliable, and it has to be human-centered. And the

re are various tests that you have  to pass for trust, tests that you have to pass for reliability. And when we're building  models, we think about each of these things. So, on a bedrock of transparency and  accountability, assessing for fairness, reliability and safety, privacy and  security, and inclusiveness. Most life science trials that exist on the planet are not  representative of people that look like me. I'm a 6'4" South Asian guy from northwest London.  Most trials that exist for chole

sterol drugs, diabetes drugs are not for me. Therefore,  generally, 40-year-old white men, they're not designed for people like us. We  already have a bias in our healthcare system. I think that if we were to have a conversation  about the bias, challenge ourselves, and move that we are pushing for heterogeneity of  data in the development of our models, thinking very carefully about how we deploy  these; thinking about the end user impact; and push ourselves to be much more inclusive  around da

ta, where I think organizations like Kaiser can really lead the way. How  might we be better at addressing some of these things? This will not be a magic  bullet, but we will get there over time. And if we can identify issues with the outputs,  measure them, and mitigate them will be way, way better off. Many of you have  seen blood test forms like this. These models are pretty smart. These  models can take your blood test form, and you can ask it a question and it will  give you a response. Say

, "In simple terms, explain this pathology result to me."  Saves massive amounts of time certainly in my primary care clinic where I have to do  42 patients a day, 10-minute appointments each. And what you'll begin to see is what we didn't  do at the beginning, but we do a lot more now is ground the answers with reference points  within the internet. So, each of the references you can see underneath, and you can go back and  validate against them to reduce the hallucinations and confabulations t

hat many of you may have  heard of in the past. A lot of that is what's happening on the tech space. There's research  saying, how might we be able to actually begin to engage with images in a different way?  So, what can we do with two-dimensional and three-dimensional images? And I'm sure  colleagues will talk in depth about these. But potentially having a conversation with  the images, if you're in an ER or in an HD or an ITU setting, and you can't wait for  the radiologist to give you the op

inion and the review, could you figure out if there's  a pneumothorax here? Could you figure out whether the tube has been placed in the right  place? Can you figure out if there's a temporal dimension to the tumor? And it's changing  over time. And we're getting better at that. In the U.K., this is Dr. Rajendra [spelled  phonetically] on the left-hand side. We engage in a seven-year piece of research with him to think  about prostate cancers and radiation oncology, radiotherapy treatment for pr

ostate cancer.  It typically takes him from 30 minutes to 3 hours to contour through every single CT  scan, identify a tumor versus nontumor. So, in a typical day, he could plan six  patients' worth of radiotherapy. We've applied open-source AI models to that. That  takes now 13 minutes. So, six patients an hour can now have radiotherapy planning. Which  means how many more patients will now have potential access to radiotherapy planning for  the future? And it's an open-source model. He's now a

dapting that to use it for glioblastomas,  head and neck cancers, lung tumors, and others. So, we live in a world with  massive amounts of information, increasing complexity. We also have a workforce  challenge around supply and demand. I think that the doctors of the future and the nurses of  the future and the pharmacists of the future, the healthcare professionals of the future  that have access to these models and work with these models hand in hand, may  be safer practitioners than perhaps

practitioners that don't use them in the  future is what I would put it to you. I have a very short video. It's like 45 seconds.  Would that be okay to play? So, I'm not sure if this is going to work as well, but we'll give  it a go. There's a short video on the left-hand side. And I'll try and talk through if it doesn't  work out as best as I wanted to. Can we try? Oh, here we go. So, this is the model. This is  demonstrating that these models can "see and hear." So, this is a model demonstrati

ng  -- yeah, demo what you can do with a bike. So, this individual is trying to say, "Help  me change my bike seat." Move up and down a little bit. And it'll give you the instructions of  what you need to do just based on the image. So, recognize the bike and it recognizes your  challenge. And it says to you that "If you can give me a more specific picture and give me  the manual, I'll help you out even further." And you might say, "Is this the lever?" It'll say,  "No, that's not the lever. That

's the bolt." And it will say, "Help me identify what you need to  do next." So, you take a picture of your toolbox, your Allen keys. You take the manual. And it will  identify where in the toolbox you need to go, which is the Allen key that you need  to use, and what you need to do next. And it does this without any specialized training.  I show this because actually, these are tools that will be ubiquitous in our day-to-day lives. They  are not specific to healthcare only. These tools are now

democratized around us. It's as if we're  living in the iPhone moment, the internet moment. And the question for us, I think, is how do we  responsibly deploy these? How do we responsibly adopt these and really think about the change  management that needs to take place for us to leverage these in a responsible, equitable  manner for the future? Thank you very, very much. [applause] Julia Adler-Milstein: Thank you. Thank you so much for having me. Two terrific sessions already,  and I'm excited

to continue. I will say that I also have a 12-year-old. And so, I think any  chat GPT output that doesn't include the use of the word "bruh" with at least five years  is definitely not doing its job and will need some fine-tuning. So, that's the easy case where  you can easily verify the quality of the output. What I will be talking about, I think,  unfortunately, is that the pathway to the future does involve some more complexity  that is really happening on the front lines of healthcare delive

ry organizations that  are having to make the decisions of are these tools ready for prime time, and what  does prime time mean in terms of safety, effectiveness, equity, et cetera? So,  I think you're already seeing a theme of talking about the future really is about  talking about where we are in the present. And I'm going to sort of double-click down  and give you a perspective from our health system. What does this actually  look like on the front lines in terms of how we are approaching  th

e challenge of where the tools are today, and whether they are ready for  wide-scale use in our patient population? So, where we started this journey  was really the need for a vision, right? What are we trying to do with  these tools? We don't want to just implement technology for technology's sake. We  really want to ensure that they are solving our real-world problems and doing so in ways that  are trustworthy, which again, is, you know, a word that we all kind of intuitively understand.  But

we have to operationalize that. What does it mean to be trustworthy? And how do we develop  processes that ensure that the AI does meet principles of trustworthiness? That includes  some of the things that you've already heard about today in terms of longitudinal AI and  impact monitoring. So, I'll talk about that. And the other piece is AI at scale that  it's really easy to say we have, you know, 2,000 models implemented. What we're really trying  to do is to say we don't want 2,000 AI pilots.

We really want to figure out which are the tools  that are most valuable and get them to enterprise scale so that they can impact our entire patient  population. So, we are thinking a lot about how do you start with a pilot, rapidly determine whether  it's doing what it's supposed to do, and then have a plan for scaling that so it does get to the  enterprise level. So, those are really the two dimensions that we are emphasizing trustworthiness  and ability for the tools to get to scale. So, we

think a lot about this three-horizons  framework in terms of whether we are using our resources around deploying AI effectively. I  really like this because I think it's intuitive to think about what is your mature business,  what is your rapidly growing near-tum business, and what is an emerging future business  opportunity. Thinking about gas-powered cars, electric-powered cars, and the future of  autonomous vehicles. And the point is, we need to be investing on all three horizons,  right? We

need to make gas-powered cars better. We need to make electric vehicles better. And  we need to be designing for this future state. So, what does this mean for us in our  health system? It means that right now, we are just trying to make our current  fee-for-service healthcare system better by, you know, implementing AI for operational use  cases. Again, you heard about some of them already this morning. At the same time, we're thinking  about our rapidly growing business. How do we think about

generative AI for augmenting clinician  intelligence and really bringing together the best of human and computer intelligence as  that exists today and in the near future? And at the same time, we can start to  see a model for AI-driven virtual care, right, where we can get a sense for what might  happen with a given patient trajectory. Start to anticipate some of the lab tests they may need  and send them for lab tests before they even come and see a physician. So, such that interaction  is rea

lly fully optimized even before the patient touches our system for a particular issue. And  again, there's probably even more opportunities for that future state that we're just still  trying to get a handle on. So, we are, again, trying to design strategies that will allow  us to invest in AI on all three-time horizons. It takes a team to do this work well. I don't  think we have the answer. Is this the right team? Are we the right expertise? Is it the  right number? We are really all learning

this on the fly as we do it. At the heart of our  operation is our chief health AI officer. I think you see a growing number of health  systems that have these roles. They are the sort of data science and technical team, as  well as people with expertise in different sort of domains of clinical informatics. And they  oversee our governance process, our technology, and then they're really supported by people in  these other roles. So, my division is really the group that's being brought in to thi

nk about  evaluation and impact and measures of impact. And then we have our sort of clinical systems  health IT team. We have people who are really thinking on sort of that third horizon in  terms of like the research capabilities, doing more of the precision medicine work. And  we have a newly named chief research information officer that's, again, really pushing us  to that third horizon. While my group, even though we do research, is a little bit more,  you know, current state. And then of c

ourse, all the IT infrastructure and security and  our chief data officer that's really trying to work to ensure that we have that solid data  foundation that you already heard about today. So, I think a lot of what we're going  to learn about how to do this work well is who is the team that needs to be at the  table? How do you think about the expertise, how they work together? What are  the structures and functions? Our evolution over time has really started with,  like, what is the technical

infrastructure that we need to be able to rapidly  deploy different types of AI models? We're an epic shop. And so, we started  with Epic's cognitive computing platform and pretty quickly realized that it was not  going to be sufficient to meet the needs of AI at an academic medical center. So, we built  basically our own platform that we call HIPAC, that allows a much broader set of data  to feed into models in real-time. You can again see sort of the basic architecture  of it and how it integr

ates with Epic, but also has some unique features that, again,  allow us to deploy and monitor our models. Where are we today? We, frankly, even though I  think have been at this a while, we are not in the thousands of models deployed. If we look at  sort of where we are at an enterprise level scale, we're still in the early days. And we are doing  sort of a mix of models that are coming from our electronic health record vendor and some models  that we have homegrown or self-developed either by

people within our health system or by  researchers. A lot of these are really still focused on those sort of operational use  cases, things like the capacity management that you heard about this morning, you know,  predicting the use of blood products. So again, sort of clinically adjacent and relevant,  but they're not predicting diagnoses or sort of directing patients to, you know,  to certain parts of our health system yet. We're always evaluating new models, again,  mostly driven by the need

s of our health system, but with some capacity for what our researchers  and frontline clinical faculty are interested in. We have evaluated and turned off models  that we thought were problematic and have written about sort of that process of trying to  understand enough about some of the models that were available to us for us to determine whether  they met our threshold for sort of being ethical. And some of them, we felt like they just weren't  at our institution's ethical benchmark for sort

of equity of deployment. Again, it's a longer  story, and happy to talk about it at lunch. And then again, a lot of models are coming through our  pipeline. So, again, just to give you a real-world sense of sort of, you know, where we are in terms  of number of models that we're able to look at. We also have now GPT integrated, a secure  -- fully secure version. So, no data goes to GPT. So, it's sort of our internal product that's  integrated with HIPAC that I described before. So, what that me

ans is that if you want to use  a generative AI model as part of the model that you're deploying, it's sort of fully  integrated with the full architecture that then flows down to our EHR. Right now, we  mostly have GPT-4, but are anticipating adding sort of [unintelligible], and you  know, we ideally want to move, frankly, to more open-source models that we can, you  know, work with in a broader set of ways. If you look at, you know, the breadth of  applications of healthcare that we hope to to

uch with AI, it really is, you know,  moving from that horizon 1 to the horizon 2 framework. We're having a really interesting  meeting in a couple of weeks about how should AI support our medical education mission, right?  So, should we have virtual AI coaches for all of our trainees so that as they are going through  the process of training -- again, this is across a breadth of different trainee types, they can  have real-time feedback on their performance. Right now, it's very labor intensive

. They tell  us that they feel like the feedback is delayed, and they don't get enough of it. So, is  AI our solution there? So again, maybe, you know, six months from now, we'll be able  to tell you where we landed there. But again, just a fascinating breadth of conversations. So, we are evaluating use cases across all these  different domains. Again, we talked about AI Ambient Scribes. How can we triage high-risk  patients? Anyway, you can see down the list. I just want to keep going to get to

this last  section on this issue of the day around sort of AI fairness. There was a question that  came in. I saw an advance of sort of what can happen at the state level. And I  will say that a lot of our work got kicked off by an inquiry that came from our  California attorney general that asked us, how are you assessing your racial and ethnic bias  in the algorithms you have deployed? And we said, "Well, give us a few months and we will let  you know." And really, what we realized is that we

had to start with even what do we have  deployed? At that point in time, we didn't even have a sort of centralized way to know all the  different models that existed at UCSF Health. And so, we had to sort of come back  to centralize that function and then stand up this whole governance process. So,  I do think even just the inquiry can be very powerful at the state level to know that  someone's looking and wants to know what you're doing before we even get into sort of how  do you regulate this

and all of that complexity. So, as I said, we have adopted a set of  trustworthy AI guidelines. We decided to adopt the HHS guidelines because we think that they  really do emphasize the key dimensions. Again, so much complexity in how you operationalize  these trade-offs between different dimensions. I don't know that we've got it right,  but these are the conversations we need to be having amongst health systems to  understand how they're thinking about this. What we have stood up is what we

call our AI  Governance Committee. They are the final sort of gatekeeper for what models get deployed to  our patients. And so, I sit on that committee, a bunch of different people with different  expertise. And the models -- every model, whether that's research or an operational model  or an EHR vendor model -- come through the same governance process. And we sit down,  and we basically look at the models. We start with just discovery. Like, is this model  trying to solve a real problem? Have t

hey thought about the entirety of the solution? Not just  the model has a high-predictive value but like, do they understand how to integrate  it into workflow? If we implement it, are we sure that the interventions that will  come because of it are equitable? So, this whole notion that hasn't come up yet today of,  you know, patient-positive interventions, right? So, if we're going to predict a patient  no-show, you can either double-book that slot, right -- which is negative [laughs] for the 

patient and if you have an already biased model, it's going to further worsen disparities -- or you  could say, "If a patient is predicted not to show, that we will provide transportation," right?  And that's a patient-positive intervention. So, we're really trying from the start to make  sure that we're thinking about implementation of these models holistically and sort of how  humans will use them and interact with them. We then do the development and evaluation,  retrospective evaluation, pro

spective evaluation, moving into pilot or RCT, and then the  adoption and ongoing monitoring. And what's important about this is it means we're touching  these models many, many different times. So, this is a huge amount of work just to even  take one model through this process. And so, what we're trying to figure out now is how do we  resource this if we want to be able to put 10, 20, hundreds of models through this process,  right? It's a huge amount of investment. Anyway, you know, lots of ch

allenges to doing  this. I think we're still learning how to do it well. It's something we're trying to do in  collaboration with a broad set of partners. So, if these names aren't familiar to you, I do  recommend familiarizing yourself with these groups of organizations that are trying to  come together and develop best practices. And as we head towards the end, I'll just say  that, you know, I keep coming back to making sure that the models are good and making sure that the  humans are good at

using the models. And we really have to think about those two pieces together. So,  yes, we have to think about algorithmic vigilance, algorithmic drift, but we also have  to think about clinician vigilance. Is it really realistic to think that if  a model gives a clinician bad output, that a human's going to be able to recognize that  and capture it and prevent that from getting to the patient? So, we're really trying to  bring these two together and think about measures and methods for both a

lgorithmic  vigilance and clinician vigilance. So, happy to talk more about our strategy  in the break, but we'll pass it off. [applause] Judy Gichoya: Good morning, everyone. So, nice segue to talk  about the future looking at capacity building, especially with clinical vigilance, a lot of  pitfalls. Very tough on me because, I say, I work in four areas. I'm an associate professor  of Radiology and Informatics at Emory University. And one of them is bias and fairness, which I'm  very passionate

about. And I'm like "Well, maybe I should have talked about my work about that,  or the second one, which is capacity building." And so, here are my disclosures. So,  2016, this one godfather of AI said we should stop training radiologists. And this  had actually an impact. A lot of students became less interested in pursuing  Radiology as a specialty. Thankfully, you know, it's 2024. There's not anyone  who has enough radiologists to work for them. And he actually said he's leaving Google  too

because of these existential risks of AI. I show you this just because we  have a lot of hype even now, again, with the large language models that have  an impact. They have an impact early from training. You do have to really invest a lot  in making sure that the right message gets out, which is actually getting a little more  difficult. But I want to show you why doing this work -- again, I'm just following around  the same theme that it's very, very difficult. So, I hope some of you or most

of you have  heard of these two studies, one that shows pulse oximeter doesn't work well for dark-skinned  patients, and also the temporal thermometers also don't work as well. And, you know, some of  the people who wrote this work are actually very good friends of mine. We do this health  equity research using data science together. And I asked them "How has your practice  changed?" Nothing. You know, they know it doesn't work -- these researchers in this space.  We know the problem of hidden h

ypoxemia where you can send patients who really don't have enough  oxygen just because of the tools you're measuring. And it's really difficult to translate to  practice. And so, it's that theme that I'm going to use to inform sort of like what we -- what  should we be thinking about the future for us? Well, here's an example from a mammography  experimental test. What they did is that they had 50 mammograms, and they -- you  know, they changed the output of, you know, some of the mammograms. An

d then they give  them to read as of different experience. And it turns out that when AI is correct,  this -- unfortunately, if you're colorblind, it's going to be the last bar there. But the red  bar is your least -- your inexperienced reader. So, if you give a correct AI output to  your inexperienced reader, you improve their performance. And, you know, you raise them  up to almost the expert level. But if you give the wrong explanation -- in mammograms, you can  either say the patient has can

cer, or they really don't have cancer, and you say they have cancer;  or the other way around, that they have cancer, but you say they don't have cancer. And it shows  that you even bias your expert performance. And this sort of -- one of the big futures is  understanding how does AI and the human -- in case we're not anticipating replacement  -- hold hands together for an impact, remains one of the biggest challenges. And  here's an example in psychology. They are able to get through a beat for

these sort of  experiments where they can lie to the reader. And one of them here is just they run  three experiments with an imaginary use case that is saying one patient has disease,  one doesn't have disease. And what they show is that when you -- one of the experiments  always has an error, a systematic error. Forty percent are always incorrectly classified.  And they show, when you give AI assistance, the error rate goes up rather than when  you just let the human work by themselves. And t

he second phase, they start off, you know,  giving one group -- the blue one has an error and -- has -- uses AI, and the down one doesn't.  And it shows that you start off using AI and then you don't. And those participants showed that  when you use AI, your error rate is high. When you stop, it goes down. And then they flipped the  experiment. For the people who did not use AI, they used the AI in the second phase.  And they showed that the errors persist. I think this study -- it wasn't really

  saying, "Hey, it's pneumothorax," but to me, it was very concerning because it shows  that the automation bias can persist even when the AI is withdrawn. We've seen sort of a  system that has actually pulled out AI. And so, to me, again, we -- you know, I had  to still sneak in a bias slide here. But it shows that we have a very, very complex  system, as we start to think about what the future deployment of these systems is going to  be. You have to decide, are you going to think about how you

deploy, or are you going to think  about how you build? We had some comments that, you know, "We are building a fair dataset." I  can tell you that that's impossible, you know, the model evolution or how you deploy your  system across the complexities of organizations. So, my talk is around capacity building  I've been involved in many, many years. The first time when Geoffrey Hinton caused  a drop in the radiologist's interest in AI, I went back and did a lot of outreach to try  to think, how

do we educate? Today, I run the AI elective at Emory University. And what I can  tell you is that this field is changing so fast. Today, when I look at my computer scientist versus  two years ago, they rarely rely on the computer scientist when its residents because they have  copilot that helps them even program. So, domain expertise is becoming very important. And so,  I think one of the biggest takeaways if I think about the future, even if I don't have a crystal  ball, is to think about whos

e voices are in the room. And to me, that's what policy is, as someone  who's just learning about how to be in this space. And if you think about epistemic harm, it's "the  contribution of health equities that is rooted in knowledge itself, its formation, shape,  set-up, and effectiveness. And it's embedded in the knowledge, in the ways in which it's  created and used to exert power." And really, it's around who gets a seat at the table?  And if you see, this is not an imaginary case about whose

voices is represented. It's  mainly the CEOs of the big companies. And they're very far removed, in my opinion,  still from the realities of healthcare. And so, we see quite optimism that we are going to  change how they burn out. We're going to make sure the patients get healthcare better. But the person  who's left behind is really where -- you know, who -- the person who's taking care of the  patient. And, you know -- and I'll show you that sometimes we think about the mental  models of how

we think about our world, how it should and could be, the way it is, the  way it's according to data, and the way it's according to model. And now, we have according  to AI. And the person who's determining how this world is according to AI is very different  from the person who's caring for the patient. And so, some of the examples, for example, social  problems, can be fixed by for-profit data-driven solutions. We've had a prior presentation saying  maybe we're thinking about open source. We a

lso have to think about closed models that are  open for use that encourage innovation. We see quite a lot of abstracting how much you  eat, but they don't really think about the underlying complexity of obesity or access  to -- you know, to the correct food. And so, we see this a lot of technology being  postulated as the solution for everything. We also see this focus on, you know, efficiency.  Today, we want to solve doctor's burnout. But I can tell you, soon, it's going to be how many more 

patients can you treat and, you know, squeezing and squeezing and squeezing the efficiency model.  I can tell you that we have to think still about, what is the value? And also, big -- you know,  thinking about that investment in technology will augment human intelligence more than  investment in our fundamental pillars, for example, education, culture, you know, and  the institutions. And so, it's this gap about where should we put our limited resources,  and how that is going to change the fut

ure? And so, when I think about can we really -- what  have I learned in my years working in this space, it's really been around learning with  others. And that's -- I can try to tell you what I -- what my thought process has  been. And it's this concept that we realize it's very difficult to go through school to  change curriculums to really learn, "Okay, the Kaiser data, what are pitfalls," unless  you have people working on those datasets. And so, for a long time, we've ran this  Datathon. So

, we bring different experts to work on the datasets. And I can tell  you some of the realities tend to be "Oh, I didn't realize that my recording of the,  you know, arterial blood gas five minutes later -- because it's not automatically captured  -- has an impact on how the data are analyzed, you know." So, it's this gap. But it turns  out, with the technology that is getting there, that programming is really -- can  be done by high school students. We have to really rethink about how do we  th

ink about educating and localizing concepts of fairness, accountability, transparency, and  thinking and moving to this HASTE camps for, how do we think about this concept of  discussion around policy? And, you know, this is an example of one that was done in Uganda.  I can tell you, as someone who does fairness work, I only knew that I'm Black when I moved  here 10 years ago into the U.S. And so, fairness has a very, very different  concept based on where it's done. We know that in Europe, ther

e's actually a  penalty for social risk classification and social scoring systems. We know that in our  country, there's a big debate about the woke and the not woke. And it is getting very,  very tough to do this type of work. And so, we have to be really intentional in terms  of how we're thinking and actualizing and conceptualizing what we do and think about  policy. So, I started off with 2016. Thankfully, I still have a job. And the dance are dropping  off, but also trying to tell you what

has informed really is looking for advice, how should we  be thinking about even how we fashion policy? And I want to leave you with a call  for you to emphasize on learning, learning from the past mistakes. We see a lot  of technology being slapped on, for example, EMRs, which all the doctors and the nurses  hate. We think about how do we empower this learning workforce and learning healthcare  system if we're going to think about policy that doesn't come 10 years later? And then it's  this emp

hasis on this rapid learning cycles. How are we -- by the time our paper is  published, that's almost like, you know, two or three years down the line from  the intervention. How do we encourage especially learning from failures and  regarding -- especially from the past? If you think about most of the innovation  has been focused on a very small subset, if we are going to envision a future that  cares for everyone, we're going to need to do a lot. And so, thank you, again, for the  invitation t

o be part of this presentation. [applause] Ainsley MacLean: Right. A big round of applause once again for all of  our panelists. Thank you so much. [applause] So, I'm sort of still processing so many messages  that came through really strong and powerful, and I'm sure the rest of you are too. I think  I'll start off by distilling down a lot of what you said and just asking, if there's  one takeaway you have for our audience, as they go off and do all the  important work they're doing related to

implementing AI in healthcare,  what would that be? I'll start -- yeah. [laughter] Junaid Bajwa: Start with me. Thank you. So -- can I have two? Ainsley MacLean: Yeah. You can have two [laughs]. Junaid Bajwa: So, one is -- so, I think about two-by-two matrix, if you  will, around risk versus complexity. And if I think about healthcare all up, I think  the clinical facing use cases that we consider are generally high-risk, high-complexity  work. And like colleagues have described, like Julia and

Judy were talking, there's  a lot of work that needs to be done if we think about the implementation of those.  But the business of healthcare may afford us the opportunity to build a muscle on  low-risk, low-complexity kinds of work. So, I wonder how much productivity we might  be able to unlock if we think about the impact of these tools in the finance part of our  organizations, the HR parts of our organizations, and focus on boring, mundane activities  that might unlock hidden potential. So,

that would be one kind of  axis for us to think about. I also would just encourage people to play.  Think about what you need to do to build your own experiences and muscle. Don't immediately  deploy them into clinical patient-facing tools and services, but just experiment with these  models and tools and see what value you can have. Don't think about it as prompt engineering  but think about the act of just posing beautiful questions and seeing what the response  might be and focusing on your

own curiosity. Ainsley MacLean: I love that advice. And that concept is one I've heard a lot recently,  really trying to focus on the lower risk, but really high-impact areas. And then before --  I think sometimes we're tempted to kind of dive right into that slightly higher risk area which  can be more challenging. Thank you so much. Julia? Julia Adler-Milstein: Yeah. I think my key takeaway is that we have to do this work  collaboratively. And there are many forces that push you to not be coll

aborative, right? In  some ways, the work is so hard and complex that, like, we can just spend all our time trying  to figure out how to get it right for UCSF and really miss the important opportunities to have  conversations like the one we're having today. And so, I think it's to really sort of push  yourself to, like, come to these events, talk to, you know, other organizations  like yours. Because I think that is really where the rapid learning and the sort of  avoiding common pitfalls will

happen is by doing this collaboratively. And I worry  that we don't have enough structures that are forcing that to happen. And we have  sort of all the pressures just to stay in the four walls of our own organizations  optimized for what we're doing locally. So, that's, I think -- you know, just  continue to come to these -- you know, there's a lot of different coalition's that  are trying to put in place those collaborative structures. So, just, you know, go to one  more of those than you migh

t otherwise, because I think that is really what  the key to the future is going to be. Ainsley MacLean: What great advice. I was struck hearing UCSF's journey, how similar it was to Kaiser Permanente.  So, it was really refreshing. Judy, how about you? Judy Gichoya: So, in addition to learning, which is what I would say is -- my reaction to your comments is  you may not have a choice really in what's coming down if you see sort of like every week there's  a new policy document or new guidance w

e see now every time I go to work. When the Joint Commission  is coming to visit, there's even an overhead like the Code Blue announcement that the Joint  Commission is here today. And the Joint Commission is now interested in what you're doing with this.  We see, for example, the AG of California is here. And I think that can get very, very overwhelming.  So, my thing is to not be afraid to fail and to just get started. And usually, I say sometimes it  just means picking a simple thing. It does

n't need to be, again, patient-facing and just touching,  because it shows you so much about your own internal organization. For example, which deputy  should you deploy? Even just that simple thing, you'll find out that there are so many  bureaucracies and partnerships between AWS, within Azure, within every cloud provider.  That's going to spend years for you to solve. So, if you think about your small  experiments as learning and helping you get to a pillar that is going to help  you to be su

ccessful, I think, ultimately, you're going to be successful. And that also  means looking in your own divisions to see what people are doing. Sometimes you're  very blinded to what medicine is doing, and yet Radiology is miles away. And that  can be an easy way to just get started. Ainsley MacLean: I love that. And I loved what you shared also about Radiology.  And I think that the lessons from Radiology are really a lesson for everyone in the room who's  in a profession that's been told that A

I might ultimately replace them. I think Radiology  is now the leading medical specialty in publications within artificial intelligence. And  I, too, was in the room many years ago and was told that everyone in the room should stand up  and wave goodbye to the table of radiologists. [laughter] So, I always like to say we're  still here, right? Sorry about that. [laughter] So, such great answers. So, I guess I'm  going to get a little bit trickier. And maybe I'll start with you, Judy. When  you'r

e assessing a project to implement, an AI project, how do you determine the  ROI? How do you determine if this is the right investment we make, whether it's a  pilot that we're going to scale for Emory? Judy Gichoya: So -- and this is the problem of being a researcher. Because you can do so many  grandiose things without -- and kill them without consequence. My work has always been really to  think about how AI works, does it really bring value? So, anything that I -- I stopped moving  from diag

nostic models quite early in my career. Although I'll tell you that I'm back there for  one reason, is because I felt that -- and really, even the research now shows that you don't really  get a lot of efficiency gains when you use AI. You save radiologists maybe an hour or two. Like,  it's really not worth all that investment. And so, the thing that I have come to when I pick  a project, I do obviously have a lot of students, so I look at the learning opportunity. But also,  when I think about

the value for healthcare, I look at what is going -- what is it going to  do? And living in this gray area which is our AI world is something that everyone in this  room has to be. And the reason for that is, for example, one of my lessons this year is that  all the metrics that we use for AUCs, they're useless when it comes to a radiologist, how I use  AI. And it took me a long time to come to that. Because what I want to know is, if I'm looking  at this chest x-ray, is the AI usually right or

wrong? And that's almost like a positive  predictive value or a negative predictive value, which are never ever reported. And I would  never use those to buy. I want to know, if I'm going to hedge and say, correlate  clinically, how forceful am I going to come down? And so, to me, I feel that the  personalized metrics are a little off. The second area, as someone who works in  health equity, is that the metrics are all over the place and really don't translate  to outcomes. So, I love the word c

linical vigilance. Because if you give me a model that  performs at 56 percent ejection fraction versus 55 percent ejection fraction, to me, they're  kind of a little bit the same. It doesn't make sense to keep tuning and tuning because I'm  not going to change my management. And so, that bridge to outcome and that  bridge to personalization is how, today, I'm thinking about it, but that's  also an evolving process over these times. I said that I'm going to come back to  the predictive model. I

just came back from home in Africa, and the quality of care  still needs to get better. And in that place, I really should be building predictive  models and more predictive models. Ainsley MacLean: Great. Thanks, Judy. Julia, how about you at UCSF? How do you determine whether to make  an investment in a new technology or use case? Julia Adler-Milstein: Yeah. I mean, I'll agree that, like, we are not doing traditional ROI calculations  because I think we don't have the investment piece well-def

ined [laughs], and we don't  have the return piece well-defined. We're still trying to figure out just what are the right  metrics to assess. But I think we are starting, as has come up before, with like the use  cases that are in your sort of high-value, low-complexity bucket, like what we call keyboard  liberation, I'd say is mostly our focus and where we're just convinced that there is a return on  investment because we know that our clinicians want to be doing less documentation. And  we've

seen the performance of the tools, and I think are pretty convinced that  they are good enough for scale deployment. So, I think, right now, it's -- we are --  it's almost more of a sense [laughs] that there is an ROI there rather than like  a traditional calculation. And I think it'll get more and more complicated as we  move into some of those, sort of, you know, second and third horizon to think about,  what are the right metrics to -- you know, to assess, in particular, the benefits?  I thin

k we'll get a better handle on the cost. But I think that benefit one is tricky  because there's lots of different dimension of benefit. There's also going to be risk in  there. So, I don't see that we're going to use sort of hard and fast ROI calculations and  thresholds for decision making in the near term. Ainsley MacLean: Julia, could you talk a little bit more for our audience  about what those tools are to really help with physicians? Because some of  them may not be as familiar with them.

Julia Adler-Milstein: Yeah. Absolutely. I mean, I'd say the two that we are focusing on again at this scale benchmark  are, like, Ambient -- you know, basically, an AI scribe. So, it is something that sits in  the exam room, listens to the interaction between the clinician and the patient, and then generates  the draft note for the clinician to review, edit, and then that goes in. So, it's still a human in  the loop. It's not sort of fully autonomous. But again, we've seen just even in the last

year that  the tools are, I think, maturing quite quickly. I'll also note we're going to pilot two  different tools to see which one our clinicians like better and are partnering  with another health system that's going to pilot one of the same tools and then a third  tool. And our goal then is to put together the results of the two pilots to basically be  able to do an A to B to C comparison. So, again, this is where I think the  collaborative work can really be valuable. The other thing worth

turning on is GPT  response to inbox messages. We know that our clinicians are mostly feeling crushed by the  volume of patient messages that they're getting, and we don't want to turn that off. But if we can  make those a little bit faster and more efficient by offering them a pre-populated drafted response,  then that I think is a win-win. But again, are they going to take the time to edit it? Will  they catch a hallucination? You know, these are some of the risks of turning on these models. 

So, we're also trying to sort of, you know, validate and make sure that they are sort of  clinically ready across a breadth of scenarios. Ainsley MacLean: I love that. And our radiologists have been using sort of GPT-generated  responses in their impressions of their radiology reports. And one of the things that we hear is  just how much that cognitive burden is really decreased when they're using that generative  AI for just a small part of the report. So, after a long shift, they're able to,

you know,  go out to the gym, drive their kids around, whereas before they would have just crashed in  their bed [laughs]. So, I was curious, Julia, what sort of response have physicians been giving  to this technology work you've been implementing? Julia Adler-Milstein: Yeah. I mean, I think it's exactly what you said. It's not necessarily a  huge time reduction -- I think you mentioned this too -- but it is less cognitive work. And  it is more empathetic. And, you know -- so, it's -- I think o

ur sense is that -- you know,  that it is relieving some of the burden. Again, we're really early on in the journey. And,  you know, we're going to measure and monitor, like, which clinicians are continuing to use  it. You can turn it off and stop using it. So, I think once it's sort of out there at  scale, we'll really understand, you know, who's finding it beneficial, who's not? And  like, what explains that? But I think for now, there's a lot of interest in the tools. I'll  just note, I'm per

sonally very curious because some of our clinicians have started to say,  "Well, the GPT might say, 'I recommend you come in for a visit,'" right? And then does  that shift the physician's propensity to say, "Actually, yes, I agree. I think that patient  should come in for a visit," or not. I mean, I think there's going to be a lot of,  sort of, nudges and other pieces of these. So, again, it's just why, when we talk about  the -- thinking about the outcomes and impact, you have to take just suc

h a broad set of  measures. So, they could tell us it is, you know, less cognitively burdensome,  or it isn't saving time. And like, we're going to have to put  this all together and say, like, "Is this the win [laughs] that we want or  not?" And it's very complicated to kind of make that final call in terms of, like, whether  the tool is doing what we thought it would. Ainsley MacLean: Wonderful. And I do think it's interesting how the areas you focused on, as been  seen with KP, are the areas

that have the biggest bang for the buck for our people. Because I do  feel that, you know, our people -- and I'm hearing that from you as well -- are really our greatest  resource. Wonderful. Thank you. Junaid, talk about -- you're looking at it from a different  angle, right, not necessarily care delivery, but for Microsoft. How does your company determine  where that -- those investment dollars should go? Junaid Bajwa: So, from a health and life sciences perspective, I sit on the R&D  side of

Microsoft. And what we don't do is say research for hire. But we do think about,  actually, what are the biggest problems and challenges that we could try and address? And  so, if I think about that exam question on how might we transform the practice of medicine  with trusted, reliable, human-centered AI, you could do lots and lots of things, right?  You could choose to work provider centric. You could choose to work in drug discovery, in drug  development, or across the entire continuum. And w

hen we think about where we're going  to spend our time, resource, and focus, it has to be on problems, ideally, that nobody  else has solved before. So, we think about the horizon three category that Julia was  referencing. And we think about those deep, deep problems that can have transformative  impact and move the needle on an industry but can also have real world impact. And  almost think about almost a public value proposition if you will. And if we were to  do this, what would be the tota

l impact on society that aligns with the broader mission  around empowering everybody to achieve more? So, there are some things I could probably  share -- perhaps over lunch -- certain things I can't talk about publicly just yet. But we get a  chance to really explore, with these novel tools, what the art of the possible might be? We kill  way more things than we actually then deploy into practice. But things like that radiotherapy  planning tool, for example, do something that could tangibly m

ake a difference from a  3-hour interaction to a 30-minute interaction, adds both productivity and reduces the  cognitive load, but also has a deep and meaningful impact on cancer potential outcomes  is -- are the kinds of work that we like to do. Ainsley MacLean: Wonderful. And I'll put it back to you. What are you most excited  about as you look at this next year? What are, sort of, the big things we  should be on the lookout for? Junaid Bajwa: So, I -- can I go longer than a year? Because --

[laughter] -- I think in a year's time, I would hope  that organizations such as yourselves, Emory, UCSF, have become a lot more confident  on where these tools can really work and where actually you need more evidence to deploy  them in a meaningful way but if we kind of extend it out a little bit. People talk a  lot about this notion of precision medicine. I think we live in a world where we practice  medicine on a law of averages. We discussed earlier the challenges around equity, fairness, 

and others. Can we push ourselves to be better on that? And could we move to a future  where we have more precision diagnostics, more precision therapeutics, more precision  ambient intelligence that allows my practice of medicine to change and allows me to  actually practice in the future that allows me to be much more precise over  time and way less focused on averages? And if I had access to information when -- so,  I work in primary care. It's very different to secondary care settings. But i

f I knew not only  what's in my medical record, but I knew more details around genotype, phenotype, environment,  social circumstances, if you will, I think I could really offer better care to the patients in public  that I serve as a consequence of that. And I think that's the future that we might be able to enable  if we make the right choices now for the future. Ainsley MacLean: So, you're talking about within genomics and  pharma and targeted treatments? Junaid Bajwa: All of the above. Ainsl

ey MacLean: Okay. Great. I like that. Julia, how about you? What are you most excited  about? And you can go beyond a year also. Junaid Bajwa: [laughs] Julia Adler-Milstein: Yeah. I mean, I feel like at the most conceptual level, I am excited for the  moment when our care teams, our patients say, like, "We love this technology [laughs].  It is making healthcare better." I mean, I just feel like we've been  talking about the promise for now like decades. And it felt -- it's  felt like it's contin

ued to fall short. And I just -- I really do think that in the next  few years, we're going to have that breakthrough moment where the technology is going to start  working for us better. It's going to make, you know, the whole process feels more efficient,  safer, you know, like it's really putting our people, as you said, at the center. And it just  -- I just think that will be such a satisfying moment because we've been putting in so much  time and effort, right? So much data has gone into th

ese systems. It feels like what are  we really doing with it? Is it adding value? And I think we're going to start  to see it really feel tangible, that value moment. And then we're going  to get to do so much more exciting stuff because we all sort of won back the hearts and  minds of the people. And they will then say, "Okay, I do love my EHR as much as I love my  iPhone," right? Like, that will be a huge moment. Ainsley MacLean: Wow. Julia Adler-Milstein: And, you know, I'm not saying that's

one -- Ainsley MacLean: I'll retire on that day [laughs]. Julia Adler-Milstein: -- to two years. But like -- I mean, I really do feel like we're -- it's starting  to feel like we're flipping towards that. And I just think that's going to be such an  exciting moment. Again, a lot of hard work still to get there. I don't want to trivialize  it. But since you asked what I'm excited about, it is really that moment in the sense that  it is finally starting to be on the horizon. Ainsley MacLean: I lov

e that. I'm getting chills as you're talking about it because I also feel that coming. Judy,  how about you? What are you most excited about? Judy Gichoya: I think two things. One is we're going into an election year. And  I think we've seen quite a lot of generative AI uses beyond healthcare. And I think they're going  to serve as a big lesson to the potential and the harms of this technology. So, we're going  to work what the Hype Cycle does. I think, as of today, I saw, for example,  the Meta

toolkit that was done to show this is AI-generated, did not  work, was broken down in a few hours. And it's going to really show the complexity of  working in this space, not just for healthcare, for this general business. And that's going, I  think, to ground or move some general policies, for example, health AI literacy for the  population, even if it's at a sixth-grade level. And so, I think we're going to see those societal  changes. And that's going to be good to allow the hard work to con

tinue, for example, in mission  critical areas like defense and healthcare. And then the -- personally, I'm very, very  excited about the potential of using AI as a hypothesis generation ad when I think  about, if I had all the data at Kaiser, that's what I would do. And the reason for  this is that AI has amazing abilities to pick up trends that are not very obvious to us, that  will take us years to come up with hypotheses, not just for patient care, but just our practice.  I'll give you an ex

ample. As someone who does Radiology AI from a chest x-ray, today, I can  tell you the image-based age of the patient, the risk of the patient, their cardiac ICD codes,  their healthcare costs with amazing ability and even a mark of kind of where they live when you  think about the area in the provision index. And those things, if you have to hire the  biggest workforce to do that underwriting for you, it would be impossible. It has tremendous  opportunity for harm. Because you can say, "I'm not

going to cover this group of patients."  But if imaging only, there's more to an image, and we can harness that from AI. This is, to  me, much more important than ever augmenting a radiologist. And I hope that we're going  to see some of this bold population-based use cases and opportunistic screening as  something that I'm very excited about. Ainsley MacLean: I love that. And I'm really excited about the role of AI within women's  health, especially in the breast cancer screening end-to-end so

rt of diagnosis. So, thank you so  much. So, I think now we'll open it up to our audience and see if we have any questions  from you all. I'm sure you have several. Amy Andersen: Well, thank you so much. This is an amazing forum. I'm so happy to be  here. Amy Andersen, I'm with Oracle Health. And I just want to thank the clinicians and researchers  for grounding this discussion where it should be. I spent a lot of my time within Oracle, you  know, engaging with our leaders around sort of first p

rinciples. It's about -- this is  -- I've loved how Vivian mentioned, you know, people caring for people. And that's really what  we need to focus on. I think what I learned from this forum particularly is the importance  of really looking at the delivery of care, how data can inform, and then where do  we -- where do we apply our priorities for AI? We hear this all the time from  our customers. Where do we start, right? And I'm really passionate about looking at  sort of how can we enable the s

tandard of care and practice, which we know  is evolving all the time? And I think the highlights that you have mentioned,  particularly around the clinical experience, is what are those roadblocks, right?  So, we see with personalized medicine, in particular, the increasing use of  NGS, right? However, what we know from clinicians is, those NGS reports are not  necessarily helpful in and of themselves. So, you know, one of the things that we're doing  is working with some innovative companies t

hat are applying, you know, AI to those tests in order  to deliver insights that are based on, you know, evidence. So, I love this. I love kind of thinking  about, you know, being practical innovators and also ensuring that we have the ethical guide  -- you know, guidepost for us. So, thank you. Ainsley MacLean: Thank you. So, I'm not sure if there was a question, but you could  -- I would just latch on to one word you said, which is roadblocks. So -- and I think that I do  want folks to leave h

ere knowing that sometimes there are roadblocks almost all the time in  terms of implementing them. And I don't know if I could put that to the panelists because  I think it's a great question. What are the biggest roadblocks you've seen in terms  of -- and obviously, you're going to come at it from a different angle, but in terms of  trying to get this technology out to be used. Junaid Bajwa: So, maybe I can answer it from some of the  NHS experience, if you will -- Ainsley MacLean: Okay. Junai

d Bajwa -- right? Ainsley MacLean: Wonderful. Junaid Bajwa So, implementing any change is extraordinarily difficult, right,  from the simplest thing that you have to think about doing. And even if you believe that  the right -- it's the right thing to do, do all of your stakeholders believe it's the  right thing to do? So, we've -- at the hospital I sit on the board of, we implemented  Epic, pre-pandemic, massive investment, really good change management. And the change  management investment ac

tually paid massive dividends because it took the clinical workforce  and the non-clinical workforce alongside. And I think where I've seen things fail is when  the change management hasn't been considered, and you just say, "Hey, deploy this  thing, and you'll reap the benefits, won't you?" And they haven't taken  the entire multiple tribes that exist within any given healthcare  system along that journey with you. But making -- understanding that change is  hard and understanding that you have

to get stakeholder buy-in and understanding that you can  start off with some relatively small, meaningful, impactful projects in a view to getting on to  more complex things is probably the journey that I would encourage people to consider. And  the NHS has multiple, multiple instances of where things have failed over time. So, learn  from others and experience from others too. Ainsley MacLean: I love that. So, change management, absolutely -- Junaid Bajwa: Yeah. Ainsley MacLean: -- critical w

ith this. Julia, how about you? What roadblocks do  you see being sort of the most -- Julia Adler-Milstein: Yeah. Ainsley MacLean: -- significant? Julia Adler-Milstein: I mean, I frankly would say I don't think that there are true roadblocks in  terms of things that are stopping us. I think what they are, are speed bumps. And we're trying to  decide how much to slow down as we see them. So, probably the hardest things are,  like, safety and equity, right? As we look at the tool, like what is the

  actual performance on safety and on equity that is good enough to deploy, right? And  then you have to just have really complex, nuanced conversations about, like, what is  the current state, right? Because it may be inequitable but as long as it's better than  the current state, like, is that good enough? Ainsley MacLean: [affirmative] Julia Adler-Milstein: And so, I feel like you get very quickly into sort of ethics and  ethical discussions. And then I will just stay on our AI Governance Com

mittee. It feels  like, are we really well equipped to make these decisions? You know, I'm not an ethicist  and -- you know? So, I just think that -- Ainsley MacLean: [coughs] Excuse me. Julia Adler-Milstein: -- that is where -- it's like how sort of slow or quickly to move is really, to me, what feels like  the hardest and scariest part of these decisions, right? Because it's sort of one -- you know, one  wrong word and a GenAI message that goes back to a patient could really be life or death,

and like we  are the final arbiter of whether that model should go forward or not. So, the stakes feel very high.  But yet, if there's broad value to that tool, we don't want that, you know, possibility  of one thing going wrong to stop us. So, it's -- I'd say, it's those that are -- to me,  feel like the biggest challenges right now. Ainsley MacLean: Thank you, Julia. Judy, how about you? Judy Gichoya: I'm going to just respond to this as opportunity. What's  the opportunity today? And first of

all, you should have a team, and you should staff it  and provide the resources that you need for it. I think we are missing the patient voice. And  this is because we're trying to reinvent how to communicate results to patients. And I believe  that there's quite a lot of opportunities to have an engagement. For example, you show me that  GPT will be used for a rare disease diagnosis, then you know the realities of patient privacy  or anonymizing someone with a very, very rare condition is near

ly impossible. Yet those are the  same group of patients that can benefit a lot. We see a lot of patient advocacy groups  for those patients with rare diseases, but they're completely, you know, not at the  table. And I think that if we can engage them, I think we're going to mitigate some of the  risks downstream, because we will, you know, be very forward with what worked, what failed.  And I think it's just a blind spot to most of us. And then the other surprising thing for me is  -- as watch

ing this deployed to systems for Radiology at Emory, is the human factors. And I  think we underestimate what that will look like. For example, one of this is triage algorithm. We  know outpatients are long, long, long. You don't want to be sitting on a patient with a bleed  when you should have read the study early. But showing people where they missed, where they  -- you know, and documenting them has a tremendous impact even how you deploy the system. Because is  this discoverable? Are you go

ing to start to say, "Judy misses a lot of these cases all the time.  Maybe she's not a good radiologist," when they come to review me in my annual feedback? You  know, you're going to say, "Who else knows?" And that can have such negative consequences that  some people say, "We don't want you to deploy it." So, the technology, I do believe, with the answer is we'll get better. But the  human factors and the human side are really, really behind and is underestimated in  the success of this techn

ology deployment. Ainsley MacLean: Thank you. I think you can see why these are all such  seasoned leaders. We went from roadblocks to change management,  to speed bumps, to opportunities. [laughter] All right. Great. And any other  questions? I see a couple of hands. Neil Carpenter: Sorry. I'm trying to figure out how to turn this on. Okay. Sorry  about that. Neil Carpenter, Redesign Health, and Tau Ventures. Let me just kind of change the  level of question, if we can, for just a second. A lot

of how we've been talking about this is from  sort of inside the care delivery system, out. So, let me ask a very different question, is if  we sort of took a step back to a societal level -- because this is like game-changing technology  and said -- as a society, how do we need to think -- what is something that healthcare system  could do differently with this technology that'd be beneficial to society, which is kind of above  the clinician, above a specific patient, above an administrator, a

bove a P&L? What is maybe one  thing that it could do in the next 5 to 10 years? Ainsley MacLean: Well, that's a very easy question. So, I'm going to turn -- [laughter] Judy, I'll start with you,  just to switch of the -- Judy Gichoya: So, this is society -- I would say that when I flew yesterday through Atlanta Airport, one  of the busiest airports in the world, I saw a lot of digitalization, digital ID, CLEAR, TSA Pre. I  think we're underestimating displacement of jobs. And I think we should

be looking at what that  future is going to look like. Not replacement, but there will be -- I can tell you that in  five years, when I go through TSA to boarding, that there'll be very, very minimal human  contact. It's very clear that you can see that this is going to happen. But we are  not thinking about how to educate today our students to live in this world where they can  search online and get the answers. What are those important skills to build the next generation  workforce is what I w

ould spend my money on today. Ainsley MacLean: Wonderful. Julia? Julia Adler-Milstein: Are we getting signaled? Am I going to -- Judy Gichoya: [laughs] Julia Adler-Milstein: -- escape with our time's up, or should I go ahead? Judy Gichoya: [laughs] Ainsley MacLean: You know what, I think that's the perfect place to stop, TSA -- [laughter] -- since we're all dealing with  the TSA. All right. Great. Well, thank you once again for all  of you for being, I think, so honest really with the problems y

ou're  dealing with and also being very brave and paving the way for the rest of us. So, thank  you. A big round of applause for our panelists. [applause] Rebecca Flournoy: All right. A huge thank you to Dr. MacLean and our panelists for such an interesting  conversation. We are going to transition now to lunch, which is going to be served in the very  back, buffet style. We encourage you to bring your lunch back to your tables. Or for those of you  not seated at a table, there are tables near t

he buffet area. And please do be back in your seat on  time. We will start right at 12:30 p.m. Thank you. All right. So, our final panel today focuses  on addressing risks and harnessing benefits of AI in healthcare through policy  and practice. I'm so excited about this panel. And I'm very excited that Dr.  Daniel Yang will be moderating. And we've also asked him to share some closing  reflections after the panel as well. So, Dr. Yang recently joined Kaiser  Permanente as our new vice president

for Artificial Intelligence and Emerging  Technologies. And in this role -- [applause] -- yes, people are clapping. We're  very excited. Yes. In this role, he's responsible for ensuring oversight for  all AI applications for the organization across clinical operations, research,  education, and administrative functions. Before joining Kaiser Permanente, he was a program  director at the Gordon and Betty Moore Foundation, where he founded and led a philanthropic program  on diagnostic excellence

. In this role, he helped to establish several public-private partnerships  to promote responsible use of AI in healthcare. He created infrastructure to support development,  implementation, and evaluation of diagnostic AI algorithms. And he advanced research methods for  evaluating clinical impact of AI in real-world settings. Dr. Yang is also a practicing internal  medicine physician. And he completed his residency at UCSF and a fellowship in Healthcare Systems  Design at Stanford. Dr. Yang, I

'll pass it to you. Daniel Yang: All right. Hi, everyone. First of all, I'm really excited  to just be here at this event. I feel like it's a homecoming in many ways. I'm able to  connect with all my KP colleagues I've been seeing on Teams in these little boxes.  And to see them in person is remarkable and really just understand the national  impact and footprint of this organization. And too, I just see so many familiar  friends and colleagues that I've gotten to know over the years, in the  au

dience, on the stage. So, for me, this is really kind of bringing these two worlds  together to talk about a topic that, I think, we're all passionate about, which is ensuring that  we're deploying AI in a way that's responsible. I'm really excited to be joined on the  stage by three people that I admire that bring complementary perspective and expertise to  this topic, particularly as we think about the risks and benefits of AI and what that means for  policy and practice. The theme for the dis

cussion is really, how do we move from what we know we  should be doing, to actually getting it done? And so, let me introduce my  speakers. I'm not going to embarrass you the way that Rebecca  embarrassed me with my whole bio. [laughter] I'm just going to provide some, you know,  key bullet points to provide context of where you're coming from, and the expertise that you  bring. So, first of all, we have Tom Romanoff, who is the director of the Technology Project  at the Bipartisan Policy Cente

r. Now, prior to working at BPC, Tom led IT initiatives  for several federal agencies and advised executive leadership on the impact of new and  emerging technologies on government operations. Second, we have Laura Adams who is special  advisor at the National Academy of Medicine, where she leads the development of  an AI code of conduct. Prior to NAM, Laura was the president and CEO of  the Rhode Island Quality Institute and has been a longtime expert in  healthcare data interoperability. And l

ast but not least, we're joined by Dr.  Maia Hightower who is co-founder and CEO of Equality AI, which helps data scientists  develop fair and unbiased algorithms to eliminate discrimination in ML models. Prior  to Equality AI, Dr. Hightower was a physician executive at multiple academic medical centers.  Most recently, she was EVP and chief digital transformation officer at the University of  Chicago. And she was also the chief medical information officer at both the University  of Utah and als

o at the University of Iowa. So, just a quick note on how I plan to run this  panel. It'll be slightly different than the last panel. I'm going to bring up each panelist to  provide some remarks. But as I'm bringing them up, I'm going to channel the thoughts and questions  what I think the audience has in their mind. And after each of the panelist finish their remarks,  I'm going to open it up for the other panelists to kind of function as a quick reactor panel. And  then after we get through ea

ch of their remarks, we'll just open it up to a more general moderated  discussion and finally, for an open Q&A. So, Tom, you'll be our first speaker.  And so, as you're coming to the podium, here's what I think the audience  is thinking about in their head. The current policy landscape for AI is messy.  We've got guidance from the FDA on AI-enabled clinical decision support tool. We've got  the final rule recently released from the ONC on predicting decision support interventions.  We've got th

e White House Executive Order on AI. We've also got state-level efforts. We heard  about the California Attorney General letter. So, can you help us clarify the current policy  and regulatory landscape for health AI? What are the key issues? And what can we learn  from AI policy approaches in other areas outside of healthcare and in other settings  outside the United States? So, Tom, please. Tom Romanoff: All right. Good afternoon. Some easy questions there, and I  will do my best to address the

m. But also, fully aware that it's after lunch. So, if you  need to take a nap or anything like that -- [laughter] -- you know, I won't fault you for it. But,  you know, just beginning with, you know, who I am. I'm the director of the Technology  Project. And to answer Dan's question about the landscape, it is messy. We have seen a lot of  interest in AI over the last couple of -- last couple of months, couple of years. And, you know,  in Congress, we're looking at around 43 different AI-related

bills, a number of executive orders  that are up there, all applied to artificial intelligence. And we haven't even started talking  about the state-level legislation framework. What we're seeing is that, you know, AI has been  around for a while. These folks that have been working with this technology know that there's  not -- you know, there's new products out there, but the technology itself is fairly known in  the software development world. But there's new applications, and we're just now

catching  the attention of policymakers on how those applications are being used in ways that, you  know, frankly, have some serious concerns. So, when I say that AI is not new --  just to put it in context. Okay? So, 1956, the concept about AI and the mathematical  equation that led to what we have today was first conceptualized. I'm not going to go through all  of these. But you can see from 1956 to today, there's been a number of different advances. And  it's always been that way with this te

chnology where the technology advances a little  bit, and then people start to freak out. Are we having a superhuman intelligence? Is --  how's this going to put everybody out of a job? It also doesn't help that sci-fi has oftentimes  pointed to artificial intelligence as the villain, right? So, I can't get through an AI presentation  without mentioning the "Terminator." And, you know, in some ways, you know, you  can conceptualize a path forward towards, you know, AI being able to do a  lot tha

t we think of as sentient, but the clear path is not there yet.  And so, whenever somebody asks, "How many years are we away from superhuman artificial  intelligence," I don't even put it on a timescale. But what I do want to say is that what we are  seeing now means that we can't go back to the way that things were pre-ChatGPT or pre-large language  models. The genie is out of the bottle. Pandora's box is open. And, you know, putting it also in  the context of where we were on a global scale, y

ou know, in the last two or three years, I  pointed to state-level legislatures and some congressional AI bills that are out there. You  mentioned some of the healthcare-specific ones. But if you can see it, there's been a lot of  talk -- this graph goes all the way back to 2016 -- around how to strategize around AI,  how to regulate it. 2016 is when a lot of the militaries have all agreed that they  would not use autonomous weapons against each other. And some of those militaries  included Chin

a and the United States. But what I'm -- what I want to convey here is that  no one is really pointing to these agreements or these publications and saying, "Look at the  history here. Look at how much we've already been thinking about AI." By and large, these are  all forgotten. And I will put -- my organization's efforts in 2019, we put out a national strategy.  No reporters are calling me up about [laughs] the national strategy we put out in 2020. It's  okay. I'm a little hurt by it, but it's

fine. [laughter] And so, you know, when one of the speakers  was talking about, how do we localize AI, I think this is a really good example. Because  the other thing I want to convey is that, you know, AI is a reflection -- in this case  with ChatGPT -- of language and how we use language. It's -- and language often has kind  of that anthropological effect of, you know, reflecting how, you know, certain cultures think,  or religions practice whatever it might be. So, localizing AI is going to

be exceptionally  difficult. Because on a global scale, we don't have common thinking about how to regulate this  AI, or what is the common definition of fair, or what is the common definition of equity and  access? And I say that as a pragmatic point of view, not as a kind of general -- you know? I  can point to what I think of as fair in practice, but I do realize and recognize that  there's other definitions out there. And so, with that, I just want to also point out  that, you know, we are i

n a new era, right? And so, what is different now? So, this thing called  Moore's Law where you have exponential growth in compute, in memory, and all that fun stuff  is leading to a lot of what we're seeing in kind of these incremental advances in AI. At the  same time, if you look at the graph on the right, we're seeing increased connectivity. IoT  devices are nearly universal in the United States. And they're growing -- they're  expected to continue to grow through 2030. And so, we have a lot

of data out there. And  all that data is -- from a human perspective, really difficult to recognize  those patterns to create, you know, predictions. But AI is an excellent prediction  machine. That's also a really good book if you want to read something on that. And it's  also something that I like to convey, is called exponential growth being very  difficult for humans to understand. And so, while we're talking about this -- how to  regulate and how to make it safe in healthcare, today, it's

already grown by leaps and  bounds. And exponential growth is a concept that humans just really can't wrap their  heads around. So, you know, a lot of -- so, a good example again is ChatGPT. So,  when it comes to exponential growth, it's always -- they're finding new products for  it. They're finding new ways to make it learn. Somebody mentioned about, you know, having ChatGPT  write a poem or whatever it might be. Well, the other example I want to use is ChatGPT,  when it first came out -- I do

n't want to know how many hands go up on this. But how  many of you actually went in there and said, "Let me see if it can do my job a little bit  better than I can do it"? Right? Let me see if it can write an outline for something  I want to write or whatever it might be. And then you got the product, and it wasn't  that great. There was something you needed to go fix. Like, "Phew, I've got another two  years until [laughs] the technology catches up, and it's better than mine." Right?  But that

two years is coming. And, you know, what I like to say in terms of  job disruption in the space is that what is cognitively very hard for human beings  is very easy for AI to do and vice versa. What's cognitively very easy for us  to do is very hard for the AI to do. And so, that's why you're starting to see people  use it as a first draft, and then go in and use their experience to try to build it out. And  along those -- that point, Goldman Sachs came out with a really scary report, if you ha

ven't  seen it, that, you know, current iteration of AI will raise GDP by 7 percent, but it will  also impact over 300 million jobs over time. Phew, I only have two minutes. Let me hurry up.  All right. So, problem not solved. All right. So, what does this mean? So, if -- in the  health equity space and this idea of, you know, data bias, people have been  working on this for a long time. If you're following the big data trends  and how to use it in the medical space, data bias was big in that. A

I is kind of  magnifying those issues. It's going to amplify a lot of that because it can do things  at scale that we haven't been able to do. And so, along those same lines -- and I tell  this to policymakers a lot -- is that if you cared about data bias in the healthcare space  or equity in the healthcare space or social safety nets to mediate technology disruption,  then you probably care about it as well in the AI space. Nothing has really changed there.  It's just really going to continue t

o amplify. I'm hurrying it up because I only have  1 minute and 20 seconds. Sorry. So, now, getting into the policy conversations, are  the policy solutions justified in some of these scary things? Well, you know, we  have -- policymakers have some bright lines that they don't want to cross,  mis and disinformation election space, you know, making sure that it doesn't exaggerate  existing biases. And in the context of that, we often find ourselves looking  at what other countries are doing. So,

the European Union has been called the  Silicon Valley of regulations. And that's because they're going ahead, and they're going  to go regulate. They put out the AI Act. And then in United States, that has led to this debate  around what do you do around innovation where, you know, folks will argue that innovation  can't thrive in regulatory environments? But then what do you do about things like deep  fakes and bias and job issues? And then finally, at the same time, we're having this big deba

te on  what competition with folks like China means. And so, all of these things are pieces that  policymakers are trying to put together. In the healthcare space specifically -- I'm  going to run over a little bit if that's okay. I just want to talk -- you'll see that, you  know, there's kind of two different worlds. Patients are scared of AI in some situations.  They don't want to see it rolled out too quickly without the safety norms in there. But at the  same time, healthcare is viewed as on

e of the biggest areas of disruption for this technology, one of the biggest opportunities for cost savings. And so, as you see these things play  out in the industry, the idea of trust is really hard in this space. Because  take, for example, facial recognition, right? Facial recognition is something that they  rolled out in the U.K. specifically, and it did not do well with people of Color or people of  East Asian descent. It was very inaccurate. And that idea of AI -- facial recognition is AI

  -- being inaccurate persists today. Even though the technology has continued to excel, once you've  lost that trust, it's really hard to gain it back. And when I say you've lost that trust  in the facial recognition space, I like to say it's Kleenex versus facial tissues,  right? AI is AI to anybody who's not, you know, in this field practicing it. And so, if there's  a group that doesn't trust it because of the outputs that happen in facial recognition, they  probably won't trust it in their

healthcare space. And so, the idea that we need to regulate  across these things, it's dangerous because we see AI as kind of this one big thing  that we need to tackle, but in reality, there are very specific industry considerations  that need to be taken into account. There are some very smart people that need to step up in  those industries to make sure that they are, you know, providing the context needed for  the AI in that space and how it's being used. And so, what I usually tell folks is

, you know,  regulating by use case is really easy but it's also really hard to do in terms of long term. And  you also need to address these problems early. So, with that, I've run over a little  bit, and I would like to thank you. [applause] Daniel Yang: All right. So, first, let me open it up for reactors --  reactions from both Maia and Laura. So, feel free to -- if you have any thoughts that  you wanted to provide, I'd love to hear them. Maia Hightower: Yeah. How far away are we from some m

eaningful regulation? Tom Romanoff: So, comprehensive AI regulation is something that we talked a lot about last year with the  AI Innovation Forums. Those have largely lost a lot of steam, and a lot of members on the House  and the Senate side have been given permission to engage folks on their committee specialization or  their bosses' specific areas of interest. And that means that a lot of the AI legislation potentially  happening will be around some of those use cases like mis and disinform

ation, deep fakes, things  like that, instead of a comprehensive type of approach that, you know, potentially was being  floated when they did the innovation side. Now, it's still not dead yet, but, you know, I  don't see it happening in the 118th. And that's always dangerous to make [laughs] predictions  about what Congress is going to do in this space. But I think smart money is not in the  18th, maybe the 19th, but we'll continue to see states regulate in that space, including in  the healthc

are applications. You'll continue to see the federal government push out, whether it  be rule changes, executive orders. There's one that happened last week around privacy which will  have an impact on AI. And so, you'll see that. The problem with that approach  is that with the executive order, it's subject to a new executive coming in and  saying they don't want to go that direction. It's really expensive to change directions  in this space. And then at the state level, it's really hard to man

age  technology with the patchwork. Laura Adams: Tom, I thought you did a fantastic overview of that. And  it does give us a sense of the complexity and especially when we think about trying to  harmonize this globally. Because I think what we understand across the globe is that  the threats don't stop at the border, and I think we don't want innovation  stopped at the border either. So, I think there's an incentive for us to get out  and do that and to act collectively, globally. And I guess I'

m seeking some advice from  you. I've just been asked to do a -- to chair a global innovations group  out of a U.K. regulatory science and innovation network there. Who would  you invite to the table for a global innovations view to assist the U.K. and also  cross-fertilize for the rest of the nations? Tom Romanoff: So, I have specific names I'd be happy to talk to you about. [laughter] Laura Adams: I was hoping that was the case. Tom Romanoff: But I would say in terms of experts, some of the bi

ggest ones that I would -- so, standards and  definitions are something that in the tech space, especially when we're talking about harmonizing  across international borders, just doesn't exist yet for the AI. And so, folks like NIST and  ISO, anybody who's kind of working on and has experience in trying to establish these technical  standards, I think is critical. It's -- speaking of after lunch, it's really boring, but it's  critical to happen. And so, I'd invite that group. The other big one,

I think, especially in this  iteration of technology and large language models, is the IP and copyright people. Because open AI  is getting sued every other week on these things, and Google also has some pending lawsuits on  that. And I just don't think they've figured out what the technology's use of the data that  it needs to consume means in the apparatus that we have in terms of IP and copyright.  And I actually think that's going to be, long term, really difficult for scaling up  the use a

nd deployment of this technology. Laura Adams: Thank you. Daniel Yang: Great. And I love how in this format, I'm actually  outsourcing my job to the panel. So, thank you for asking great questions.  So, let's move on to our next speaker. And we'll have an opportunity to dive deeper  into all these topics after your presentations. I wanted to bring up Laura Adams, senior advisor  at the National Academy of Medicine. And, Laura, as you come up, here's what I  think the audience is thinking. So, we

just heard from Tom about the messiness in the  policy landscape. If you take it one step down, there's also messiness when it comes to  guidance documents, codes of conduct, best practices that are coming from both  within government and outside of government. Just to give you a few examples, we heard about  the NIST AI risk management framework. We have a trustworthy -- sorry, blueprint for trustworthy AI  from CHAI, Coalition for Health AI. We also have some guidance documents from the World

Health  Organization, et cetera, et cetera. And so, one, what do these guidance documents  tell us? And using a clinical example, you know, we worry about alarm fatigue  in the hospital, particularly in a place like an ICU where you've just got beeping  constantly going on. You just drone it out. Are we at risk for AI guidance fatigue? You know,  are we at risk of just hearing so much of what we should be doing that we just stop caring about  it? And so, help me make the case for why the NAM AI

Code of Conduct is going to cut through the  noise and how it differs from these other efforts. Laura Adams: Sure. Thank you so much, Daniel. It's such a pleasure to be here. I'm delighted to be  able to have an opportunity to interact with all of you. I want to take that first question of what  does it say that we're developing all of these? I think the good news on this front is everybody  recognizes that there's enormous promise, and there's also significant peril. So, I  wouldn't want to be

rushing headlong into this thinking there's only an upside to it. I also  wouldn't want to be focusing only on the downside. So, I think the good news is that people are  trying to make sure that AI does not break bad, because we know that this would be a  crushing blow to something so -- with so much promise for the future. On the other  hand, I think because of these concerns, we keep seeing -- and, in fact, it was the  genesis for the Code of Conduct project in the beginning as we were appro

ached by  people that said, "Gee, for crying out loud, every day we turn around, there's a new  guideline, principle, framework that's put out." And so, the good news is that everybody's  putting out guidelines, principles, and frameworks. And the bad news is the same. Because  we have, now, fragmentation. And in healthcare, we seem to have this penchant for fragmentation  where we will go down our own silos and, as Julia said earlier today, that idea of  feeling we want to work in our own silo

and get it right there without thinking  of the impact on others in the system. And so, the Code of Conduct project is if we  can think about harmonization of data standards, harmonization interoperability, why can't  we think a little bit about governance interoperability? For me, it was so important  when I was beginning to do the living laboratory in Rhode Island to see if we could link up  all of the data sources there clinically and put it into a central repository. It was  really exciting

for me to be able to do that. But we knew we needed an ironclad privacy  framework. So, we worked with opponents like the ACLU and the Coalition Against Domestic Violence  who came in and said, "You have a breach in this system. It's like slitting a feather pillow in the  wind. You'll never get that information back." So, they were so interested in building an ironclad  privacy framework. So, we did. And then I couldn't wait to share our data with the rest  of the states in the nation. I turned

around. We made out pretty well in  the AI -- in the -- rather, the HITECH Act. We got $27 million in  90 days in just our first three grants, and we were off to the races building our health  information exchange. But we spent way too much of that funding that we got on attorneys trying  to reconcile privacy frameworks before we could either exchange any shred of data. So, can  we think a little bit about, what is that? So, the Code of Conduct project was to take a  look at what are the commona

lities here? What are we sort of in violent agreement on? Where  are the gaps? What are the things we need to be paying attention to? But that's only the first  element of the Code of Conduct. The other three elements that are, I think, critically important,  maybe more than the harmonized principal sets, were, can we distill this now into  something memorable? Five or six commitments. Almost think of it like the Ten  Commandments but not ten, just about six. We based that on complex adaptive sy

stems theory  where we can -- we saw in that theory that very, very complex behaviors in the world are governed  by very simple rules. We see that all the time. I saw it in quality improvement in my background.  Whether it was central line infections, we want to decrease perioperative  infection rates. Whatever it was, it almost always boiled down to do about  five things, do them consistently for every single patient, and you'll drop  the bottom out of those complications. And so, we started th

inking, what are those six  simple rules? So, you'll see them come out on the 29th of this month in the Code of Conduct  commentary publication for public comment. You'll see the harmonized principles. By the way,  there's a heavy overlay with the learning health system. I think with AI, if we didn't think  we needed a learning health system before, we know we need one now, because this  is an all-teach, all-learn moment. I was thinking about Judy's remarks  and how she and I spoke at lunch, and

she mentioned being a humble learner. And I  think that ought to be our goal and aspiration for our own conduct and behaviors is being humble  learners. We have a lot to learn, and we will for the foreseeable future. There won't be a time  when I think we tackle this, and we're done. So, when I think about what it means,  I want you to know who's behind the Code of Conduct. The CEO of Mayo, so we have  Gianrico Farrugia, is one of the co-chairs; Bakul Patel, who is the Digital Health  Strategy

global lead for Google; and Roy Jakobs, who is the CEO of  Royal Dutch Philips in the Netherlands. And you'll see here on our AI Code of Conduct --  if I can get this slide to change. I like to do this without names first to see how many faces  you can recognize. I know you're going to say, "Hey, there's Andy Bindman, chief  medical officer from Kaiser." Yes, there's Andy Bindman, and he even goes as  far as not just to serve on our steering committee but he's serving on one  of our work groups

that I'm going to explain a little bit about in a moment.  But I'll give you a minute to take a look. That's Peter Lee from Microsoft on there  too. Eric Horvitz, also from Microsoft, on there. Suchi Saria, brilliant mind in AI,  just absolutely amazing. Grace Cordovano, one of the most articulate, outspoken, and effective  patient advocates you'll ever come across. I was astonished watching Grace at our first couple  of steering committee meetings. Because we might be talking along the lines of

things, and we got  insular and more insular and more insular. And she would make one or two statements, and everyone's  head would snap around to what it meant to be. We were talking about something that might  be important to patients and families, and she cut through all of that by  saying, "Listen, patients and families, family members, they want their  family member to live. Patients, they want to live. They want to be able  to raise their children. They want to be able to live out their l

ives. And in doing  so, here's what they need." Crystal clear. So, I love this steering committee. I'll  give you the names on it. There we go. So, you can take a look at the names on the steering  committee. We wanted Sanjay Gupta because we wanted to be able to get good communications  advice on this. And what we're finding here is that we have strong equity and ethics experts that  have guided our work every step of the way. We, too, are worried about workforce. Kedar Mate  and his work at IH

I, quality improvement. I'm looking across the landscape of those  organizations that I'm interacting with now that are putting together their work on AI, and I  think, "Hey, those of you that made investments in quality improvement, in plan, do, study, act, in  that idea of small-scale tests of rapid change, you're way ahead." You made a smart  investment because, as we heard today, so much of the success of AI has to do with  can you implement it in your own setting? So, the Code of Conduct in

itiative goals  are that governance interoperability, want to be able to -- can we have a common  language to start with so that we're not trying to reconcile all of these? And by the  way, we do recognize that in working with these coalitions that we're working with, the  NIST risk management framework, for instance, is an exquisitely well-done document. It's  42 pages though. And if you are a sort of community health system CEO and you're trying  to look at that document for guidance, I might

recommend -- well, maybe I wouldn't go this far,  but you might want a cardiac defibrillator nearby. [laughter] Because you are going to have  palpitations when you think, "Oh, my god, there is no way I can put 42  pages of detailed requirements. I can't build that into my system." They're going  to stop breathing. So, we think that some of these publications are suited for some  audiences. Some are not suited for that. So, we're working with a number of  coalitions. The Coalition for Health AI,

many of you know that that coincided  today with their announcement of their board of directors. They're formally  incorporated now. We're super excited. They're going to be putting together a network  of certification labs for AI. We're excited at the National Academy of Medicine because  they will certify to our Code of Conduct. So, this is what it looks like when we  begin to align. The most wonderful thing about us running a little bit scared about  AI, at least having the wherewithal to kn

ow what we don't know or at least have  an inkling about what we don't know, is that we're coming together in big time,  big ways to learn. We're doing more and more. I think about Steve Jobs' admonition to  us about when we've got really difficult problems or we've got complex things to  work through, the most exciting thing to think about here is collect as many dots  as you can. The more dots you connect, the more dots that we can pull together and get  general pictures and come up with creat

ivity. So, these are the groups. The Light Collective  is that group of organizations that are rare disease advocates. They are patient  voices. And they are helping us put together the translation of what this looks  like when you take the Code of Conduct down to another level. We can't stop at the Code  of Conduct. We've got to translate it into, what does it look like from those perspectives  so that we're creating an interstitium, a connective tissue, and we don't just  proceed ahead without

regard for each other? So, that's what we mean when we're talking about  the roles and accountabilities component of that. And, boy, are the consumers giving us an earful.  They're telling us exactly what those behaviors look like if you really are including them as  patients. They're even developing a scorecard to say you can score yourselves, and, by  the way, we want to score you as well. The last thing that I'll say is that  it translated into behaviors. And then, once again, the other thin

g that we're looking  at here is when I think of bias and my concern about a digital divide, another equity divide, I'm  concerned not only that the algorithms are biased, but I'm concerned that those with the  most resources are now galloping ahead. Most Americans get their care in midsize  hospitals, in community health centers, in rural hospitals, critical access  hospitals, in these places where they're not as well resourced. We want  to, in this Code of Conduct project, describe what it mea

ns to build the resources  in the center so that all of us can proceed, so that we don't, once again, prevent the patients  being served by those entities from getting what it is that they need out of AI. Because, frankly,  they probably need it more. Thank you so much. [applause] Daniel Yang: All right. Thank you, Laura, always so eloquent. And while you  didn't say it explicitly, I think one of the advantages of the NAM AI Code of Conduct  versus these other efforts is having you. And, you kno

w, really, you talk about being a dot  connector, and I think you're one of the best dot connectors out there. So, appreciate  your and the NAM stance of really being a humble learner and kind of building on the  great work of many other efforts before it. So, let me transition back to  this reactor panel model. So, Tom or Maia, feel free to ask your  co-panelists or share any remarks. Tom Romanoff: [clears throat] Excuse me. So, I'm just curious about the -- maybe I missed  it, but the patient

input side of things. Laura Adams: Yeah. Tom Romanoff: How are you capturing that? Laura Adams: Ooh, thank you so much for asking because I ran out of time,  and I wanted to be able to tell that part. The Light Collective is a collection of  patient advocacy groups. And so, they are in the middle right now. And I don't know that  I mentioned that Daniel was the first funder in, of five funders on our Code of Conduct. He saw the  vision for it immediately. And California Health Care Foundation, P

atrick J. McGovern Foundation,  NIH, Epic, all followed. And we were so pleased. You also funded a group called The Light  Collective, and that's Andrea Downing. And she's a brilliant spokesperson for  patient advocacy. Andrea Downing found out that their BRCA3 breast cancer group  that was putting together all of this deep sharing going on in Facebook, that there  was a flaw in Facebook that was allowing their very intimate information to be  channeled to other sources. You know, we think of Ca

mbridge Analytica and places like  that. She was the first one to discover that flaw. Andrea is no dummy. She's really smart in this  field. And so, what she has done with Gordon and Betty Moore Foundation funding is to put  together the guidebook for what it looks like when you are actively engaging us. So, it's  built upon -- they did a patient-led research guidebook which was for researchers to say,  "We're going to tell you all the different ways in which we want to be involved and be  partn

ers in research. We're going to tell you what it looks like when that behavior  is poorly done. We're going to put it on a continuum until we can show you what it's like  when you really do it extraordinarily well." They're doing the same thing for their AI  rights and what they want their roles and responsibilities to be in governance of AI, in  deployment of AI. And we're taking that guide and bringing it right into the Code of Conduct.  So, a section of our final capstone paper will be that a

pplication of what consumers have told  us. "This is how -- these are the behaviors. This is what it looks like to apply this and  to have us feel as if you've done us justice." Daniel Yang: I would just add, if Andrea was in the audience, probably  the first thing she would say is, "Pay patient advocates for their time and  attention." So, we definitely want their input, but oftentimes, we don't think about  reimbursing them for their effort. Maia Hightower: Yeah. Speaking of which, that was my

question, is how do we  then translate these lessons learned on amplifying the voice of patients on a  governance level to the very local health system level or even our policymakers? How  do they do that using that same capability? Laura Adams: Yeah. I would cite Mark Sendak's work out of Duke University. He's  got a collaborative governance model paper. Go get it. Look it up, S-E-N-D-A-K, Sendak, Mark  Sendak. And Suresh Balu also co-authored that paper. They have done one of the best pieces

of  work I've seen in actually bringing the voices of the local entities around. Because as much as I  appreciate, applaud, and anticipate working with the Coalition for Health AI, we can certify  an algorithm or a model at a certain level. We're almost -- not almost. We're always going to  need to take that same algorithm -- all algorithms are local; all AI is local -- and have it  fit and tuned to the local environment. And that includes the local voices. We're working  now with indigenous nat

ions, and they have put together such ironclad principles about  how they want their data used and done, how they want their artifacts regarded,  adopted by the Smithsonian Institution. And so, I want to see us begin to permeate all  of the -- when we write an implementation manual, when we write an implementation guide,  I want it to talk about not just what you and your own silo should be doing but  what you ought to be doing in relationship to the other people. And I want that  written all th

e way through from the highest levels of governance down to  the local community health center. But I think if we can help them develop those  guides, give them templates and models for that, nobody knows their patient populations  like some of these places like a community health center. They have some of the best  opportunities in the world to make this and tune this toward the true patient needs. So,  I'd say that that's what we're looking toward, and we've got a lot of work to do  ahead, tho

ugh. I understand that. Daniel Yang: All right. Well, thank you. So, moving on to our last speaker,  last but not least, Dr. Maia Hightower, CEO and founder of Equality AI. So, Maia, the  question that I have for you is, you know, the theme throughout the day was really around  inequities and risk of algorithmic bias. We know it's a critical issue. We know it's an issue that  policymakers are aware of. But sometimes it can feel like there's this massive chasm between  acknowledging the problem a

nd actually doing something about it. So, can you illuminate  us on what policymakers, what health systems, what everyone should be doing to ensure that  the AI tools we're using are fair and equitable? Maia Hightower: Absolutely. That's what I'm going to talk about  today. It's about being pragmatic, really making sure that when you  are implementing your AI systems, that you have a strategic alignment. And that  strategic alignment includes health equity. So, AI and health equity are not two s

eparate silos or  different parts of your strategy but intertwined. And the second is really around governance.  No matter where you are in your governance structures, you can actually repurpose  your existing governance structure. You don't have to pull in, you know, a ton  of different experts as long as you use what you've got, and then grow. So,  AI governance is extremely important. And then the third pillar is measurement,  whether you're using an external source to measure or internal cap

abilities. But  you have to measure the impact of what you implement to ensure that you actually are  achieving the outcome that you seek. So, that's what I'll talk about. And  [laughs] I'm going to stand over there with my little pointer because it's  actually easier for me to stand over here. But I did want to -- for each of us to  take a moment and pause and think about the last time you saw your doctor, the last  time maybe a loved one was in a health care system. Maybe you were in an exam r

oom,  and you had one of those, you know, cute, attractive robes on that exposes your  derriere. And your doctor was probably sitting in front of a computer typing  away. Maybe now there's an ambient system. But how much did you trust that the AI behind  the curtain actually was personalized for you? How much did you trust that your loved one in  the emergency room or in the operating room, that the AI behind the curtain actually  applied to their specific circumstances, the data, the personaliz

ation,  their genomics, their labs? And for the vast majority of us, you  know, given the right circumstances, we each are at risk of being an outlier. We  are each at risk of not being in the middle of the bell curve that that particular  model may have been trained on. And so, that's why, you know, making sure that we  are measuring bias that we are addressing is so important because we have a long way  to go. AI truly has the potential to be our pathway to personalized precision medicine if 

we do it right. We are at an inflection point. We've talked so much about the promise of  AI already. Again, it really does have the opportunity to improve quality of care, drive  down costs, and create these amazing experiences, amazing experiences for our patients, amazing  experiences for our providers. We just had the whole panel talk about how we're so close  for the EHR actually to be a joy. Never in my career did I think that would be a possibility  since I've spent my whole career wearin

g like a flak jacket when it came to the EHR. But  to be so close, yet we have so much to do. When it comes to health equity, you know, Martin  Luther King said it best. "Of all forms of injustice -- inequity, injustice in health care  is the most shocking and inhumane." And in the last 50 years, what have we done? Have we done  anything since Dr. King said this statement? No. We continue to have the same inequities we had 50  years ago today. So, do we have this opportunity with AI to instead o

f widening the chasm,  to actually close it? And I do think we do if we implement our AI systems responsibly with  health equity intertwined in our AI strategies. And the reason why we know that there is a risk  of widening health disparities is because we have caused harm with AI already. We have widened  disparities already with AI systems. Example, Obermeyer et al., many of you probably are  familiar with the work of Ziad Obermeyer, but he was able to measure the impact of a biased algorithm

that was deployed in hundreds of  healthcare systems. Probably at this point, has been calculated to have been exposed to  80 million people that decreased resources, case management referrals to Black patients  compared to White patients. Black patients, while being equally sick, had 50 percent  less referral rate for case management. The good news is that it was detected and  mitigated, that we do have that capability of detecting a bias within a model and  fixing it. We no longer have to have

a model be deployed to 80 million people or 80  million exposures before we recognize that harm has been done. And so, bias occurs.  And again, this is sort of repetition from earlier today. We had that beautiful  example from our colleague from Emory. But bias occurs across the AI life cycle from  those that have the ability to ask a question, so problem formulation. Not all  problems have equal voice. If we asked our community members what is the most  important problem they want solved from

AI, it probably isn't to improve their revenue  cycle -- rev cycle management or auto denials, right? Yet that often is where a huge amount  of AI resources is currently targeted, is in rev cycle, in cost reduction, in process  improvement, which is also very important. But bias occurs from the moment of problem  formulation to the real-world data that is embedded within our EHR, to how that data  is acquired, to the way that the model is developed. There are thousands of decision  points in mod

eling in and of itself. In the Obermeyer example, the error that was  created was called a labeling error. So, they used cost as a proxy for risk. And we  know systematically that African-American patients spend less health care dollars  while being equally sick. And so, that was the actual error that occurred. So,  that occurred during the modeling process. The data in itself, the data set actually had  plenty of higher-quality labels that would have produced -- that ultimately produced a bette

r  model to the way that the model is evaluated and to where -- how it's deployed. We had  an example earlier today about a punitive, a negative model versus a positive  one, where a very neutral model on an AI -- a very neutral model on no-show  prediction, you can either be punitive as a health system and double book or you  can be assistive as a health system and provide resources that actually addresses why  somebody might be at high risk for no show. So, bias can occur throughout the AI lif

e cycle,  and the great thing is that bias mitigation can occur throughout the life cycle. And so, when  it comes to different mitigation techniques, we have a combination of social methods. So,  some of them do not require any technology in and of itself. You can start with diverse teams,  with AI governance, you know, with our regulatory environment, with applying some of these  principles that we have talked about by my panel members, all the way to more technical approaches  where you can ac

tually dissect the data set, you know, the model itself, and measure for  precision and performance by subpopulation. You know, for that same instance  of, say, you go to the doctor, and you want to know if it's personalized to you,  you could actually ask the folks at Kaiser for the performance of the model by subpopulation.  I heard that this morning, which is fantastic. Is it accurate if you're African-American? Is  it accurate if you're Asian-American? Right? And then the fairness of it. Is

the distribution  of resources equitable across subpopulations? Are the mammogram referral rates that this  model may be triggering equitable? Does it match the demographics of the people that that  population is serving? So, those are some of the very technical and can get way into the weeds  on the technical approaches to bias mitigation. But what that leaves, is this challenge, right?  There is -- when you talk to healthcare -- when I talk to healthcare leaders, they're feeling  this overwhel

ming sense of competing priorities. "Where do I start?" And that's really where --  again, three simple steps: your AI strategy, making sure that your organization is prioritizing  what's important, including health equity; and with that role is your AI governance, making  sure you have an AI governance system in place; and then holding your AI governance  system accountable through measurement. You can measure through an audit. You can  measure through technical methods. There's a lot of ways o

f measuring, but measurement is  so important. So, the way that we approach it at Equality AI is really helping health systems  find that alignment by domain and making sure that their AI strategy really does align to  what's important for that healthcare system. And then AI governance, there's a lot of different  frameworks for AI governance. Personally, NIST and ISO, yes, it may be dizzying. If  you think 42 pages for NIST is extensive, what about the 200 pages for the  ISO standard [laughs]?

But these are -- there are more simplified versions.  You just talked about Mark Sendak's work. That is the Duke work over on that bottom  right-hand corner. And then Obermeyer et al., Ziad Obermeyer has his own playbook, the Algorithm  Bias Playbook that Booth UChicago has published. And then the model, when you audit a model, you actually audit by two different  methods: the technical approach, but also process. So, did the model go through  the AI governance process as it was intended? So, we

each have a role to play  regardless of what hat you wear. If you're a healthcare administrator, your  role is to make sure that from the top, that your AI strategy includes health equity,  that you're appropriately resourcing your teams to provide AI governance policy  accountability within your system, and that you're actually checking to make  sure that those processes are working. If you are a doctor, if you're a clinician, it's  being a part of those AI governance committees, making sure y

our voice is part of the  solution; if you're a patient advocate, that you too are part of AI governance  and that your voice is being represented; if you are a policymaker, making  sure that we have some guardrails, we have some foundation. Because right now,  it is very confusing for healthcare systems. And so, I'll just -- the call to action  really is straightforward; again, AI strategy aligning with your health equity goals,  governance, and then measure. So, that's it. [applause] Daniel Ya

ng: All right. So, let me open it up. Laura, Tom. Laura Adams: I loved it. I have -- what I think I heard you say is that  AI is the problem and AI is the solution -- Maia Hightower: [laughs] Laura Adams: -- with regard to its ability to detect and to help mitigate bias. So, the very problem  is sort of creating -- well, it's not creating. It was created before. But it scales and exacerbates  the problem can also be part of the solution. Can you say, Maia, more places where you think AI  will be

the solution to some of these super intractable long-term problems that we've had very  little success in actually, you know, addressing? Maia Hightower: Well, if you were to ask the folks at Microsoft and Google, they'd say the environmental  issues. But the same -- like one of the biggest challenges of AI and big data and high-performance  compute is the incredible amount of energy. We don't talk about -- we haven't talked about the  incredible amount of energy it consumes. But the argument i

s that, as well, that there may  be some innovative solutions that AI is able to generate that may solve some of our climate  concerns. Not saying that it's true [laughs], but besides health equity and healthcare in and  of itself, precision performance medicine. Tom Romanoff: And also, water, consumes a lot of water as well. Laura Adams: Yeah. Maia Hightower: Yeah. Tom Romanoff: So, I'm, you know, very cognizant of the lack of trust that  some companies have with policymakers in terms of the vo

lunteer solutions that are  out there. How do you make this enforceable? Maia Hightower: Yeah. The only way that you can make it enforceable is through regulation. I do think  you talked about the use case approach within healthcare. There is an incredible opportunity  for HHS and the various HHS agencies to use the executive orders that they've been given to  actually implement regulations that are very targeted. So, ONC has done a great job where they  now -- their proposed rule has become a f

inal rule by which EHRs are required to have certain checks  and balances in place when it comes to AI systems. It will be -- the FDA is moving in a similar  direction when it comes to software as a medical device. They have guidelines right  now. And many of -- much of pharma are already adopting some of these guidelines when  they are using software as a medical device. A lot of healthcare tech companies want to be  FDA-certified as a marker of good citizenship, so similar to, you know, in env

ironmentalism,  to be a good environmental partner. And so, I think that there is an opportunity at  that very targeted HHS agency level to have meaningful regulations  sooner rather than later. Daniel Yang: All right. Well, we're going to transition to a moderated  Q&A and then an open Q&A. But actually, I wanted to -- because, Laura, you stole one of  my questions around the positive uses of AI for addressing inequity as opposed to exacerbating  it. And for me, my favorite example is actually

also from Ziad Obermeyer and Emma Pierson  around predicting pain from an x-ray image of a knee film. And so, everyone knows Ziad's work  around the algorithm around case management that was biased. But he also did another study that I  loved, which is they simply trained an algorithm to predict a patient's subjective report of  pain, which they happened to be able to collect. And what they found was that human radiologists  were very biased in identifying or accurately assessing pain in Black p

atients. It's partly  because they were trained in kind of severity levels of osteoarthritis based on, you know,  traditional cohorts of patients in the past. And the AI algorithm was not biased by those same  training approaches. And so, they were just much better at accurately predicting subjective pain  in Black patients than human radiologists were. And so, in the same way, I mean,  the AI is intrinsically -- you know, it's as biased as we want it to be.  And so, a lot of it is in the proble

m definition. But there are a lot of examples  there if you're training towards outcomes as opposed to training towards human  interpretation that you can actually get less biased results. But let's transition,  or people may have thoughts on that comment. All right. Okay. So, Tom, I have a question  around regulatory burden, and Laura, you actually mentioned this in your remarks, this concern that,  yes, we need regulation. We need guardrails here. But if the burden is a little bit too high,  i

t's one thing for Kaiser Permanente -- I mean, we've got a lot of people. We've got just  tons of expertise when it comes to compliance, when it comes to legal functions, when  it comes to the technical expertise. So, you know, we're welcome. We're happy to kind  of, you know, to excel in those environments. But I worry about what Laura was describing.  You know, care oftentimes happens in the non-, you know, Mayos or Stanfords or Dukes of the  world. How do we provide policy solutions that work

for them that doesn't actually kind of  create the system of AI haves and have-nots? Tom Romanoff: Yeah. It's a great question, and I think it kind of gets to the heart of the debate currently  with this. You know, the folks that are incumbent dominant players in this space and the folks  that represent industries that are already highly regulated, they know how to approach these things.  They know how to either create products that meet compliance or, you know, that are catered towards  the re

gulatory environment that they're in. And the smaller folks and the ones that are  startups, they don't have either the capital or the expertise on staff. And so, that  is a debate that happens quite often is, you know, how do you make sure that you  are addressing some of these issues around, you know, very real negative  externalities while at the same time, not favoring the big incumbents and the  regulatory folks that are in the space? And I think that's really hard to do. You know,  I strug

gle with that question quite often, but I also struggle with the  conversation of, you know, what we would -- what would happen if  we don't do anything, right? And so, if we completely ignore this idea and kind of  embrace nonregulation as the fuel for innovation, well, then we get some negative use cases  around, you know -- I'm not going to name names, but insurance companies' unit for end-of-life  care, whatever it might be. And that was a really bad outcome that I think drives the policy  q

uestions around, "Well, we can't do nothing." One area that I think we can look at is a  lot of tech companies, they don't want to be the ones that come out and say, "This is  what we need to do in this space," same with, you know, healthcare, I imagine. Though I'm  not a health care guy. And they want to codify what they're already doing because they build  processes and teams around what they're already doing. And if they can standardize that,  then that means that they can standardize it for

a lot. They can minimize some of  the risks that they have in that space. And I think there's something to be said about  that when it comes to the voluntary commitments that a lot of folks have already talked about. The  White House came out on what NIST is doing and the huge amount of support that's happened there.  And so, there are kind of these indicators of, you know, different regulatory practices that  won't inhibit further development of this. Daniel Yang: Laura. Laura Adams: I would lo

ve to see -- when we think about when we needed EHRs to  get implemented, I mean, Economist magazine had ranked us second only to mining. Healthcare,  second only to mining for lack of investment in health IT. You know, we like technology.  We can look at every molecule of the human body. We just couldn't get your lab test across  the street even if your life depended on it. So, when I think about the approach the  federal government used at that time, it was to ask ONC to put together, and we p

ut  together this in the HITECH Act. Every single solitary physician provider out there got up  to $63,000 per provider to do the transition, to do what it took to get up to speed,  to understand, to acquire technology, and do those things. We also had things put  together under that same act called the regional extension centers. You know, these are like in  agriculture where you go out and you help small farmers understand the newest technologies,  the newest science to go on with things. I th

ink we need to begin to replicate that and  put that back in place for all of the places that do not have the resources. Create regional  collaborations, create vaults and banks where we can start to see how algorithms are functioning  in critical access hospitals, create communication nexus where they can begin to share. I loved what  Julia said about, "We're working with some other healthcare facility over here. They're going to be  testing and trying this one. We're going to test and try this

one over here. And then we're going  to combine our learning." Let's do that at-scale, and let's do it directed toward those that  do not have the resources to do it. I think we can replicate the regional extension  centers and really take a run at this. Daniel Yang: What about you, Maia? Maia Hightower: Yeah, I love that idea of recreating the HITECH Act because I was a primary care doctor  in private practice when we implemented the HER, and I got one of those cool checks. Like,  it was like

literally like $30,000 -- [laughter] -- at the end of the year. And I was  like, cha-ching. I was already over my margin for the year. When you're in private  practice, you know, every dollar counts, so definitely recall that. But I do think  that there is plenty of opportunity to, again, align, say, HHS with whether its  meaningful use ONC and meaningful use type of approach or just plain CMS and  CMMI having some sort of incentivized projects and innovation projects around  appropriate, respon

sible AI implementation. I think there's plenty of opportunity for both  some sort of carrot versus stick approach and definitely starting with carrots. That has  always been very helpful in healthcare is when they start with carrots, incentives. And  then, you know, by the time 10-15 years goes by, the penalties start kicking in. But there's  plenty of frameworks that HHS and ONC and CMS can use as examples and frame a  similar type of policy and incentives. Daniel Yang: My next question actual

ly goes back to you, Maia. The last time that we met,  you were still at the University of Chicago -- Maia Hightower: Yes. Daniel Yang: -- as the EVP and chief digital transformation officer there.  You've since gone, you know, and full time into this startup. And you talk about the importance  of AI governance and getting this right. And so, it's a bit of a meta question, but can you  simulate the conversation between Maia, the physician executive that has tight margins  that are just coming ou

t of COVID and, you know, were told to cut costs, and then, you know,  Maia, the startup CEO that's trying to sell into University of Chicago of why we need  this. What does that conversation look like? We could probably, I don't know,  simulate that in ChatGPT as well, but -- [laughter] Maia Hightower: I think the conversation really is, "Okay, what's keeping you up at night?" And  I've spoken to many healthcare IT leaders, and what's keeping them up at night is  the loss of focus on health equ

ity. This sense that we've lost -- that the demand  for increased productivity and decreased cost has taken away from the overarching  mission. If you ask healthcare IT leaders, those that were in the throes of the pandemic that  had this huge rallying cry with purpose around the pandemic and health equity, now, it feels  like we've made this about shift and no longer is equity and quality and all the rest of the  mission so important, but we have to drive down costs and increase productivity. W

e need another  tool that's going to help our providers, you know, see more patients. That's not -- sometimes  it's fun as a wellness tool, but in the back of the head is like, "How are we going to pay for  it," which usually means increased productivity. So, what I -- my conversation to the executive  me is wouldn't there -- there is a possibility for aligning your health equity and quality and  patient experience mission that has always been part of our mission with AI, without the AI  being a

detractor and just about increasing productivity, decreasing cost, and potentially  the cost of some of our employees, that there is this bright side if we make these choices around,  you know, setting up our AI strategy in the very beginning to align with health equity, that we  continue to -- that we have AI governance, which AI governance is just a fancy word for clinical  decision support governance. You already have clinical decision support governance. Now you just  need a few extra exper

ts. But it's not that much different. And then really making sure that you're  measuring. So, the conversation that I have is -- Daniel Yang: That you're having in your own head. Maia Hightower: -- in my own head is how do you get the joy back for the poor IT people? Everyone talked about the  joy of medicine for the clinicians. Believe me, our IT teams are so tired. They are -- they've  been, you know, adapting to change for the last four years nonstop. The clinicians were the  heroes, but in t

he back end, the IT teams were basically duplicating, creating whole hospitals,  whole health systems digitally. And so, of course, I got to hear that because these are my team  members, and yet nobody was, you know, waving the hero flag for our IT team. But they, too,  have been through a lot in the last five years. Laura Adams: I saw a meme yesterday. It was clearly maybe an 80-year-old  gentleman's picture, and it said, you know, "IT is the fun, exciting place to be," you know,  like Mark, 28

, you know, years old. And clearly, this was a picture of an 80-year-old  man, and they were assuming this was a 25-year-old person making this statement.  So, there was the picture about that. I think you're bringing up something  really critical that we would be remiss to close out a conference  like this without talking about, how are we incented? Because I have heard  people say we can't wait to use Nuance, Abridge, whatever the ambient -- picking it  up and being able to reduce the provider

time. And the providers are whispering, yeah,  so they'll add more people to our panel. We're not going to save more time, spend more  time with our patients. This is not happening. The thing why I will drop everything and speak  at a Kaiser conference above all is because you have the payment model figured out for your  patients. I wish I lived near where my family could be cared for by Kaiser because I worry  so much about our incentive model. The reason I worry about it is because it isn't a

ligned with  the patient's best interest like Kaiser's is. You're self-contained. You're integrated. You've  got it all, payer all the way through to delivery. When I think about -- I was in Rhode Island, and  at our Health Information Exchange, we were able to build capabilities onto that system where  we could identify and notify a provider in real time in the primary care setting that your patient  has gone in or out of any hospital in any state, any part of the state in Rhode Island or in  a

nd out of any ED. They could give us their high-risk panels and say, "Track these people  so our care managers know in a nanosecond." We watched people be able to be healthier, be kept  out of the hospital, get things prevented, and one of my CEOs in that state shut it off. And so, when  I went to see him, I sat down. I said, "Hey, John, you shut off the system. Why did you shut it off?"  And he said, "I had to, Laura, because it works." He said, "The system keeps these patients  out of the hosp

ital." And he said, "Frankly, I run the whole health system, and I can't  keep my hospital open unless those people are sick and admitted." He said, "So, I've got  to have them sick, so I can have that revenue from the admissions, so I can keep the other  services going." He didn't say that with a blank look on his face or with a half-smile on  his face. He said it with anguish on his face. And my sense is that we do have a health incentive  system here that doesn't set us up to do the best outs

ide of Kaiser. Because you, again, have your  incentives aligned. The rest of the world doesn't, and I don't think AI is any solution for that.  I think we've got to take that on head on. Maybe AI can help us with that, but it's no magic  bullet for it. We've got to summon the will to move toward value-based payments models  a whole lot faster than we already are. Daniel Yang: I can look at Tony and like, "We're definitely  inviting Laura to the next panel discussion." [laughter] Laura Adams: We

ll, I'll drop everything and show up because I want  everybody to be aligned like you are. Daniel Yang: Thank you. Well, let me open it up to the audience Q&A. Female Speaker: I think it's come up a few times today. Like, can we get access to  broader, better data sets that can be sort of available to anyone that wants to build  a model and train a model on it? I know it's just one of the steps along the way, but  it feels like a really important one. And so, could you just comment on the pathwa

y to that  endpoint? Like, what would it take for Kaiser to share all their data with anyone that wants  to develop a model on it? And is regulation the only way to get there, or do you see any sort of  private sector models that might get us there? Daniel Yang: I mean, maybe I guess I'll start in my KP role. You know, what really attracted  me to KP, and I think many people here as well, is -- I mean, we do really have quite remarkable  data sets. Vivian spoke about them. And one of the things

that I learned about KP is that  not only do we have deep and broad data, we have longitudinal data. You know, the average  Kaiser member stays with us for 11 years. And so, when you're really trying to look at  outcomes, we've got that outcome data. So, one is, as Laura mentioned, there's  another event, the Coalition for Health AI and this whole concept of quality assurance  labs. And, you know, how do we build the wire cutter or the consumer reports for AI in an  independent kind of third-par

ty validator service? When I look at the organizations  that are lining up to fill that role, I see the usual suspects, you know, Mayo,  Stanford. Probably UCSF is on that short list. I think KP is a great place to play. You  know, I need to make the case internally that we could. But when I think about a lot  of the issues around diversity of data sets, we've got that. You know, our 12.5 million  members across eight states and the depth of the data we have looks a lot more like  the rest of Am

erica than Palo Alto does, you know. And Stanford delivers great  care, but it just -- it doesn't look like the care we deliver in Oakland or  the populations that we care for there. So, you know, I -- but Kaiser is just one  healthcare system. I think there are others, you know, that similarly reflect that diversity.  I would love to see county health systems and rural hospitals contributing data. And  so, if I put my Moore Foundation hat on, we were funding in that space. And  so, just to give

you one example, we gave a grant to Contra Costa County, their  health system, to make their EKG data available for public use. And it took one year for  the data use agreement to get signed. So, of our grant dollars, not a single dollar  was spent because they've never signed a data sharing agreement. They didn't even know  the lawyer that had the authority to sign it. And so, I do think that there are huge almost  infrastructural gaps around making data available. It's easy to talk about it f

rom the stage  that health systems should be contributing their data. I think the challenge really is  that, you know, the academic medical centers, there is very much an intrinsic motivation for  making their data AI ready, for doing research on it. We don't have that necessarily in the  other care delivery systems. And so, I'd love to see us move forward, but I do think that there  are big structural issues that stand in the way. Laura Adams: I think that it's going to become infinitely  easie

r to get a hold of data sets soon, but only if you have money. Peter Lee, at a  recent conference that we had with Harvard, was on the panel with me, and he said, "Here's  what I see for the future. Hospitals haven't had a very good opportunity to monetize the data  that they have." And that was pre-AI. All bets are off now that this is going to become suddenly  super valuable data for those. It will become, from the hospital's point of view, "Oh,  my God, do I have a revenue stream here? And I

think that -- I think back to Michael  Millenson's point that if data is the new oil, then patient rights and privacy is climate  change. And I think we're coming up on an era where we're going to see a shift  in that for money, and I'm very, very worried about the imbalance and how that  will play out. Because I do think that's going to be suddenly an issue for patient privacy  rights where organizations are going to be willing to give up because they're dying for  the resources that they need.

And we've got a lot of conversation to be had in that arena  before that future comes, and it's soon. Maia Hightower: Absolutely. And I think we need to remember that each data point in healthcare is  a digital representation of a person's experience. Laura Adams: Yeah. Maia Hightower: So, who really owns the data and health systems were just fiduciary,  you know, managers of patient -- individual patient data. And so, until we figure out how to  actually adequately protect a patient's privacy

or the patient's wishes when it comes to their  data, I think we're going to have, you know, whether you call them structural barriers,  but I also think that they are protections, right, for who really is the owner of  that data when it's your digital twin. Tom Romanoff: Yeah. I'm not a healthcare guy, so I might butcher this.  But, you know, I was thinking about that in terms of digital twinning and, you know,  the case of Henrietta Lacks and, you know, the fact that the family took generation

s to get  a retribution for that. What does that mean in the space of a digital health world where, you  know, it might not be one person. It might be multiple -- thousands of people that ultimately  have their data used for some long-term outcome. Daniel Yang: Yeah. I think the flip side though, you know, one of my colleagues at KP, Vinnie Liu,  loves to say, "Not an ounce of data wasted." And so, you know, our patients entrust us with  this health care data, but they also trust that we're lear

ning from it and that we're  using it for good purpose as well. And so, I think most of our patients, again, if we  were to survey them, would say, "Please, like we want to advance care delivery. We want to  advance the state of the art. We want better care for ourselves. I want more personalized care.  Use my data to make my care more personalized." So, I think there's always a double-edged sword.  We certainly care about privacy. We certainly care about data rights. I also think that, you  kno

w, our patients are expecting us in some ways to really make sure that we're leveraging  this precious resource for greatest impact. Maia Hightower: Absolutely. And that would come through consent  as well as transparency. Daniel Yang: Yeah. Maia Hightower: Right? So, if you have the systems in place, then you can  definitely ensure that you're appropriately using the data that has been consented for  that use. And discovery is part of that. Laura Adams: I totally agree. It's like donating blood

[laughs], you know. It's just doing  good for the system and doing good for the whole. Daniel Yang: All right. Well, with that,  we're going to close. Oh, okay. Male Speaker: I'm just going to make one comment on this. Daniel Yang: Yes. Male Speaker: I really appreciate all of the comments around this last question  and, Julia, your question about this, which is a really, really important policy  question. We've had a number -- we've had a large number of sessions in this space that  the instit

ute has put on, around drug pricing, and at the central -- and there's a whole  bunch of really identifiable reasons why we have problems with drug pricing. And at the root  of a lot of that is the failure to think through the economic model of technology transfer  and what's happened through universities to private ownership of the goods that come  out of that to how it's been on the market. I would suggest, Daniel -- and maybe you and  I can both have this conversation with our colleagues -- t

here's policy to do in  this space. How do we take this good, which is this data, and how can we make sure  that it is used exactly the right way and doesn't create a kind -- you know, a private  ring fence that then winds up being things where people start not doing things because they  haven't solved the intellectual property problem? So, you know, how long it takes, contracts to be  signed, I mean, these are all the same problems. And it's something that all of us, I think,  can work on toget

her and think -- if we think of it as a policy problem before somebody puts a  solution forward that creates a reality we'll be living with for another 50 years. So, I think  excellent question, lots to do in this space. Daniel Yang: Thank you. All right. Well, great comments, seems like maybe a potential  topic for the next IHP meeting. But I want to thank my panelists here for your  great remarks and insights. Thank you. Laura Adams: It was great to be on the panel with you. [applause] Daniel

Yang: All right. So, it's me again. I've been asked to give some  closing remarks for today. Thank you. I mean, first of all, I just want to thank all of our  panelists, our moderators, our organizers for hosting such a wonderful and insightful event.  You know, I've got AI in my title. I'm literally paid to think about this all day long every day,  and yet I've learned so much, really, from all the people here. And it reminds me of what Julia  said earlier on. You know, sometimes we need to kin

d of stand outside of our own silo because  we have so much more to learn from each other. So, I've been working in clinical AI for the last  decade, but I've witnessed, and I think all of us have witnessed an incredible transformation in  just the last 12 to 18 months. For many years, AI has been focused on very narrow but important  clinical use cases, for example, trying to identify diabetic retinopathy from an image.  But when you talk to patients and providers, these were often viewed as ju

st incremental  improvements in care, not transformative. And this was evident when you talked to  our providers. When you talk to providers about clinical AI, the general  reaction that I got was one of skepticism. And many providers even felt  threatened by AI. They felt that it was threatening their sense of professional  competency, their sense of autonomy. But something really changed with the advent of  generative AI. Suddenly, we have general purpose AI tools that are showing impressive p

erformance  across a very wide range of use cases. Moreover, these tools are starting to solve some of  the quality-of-life pain points that have always -- that clinicians have always struggled  with, things like administrative burden. And what surprised me most is the physician's  change in attitude toward these tools, from one of skepticism to one of delight. And  now, what I'm facing is one of incredible demand. Now, the promise of these AI  tools to me is, paradoxically, they make medicine m

ore human again. These  ambient scribes that we talked about allow providers to turn their attention away  from the computer screen and back to patient faces. It gives providers more  time at the bedside or in the exam room. And so, hearing these benefits, you might think  that the natural approach is to move forward as fast as we possibly can. But I think  that to move faster and to move further, we have to proceed with caution and vigilance, especially now. Why? Because I believe  in these too

ls, and I also believe that nothing slows innovation more than patient harm  or algorithms that deliver -- that promise more than they deliver or biased algorithms that  mimic our history of inequities or biases. Now at KP, I have an incredible privilege and  responsibility to help lead our organization's program on responsible AI. For us at KP,  responsible AI is a journey. It's not a destination. And just like everything  at KP, we strive for excellence, not just compliance. And so, what does 

excellence in responsible AI look like for us? First, it starts with culture, and it starts  with system guardrails. So, I come from the world of patient safety and healthcare quality.  And if there's one thing that I've learned from that environment, it's that to err is human.  And so, we have to apply that same approach to our AI algorithms. Our algorithms,  like humans, are intrinsically fallible. There is only one way to prevent harm  and to prevent bias in health AI, and that is to never d

eploy it at all. But there's  a risk in that approach. If we all believe in the transformative positive benefits of AI, we have  to acknowledge that there is risk in deploying AI, but there's also risk in not deploying AI,  the opportunity cost. The only way that we can manage this tension is if we move forward  with a culture of responsibility that travels from frontline providers all the way to the most  senior executives. We also approach responsible AI as an organizational muscle. We underst

and  that our providers are extremely busy. And so, they look to us to do rigorous QA testing  on these algorithms to identify risks and mitigation strategies, to provide actionable  education on how they should be using these tools, and to create system guardrails that prevents  harm from happening in the first place. But we also realize that our upfront risk  assessment and mitigation strategies just provide preliminary guidance. We also have  to learn by doing. And in order to do that, we nee

d the discipline and the infrastructure  to monitor the performance of algorithms we've already deployed, looking for drifts  in performance, the introduction of new biases we hadn't thought about. We need the  discipline to quantify the real-world impact of our AI algorithms, what I describe  as these algorithms' return on health, and with the infrastructure to flag and  remediate patient harm before they occur and to learn from these system errors, what  Laura talked about as a learning health

system. So, as I said before, the work for responsible AI  is never done. It's a journey, not a destination. And so, in closing, I invite you all to join us  on this journey. If I've heard one thing today, it's that the stakes are too high for each  of us to be doing this on our own. I really hope that industry, policymakers, health  systems like KP and others can partner and share their expertise and their perspectives  as we think about national frameworks for advancing responsible AI. We've

got so much  to gain from learning together. Thank you. [applause] Rebecca Flournoy: All right. Well, thank you so much for sharing those reflections. I think you're  leaving us all inspired, which is fantastic. So, for me, today's conversation, I think, is really  a reminder that there is so much work still to be done, but it also makes me really glad that there  are so many thoughtful leaders bringing their insights and perspectives to the table. And so,  we look forward to continuing that con

versation. And on that note, I want to say a huge thank  you to all of our speakers and moderators today. Such a fascinating -- so many fascinating  conversations. Thank you also to our IHP team, Ceci, Renee, Nicole, Nani,  Krysten, and Ben. Thank you. [applause] And thank you also to our forum advisory  group for all their really helpful advice and assistance. And then thanks also to  the team who helped us host today's event, including consultants Maryland  [spelled phonetically] and Wendy, th

e Center for Total Health, Spark Street Digital,  Kian [spelled phonetically], and Ridgewells. And, of course, a huge thank you to all of  you in the audience. So, thank you. Before you leave, we hope that you will  take just a couple minutes to take the event survey to give us some feedback. It's  very helpful for future events. It's on the event page that you access through a  QR code. And if you need that again, it's up on the screen. And then we will be  posting a full recording of today's e

vent. So, we encourage you to share that with others  who may be interested but weren't able to attend today. And this concludes our  forum today. Thank you, everyone. [applause]