AI Impact and Blowback
Presenter: Dennis Bergau, CEO, KarmaSci Scientific Consulting, LLC
[The following blog was generated by Doug McKell and Akshata Naik]
Dr. Bergau presented a brief overview of the existing AI landscape, followed by a more detailed review of evolving AI’s impact on medical education and the potential downstream blowback related to unintended consequences that may be overlooked, ignored, or unforeseen. Throughout his presentation, he emphasized the need for all users to adopt a critical evaluation of the existing and developing AI tools and technologies by acknowledging the imperfections that all systems have based on inherent biases and limits on the data they use to provide requested outputs.
Dr. Bergau covered the following areas during his presentation: How to identify and avoid AI reinforcing human biases in medical diagnosis and treatment, how to decide what should be the guardrails for using AI in undergraduate medical education, how to best educate medical students and faculty about AI, including whether to teach the technical details or focus on specific AI tools, the need to consider the environmental impact of increased AI usage and incorporate it into the cost-benefit analysis when discussing AI in medical education, consideration of ways to democratize access to AI tools and resources for medical students across different institutions, the opportunity to explore the use of digital twin concepts to teach medical, nursing, and other health professions students.
Dr. Bergau began his presentation by explaining AI by functionality, including reactive AI, limited memory AI, theory of mind AI, and self-aware AI. Dr. Bergau used an example of a comparison of five different models to illustrate the variability in AI model performance. He then discussed the concept of generative AI and its evolving applications in generating various forms of content. He specifically highlighted the role of transformer models in large language models (LLMs) and the concept of foundation models. Dr. Bergau discusses the potential and limitations of AI, including the idea of hallucinations in AI-generated content. He also emphasized the importance of setting realistic expectations for AI uses and performance, noting that there is a life cycle to all innovative advances elements. He then reviewed the benefits of AI in healthcare delivery, including faster drug and medical device development and assisting providers in accessing additional information for diagnosis and treatment decisions. He stressed the importance of not letting technology dictate societal norms and the need for flexible and applicable AI programs.
Dr. Bergau shifted his presentation to address AI’s existing and future challenges, including the evolving pace of technology, the need for ethical frameworks, and the potential for unintended consequences. Dr. Bergau emphasizes the importance of collaboration between IT and healthcare professionals to address AI biases and improve model accuracy. In response to a question from a webinar attendee, he also highlights the environmental impact of AI, including increased energy consumption and the need for sustainable solutions. One of the biggest challenges he sees in the future is the increased complexities of bias in AI resulting from human-generated data sets and the need for more and better representative training data sets to avoid unfair or unethical biases in the widespread application of AI tools and technologies.
Dr. Bergau discusses the need for AI literacy and the importance of understanding AI as a tool rather than a solution. He acknowledged the need for a common language between IT and healthcare professionals to integrate AI effectively into healthcare education. Doing so requires a better understanding of each discipline’s approach to the sources, quality., analysis, and expected utility of the information AI provides the user. He recognized that there are increasing questions about the future of AI in medical education, including the potential for AI avatars and the balance between personalized and generalized data. Central to this discussion is the need to decide how to educate entering medical school students about AI and what degree of knowledge (or competence) is necessary for their future practice experience in
clinical rotations and residency. Dr. Bergau stated that AI should be taught as a tool rather than a comprehensive subject, focusing on intelligent interaction with AI.
Looking forward, Dr. Bergau outlined the following considerations of AI in health professions education: The need for a partnership between AI users and IT/curriculum development will require developing a common language; AI literacy means going from using “buzz words” to a more accurate understanding of what AI is; Expert AI users and developers must be able to explain some of the complicated AI concepts to create shared knowledge, content balance, and transparency; It is critical to remember to focus on the problem you are trying to solve, not the AI solution, since AI is just one of many tools to be applied in healthcare learning and healthcare delivery; The application of and outcome from AI is different in biological or clinical uses that in predictive mathematical applications.
Dr. Bergau concluded by stressing that we must appreciate the different expectations and uses of AI applied to industry clinical trials, academic medicine, community medicine, UME, and GME, including different countries’ medical school curricula and learning formats worldwide. Finally, he challenged us to consider if and what programming skills/experience/ knowledge will be required competencies for all healthcare providers to use AI effectively in the future.