Synthetic Actors & Conversational AI Agents as Training Aids

AI is transforming the educational landscape. Large language model (LLM) technologies offer promising applications in synthetic persona generation for training and simulation across disciplines. The integration of synthetic actors and conversational AI agents into assignments enhances student engagement and streamlines instructor workload. Rather than passively watching training videos and submitting written responses, students and trainees can interact with dynamic, conversational personas that mimic real-world scenarios. Instructors, in turn, can focus on providing individualized feedback instead of monitoring for AI misuse.

CAAI’s templated module for generating synthetic actors and conversational AI agents provides a scalable, interactive solution. Built on LLM Factory, we leverage training adapters for domain-specific customization, ensuring adaptability across different educational contexts. Human-in-the-loop feedback mechanisms further enhance model accuracy and relevance.

Associated projects include collaborations with the Center for Innovation in Population Health (IPH) and the Office of Medical Education (OME). For OME, we built a web platform that generates synthetic personas, giving medical students a unique way to practice assessing decision-making capacity in informed consent conversations. Guided by the U-ARE framework (Understand, Appreciate, Reason, Express a Choice), students interact with patient personas that may be cooperative, confused, or resistant. Students must then determine, based on their back-and-forth text interactions if the synthetic patient has decision-making capacity to consent to a proposed treatment.

The impact of this work is for the benefit of both educators and students. For educators, it provides a non-repetitive and assessable environment to test students’ capacity assessment skills. For students, it builds confidence, sharpens communication skills, and improves understanding of informed consent conversations.

For our prototype development in collaboration with OME, all profiles are synthetically AI-generated and contain no real patient data. The only persistent structured data stored outside of chat session history is the U-ARE profile, which contains truth values indicating whether a patient meets each of the decisional capacity criteria. “Patient” demographics, medical history, current symptoms, and treatment details exist within the chat session itself and are stored as part of the chat session records within a PostgreSQL database. The conversational behavior and patient variability are powered by CAAI’s LLM Factory using Llama 3.2 90B, which uses carefully designed prompts and randomization.

For our prototype development in collaboration with IPH, we leveraged a publicly available Low Rank Adapter (LoRA) training set. In social work training scenarios, sensitive topics often arise, and traditional LLMs are frequently hard-coded to avoid engaging with risky content. Many training vignettes involved sensitive areas of discussion, such as encounters with the juvenile justice system or drug use, which created additional challenges for safety and alignment. To address this, we utilized the Abiliterated uncensored adapter to fine-tune the base model (Llama 3.0) for the needs of this application and explored reintroducing unique safeguards to ensure users could not exploit the system. Just as with the above, persona profiles and messages are stored as part of the chat session records within a PostgreSQL database.

The projects described are experimental efforts. With further development, the goal is to establish a modular system that educators can integrate into their learning platforms for a variety of disciplines, not just medical decision-making capacity assessment skill-building.

Email ai@uky.edu for more information.

This project is not yet complete. All generated data is entirely synthetic and intended for educational use only.

The CAAI project team includes Mitchell Klusty, Caroline Leach, and Cody Bumgardner.

Collaborations included advisors from relevant subject matters such as social work, distance learning, design, and medical education.

CAAI’s LLM Factory is the foundation for this effort. Costs include development time, compute resources, and GPU-powered inference on our LLM Factory server running Llama models for real-time simulation.