Researchers at the University of Pennsylvania, New York University, and the University of Pennsylvania’s Linguistic Data Consortium have received a two-year, $4 million grant from the Wellcome Trust to develop a scalable, AI-powered platform to train mental health clinicians.
STELLAR (Steering-Vector Enhanced LLM Agents for Realistic Digital Twins in Mental Health) creates virtual patients, known as “digital twins,” in the form of AI-driven simulations that allow trainees to practice clinical interviews with patient profiles that can precisely adjust for psychiatric symptoms.
One of the challenges in educating mental health clinicians is preparing trainees for the complexities of real-world clinical conversations, where symptoms can overlap, change over time, and are expressed differently by different people. STELLAR’s goal is to provide an ethical and reproducible method for trainees to simulate patient interviews across a variety of conditions, backgrounds, and clinical scenarios.
“STELLAR combines behavioral data, clinical expertise, and AI to ask very practical questions,” said Sharath Chandra Gunthuk, research associate professor of computer and information science (CIS) at Penn Engineering and one of the project leaders. “Can we build training tools that help clinicians better respond to patient diversity and complexity?”
“The promise of this approach is that we can move beyond stylized and potentially biased simulations,” adds Joanne Sedok, assistant professor of technology, operations, and statistics at New York University’s Stern School of Management and another project leader. “If we can create digital patients that can controllably and plausibly simulate symptom manifestations and responsibly assess them, we can enhance current clinician training practices with the kinds of conversations that are essential for better mental health care.”
The power of patient simulation
STELLAR’s simulations are not copies of individual patients, but rather composites based on real-world data that help clinicians practice realistic conversations in a controlled environment.
In mental health training, it is especially important to be able to precisely control the symptoms that students encounter. Trainees may need to practice interviewing patients with mild anxiety and then patients whose anxiety overlaps with depression or psychosis.
STELLAR is designed to allow for these variations, allowing researchers and trainers to adjust symptom presentation, symptom intensity, and symptom interaction.
In psychiatry, the details of symptom experience are important. That is, how someone describes their distress, how symptoms overlap, how severity changes over time, and how context shapes clinical interactions. ”
Raquel Garr, Karl and Linda Rickels Professor of Psychiatry, Associate Chair of Neurology and Radiology, Perelman School of Medicine, Pennsylvania
Turn data into patient simulation
To build these simulations, researchers will utilize clinical data from the Philadelphia Neurodevelopmental Cohort, a research resource established by Penn Medicine and Children’s Hospital of Philadelphia (CHOP). This data includes psychiatric assessments and clinical interviews from thousands of young people.
The project will also leverage data from social media platforms where mental health symptoms can manifest in everyday language. “Many mental health symptoms don’t just manifest in formal clinical settings; they also manifest in people’s everyday conversations, including online,” Gangtuk points out.
Patient simulations are useful for clinician training only if they are based on real clinical speech and evaluated as clinical interactions, not just plausible AI interactions. ”
Neville Ryant, University of Pennsylvania Language Data Consortium (LDC) researcher and STELLAR project leader
“LDC’s role is to embed speech and language science at the core of the project, adapting speech recognition tools to clinical interviews, producing high-quality transcripts and annotations, and assisting in the assessment of both what is said and how it is said in simulations,” Ryant added.
“This includes evaluating the language produced by the model, the naturalness of the synthesized voices, and the extent to which those voices reflect the target speech patterns and the avatar’s behavior during real trainee interactions.”
Focused on human patients
STELLAR uses AI to create digital patients, but the project’s success goes beyond technical performance alone. Simulations also need to be evaluated by those who best understand what realistic, respectful, and helpful interactions look like.
To this end, the team includes people with lived experience with mental health conditions throughout the project, as well as family members and carers who bring valuable perspectives to the assessment process.
Their feedback will help researchers assess whether simulations reflect patterns of real-world symptom experience, avoid flattening or immobilizing patients, and prepare trainees for the complexity, diversity, and nuance of real-world clinical conversations.
“By involving individuals with lived experience throughout the project, STELLAR helps us ask not only whether digital patient interactions are clinically accurate, but also whether their interactions feel respectful, realistic, and attentive to experiences that are often overlooked,” says Garr.
Enhancing training for designated mental health physicians
Ultimately, STELLAR could provide training programs with a scalable way to prepare clinicians for more diverse and complex patient encounters, including symptom combinations that are difficult to consistently encounter in traditional training.
“By combining real-world symptom representations with controllable digital simulations, we hope to make clinician training more scalable, rigorous, and representative,” Guntuku said.
sauce:
University of Pennsylvania School of Engineering and Applied Science
