Prediction Models & Metrics

Panel Summary: This panel covers a wide range of topics related to prediction models and metrics within digital mental health. All panellists agreed about the importance of having humans in the loop and not having vulnerable people fall through the cracks. Some of the questions addressed in this panel include: What happens when we rely too much on a prediction model as being a true representation? Are the metrics we choose a true measure of what’s really going on? How are ground truths inferred from social media and other data? How can synthetic data potentially tackle issues of Healthcare AI including data sparsity, reproducibility, ethics, and interpretability? How does synthetic text generation enhance or harm privacy? What are the potential benefits of a prediction that is very accurate? How can automation-based feedback-informed psychotherapy enhance the chance of someone - who would have otherwise not improved – heading on a better path of recovery? How do negative versus positive predictions impact the agency of those delivering care? The panellists also discussed the importance of being open to multidisciplinary approaches (e.g., when designing feedback involved therapy tools, importance of face validity for clinicians etc.) and understanding who the decision makers are. How do we engage the patient or client who the truth is about and see how it relates to their truth? Should predictions be shared with patients or clients themselves? The panellists discuss the right for patients and clients to know what judgments or inferences are being made about them so that the model could be corrected (i.e., contestable AI). 

Live Panel Recording 5/08/21


Panellists

Mandana Ahmadi

CEO, Alena

Mandana (Alena).png

Mandana is the CEO and co-founder of Alena (previously Aya). She completed her PhD in Computational Neuroscience at UCL Gatsby, and BSc in Physics from University of Maryland, College Park. Mandana created Alena to apply her knowledge of how to build mathematical models of the human mind to help automate the diagnosis and treatment of mental health disorders, starting with social anxiety.


Lucia Chen

Stanford University, Center for Health Policy (CHP) / Center for Primary Care and Outcomes Research (PCOR)

Luica Chen.jpg

My name is Lucia (Lushi) Chen. I'm a postdoctoral research fellow at Stanford University, Center for Health Policy (CHP) / Center for Primary Care and Outcomes Research (PCOR). I obtained my Ph.D. in Informatics from the University of Edinburgh last month. My Ph.D. research concentrates on monitoring social media users' affective, cognitive, and behavioral changes, identifying the associations between digital signals and symptoms of mental disorders. I am interested in building human-centered health technologies, especially for low-income groups. I am also concerned about ethical issues and the potentially harmful effects of the technology developed in this line of work. I am passionate about identifying and developing good research practices and ethical standards.


Julia Ive

Imperial College London

Julie Ive.jpeg

Julia Ive is a Research Fellow in multimodal language processing at Imperial College London. In 2018-19, she has led the EPSRC pilot “Towards shareable data in clinical NLP: Generating synthetic electronic health records''. In 2019, she has co-organised the workshop ``Creating artificial medical records from real ones: are they safe for research?'' at King's College London on the pathway for actual release of synthetic text from the NHS. She is the author of many mono- and multimodal text generation approaches and currently working on modelling discourse structure for the detection of mental health issues in social media posts.


Jorge E Palacios

Senior Digital Health Scientist, silvercloud

Prof pic b&w.jpg

Jorge is a key driver the research strategy and agenda at SilverCloud through the design, execution, and publication of research trials, also working closely with collaborators across the world which include some of the most highly cited and regarded researchers in the field of psychology and digital mental health. He is a keynote speaker in principal academic and commercial conferences and global events on digital therapeutics. Jorge completed his medical degree in Mexico City, and thereafter won scholarships to undertake a Masters in Psychiatric Research and PhD in Psychological Medicine in London at the Institute of Psychiatry, Psychology, and Neuroscience (IoPPN), before joining SilverCloud in 2017. He has earned numerous awards such as the Young Investigator of the Year prize from Elsevier and the European Association of Psychosomatic Medicine. Jorge believes passionately in the potential of digital interventions to improve the mental health of populations at a large scale, helping more and more individuals have access to wellbeing solutions that fit their particular needs.


Mat Rawsthorne

Head of Research for HD Labs & NIHR MindTech

Mat-teach.jpg

Mat Rawsthorne is a chartered management accountant who worked in the public and private sector on business intelligence projects. His recurrent depression led him to volunteering in self-help groups, working in the third sector to gain expertise in peer support for long term conditions. He went on to train patients and professionals in evidence based therapeutic approaches. For the last five years he has been working with NIHR MindTech to facilitate coproduction in digital research and undertaking ESRC-funded doctoral training at the Institute of Mental Health. He is Head of Research for HD Labs, a start-up specialising in Conversational Health Intelligence.


Background Reading

Generation and evaluation of artificial mental health records for Natural Language Processing

Authors: Julia Ive, Natalia Viani, Joyce Kam, Lucia Yin, Somain Verma, Stephen Puntis, Rudolf N. Cardinal, Angus Roberts, Robert Stewart & Sumithra Velupillai

Published: NPJ Digital Medicine volume 3, Article number: 69 (2020)


ExTRA: Explainable Therapy-Related Annotations

Authors: Mat Rawsthorne, Tahseen Jilani, Jacob Andrews, Yunfei Long, Jeremie Clos, Samuel Malins, Daniel Hunt

Published: November 2020, 2nd Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence


Predicting outcomes and sudden gains from initial in-session interactions during remote cognitive–behavioural therapy for severe health anxiety

Authors: Sam Malins, Nima Moghaddam, Richard Morriss, Thomas Schröder, Paula Brown, Naomi Boycott

First Published: 28 December 2020


Patient activation in psychotherapy interactions: Developing and validating the consultation interactions coding scheme

Sam Malins, Nima Moghaddam, Richard Morriss, Thomas Schröder, Paula Brown, Naomi Boycott, Chris Atha

First Published: 11 Dec 2019, Journal of Clinical Psychology