Generative AI in Psychiatry
Journal Article Annotations
2024, 4th Quarter
Generative AI in Psychiatry
Annotations by Liliya Gershengoren, MD
January, 2025
- Application of AI in the creation of discharge summaries in psychiatric clinics.
PUBLICATION #1 — Generative AI in Psychiatry
Application of AI in the creation of discharge summaries in psychiatric clinics.
Bertrand Janota, Krzysztof Janota.
Abstract
Background:
The integration of artificial intelligence (AI; ChatGPT 4.0) into medical workflows presents a great potential to enhance efficiency and quality. The use of artificial intelligence in the creation of discharge summaries seems particularly interesting and valid. The course of each hospitalization is described in the discharge summary, which is given to each patient and then to his general practitioner at the end of hospital treatment. An exploratory analysis of discharge summaries in psychiatric clinics underscores that these documents must fulfill diverse and specific requirements. Nevertheless, AI-generated discharge summaries offer the opportunity to optimize information transfer and alleviate the workload on physicians.
Method:
The study evaluates the quality of discharge summaries produced by clinical staff and by an AI model (ChatGPT 4.0). The clinicians involved in writing of the discharge summaries were not informed about the study's purpose or methodology. The completed summaries were subsequently assessed by four attending physicians using predefined criteria. These physicians were also blinded to the study's objectives and were unaware of the individual authors of the summaries. The evaluation criteria included consistency, completeness, and comprehensibility. Additionally, the time required to prepare these summaries and its impact on overall quality were analyzed.
Results:
The results of the study indicate that discharge summaries generated by AI are more efficient than discharge summaries prepared by clinic staff. The AI was particularly effective in terms of coherence and information structure.
Conclusion:
Further research, training and development is needed to improve the accuracy and reliability of AI-generated discharge summaries.
Annotation
The finding:
The study found that artificial intelligence (AI)-generated discharge summaries were often superior in logic and structure, particularly for acute conditions. AI reduced preparation time from an average of 40–90 minutes for staff to 15 minutes, without compromising quality. However, AI summaries lacked the emotional nuance sometimes present in human-written documents, particularly for psychotherapy-focused cases.
Strength and weaknesses:
The strengths of the study are primarily highlighted by the AI outputs, which were highly rated for coherence, structure, and medical terminology, often scoring on par with or higher than human-written summaries. The study also utilized predefined evaluation criteria including clarity, completeness, and comprehensibility, offering an objective framework for assessing the quality of discharge summaries. However, it exhibited some weaknesses, such as a small sample size and the use of fictional cases, which limit its generalizability. Additionally, AI-generated summaries were noted for lacking empathy and nuanced insights, which are critical in psychiatry, and the minimal training provided may have affected the AI's optimal performance.
Relevance:
AI-generated discharge summaries can be a valuable tool for C-L psychiatrists, who often balance complex documentation needs with patient care. The time saved and improved structure could streamline processes in busy clinical environments. However, the lack of emotional depth in AI summaries underlines the necessity of human oversight, especially in psychiatry, where personalized and empathetic communication is key. Further training and refinement of AI tools could enhance their applicability in psychiatric documentation.
