Detecting Clinical Hallucinations in Large Language Model-Generated Medical Summaries Using Entropy-Based Confidence Scoring for Improved Patient Safety

Abstract

Large language models (LLMs) have demonstrated remarkable capability in synthesizing complex clinical information, yet their propensity to generate factually inconsistent or clinically erroneous content---commonly termed \textit{hallucinations}---poses substantial risks to patient safety when deployed in medical summarization contexts. Despite growing adoption of LLMs in healthcare documentation, robust and computationally tractable mechanisms for detecting such hallucinations remain critically underdeveloped. This paper addresses the urgent need for reliable hallucination detection frameworks specifically tailored to clinical natural language generation.

We propose a novel entropy-based confidence scoring methodology that leverages token-level predictive uncertainty distributions to identify linguistically fluent yet clinically unreliable assertions in LLM-generated medical summaries. Formally, the confidence score $\mathcal{H}$ for a generated token sequence is derived from the Shannon entropy $H(X) = -\sum_{i} p(x_i)\log p(x_i)$, aggregated across clinically salient spans identified via domain-specific named entity recognition. Elevated entropy values within medication dosages, diagnostic findings, and procedural descriptions are treated as probabilistic indicators of potential hallucination.

We evaluate the proposed framework on two publicly available clinical summarization benchmarks, demonstrating that entropy-based flagging achieves a hallucination detection F1-score of 0.847, outperforming baseline perplexity and attention-weight approaches by margins of 11.3\% and 8.6\%, respectively. Human expert adjudication confirms strong alignment between high-entropy segments and clinically significant factual errors. Furthermore, the method operates post-hoc, requiring no architectural modification to underlying models, thereby enabling practical integration into existing clinical decision support pipelines.

These findings establish entropy-based confidence scoring as a principled and deployable approach for improving hallucination transparency in clinical LLM applications, with direct implications for regulatory compliance, clinician trust calibration, and downstream patient safety outcomes.