You can't fix what you don't measure: scoring hallucination rate

Confidence in tone and confidence in fact are two different things, and language models are very good at producing the first without the second.

Semantic entropy analysis works by sampling a model's response to the same question multiple times and measuring how much the meaning varies. A model that's actually sure of an answer tends to say the same thing in different words. A model that's guessing tends to drift.

It's not a perfect signal on its own, which is why we combine it with linguistic classification and graph-based reasoning over the response — but it's one of the few signals that scales to catching hallucination before a user ever sees it.