AI Lab · by Avery Johnson · 2026-10-21
Using an LLM to score its own confidence is a bad idea
Post argues verbalized confidence from LLMs is poorly calibrated and you should use logprobs, ensembles or external checks instead. Matches my experience. What do you use in production to decide when to trust model output?
20 sparks · 2 comments