LLM Confidence Scales: 0-20 Works Better Than 0-100
When you ask a language model how confident it is, the scale you use matters more than you think. Research shows 0-20 outperforms the standard 0-100.
👁 114 reads
When you ask a language model how confident it is in its answer, the typical approach uses a 0-100 scale. Sounds reasonable. The problem is that no one has really questioned whether this scale is actually any good. Yuyang Dai and Yuxia Wang, in a paper published in March 2026 and revised through September 2026, argue this is a serious oversight.
The Problem with Round Numbers
The authors tested six language models across three datasets and discovered something troubling: over 78% of responses clustered around just three round values. In other words, models don’t use the full available scale - instead, they gravitate toward convenient round numbers like 50, 80, or 90. This means most of the confidence gradations are effectively unused in practice.
The authors call this phenomenon “discretization” and consider it a serious problem for anyone trying to assess model uncertainty in so-called black-box settings, where you don’t have access to the model’s internal probabilities.
Three Dimensions of the Experiment
To investigate what causes this problem and whether it can be fixed, the researchers systematically manipulated confidence scales along three dimensions. The first is granularity - how detailed the scale is. The second is boundary placement - where the scale starts and ends. The third is range regularity - whether the scale is typical or irregular.
To evaluate the quality of metacognition (how well the model knows what it knows), the authors used a measure called meta-d’. The paper doesn’t explain in detail what exactly this measure is beyond its name and application.
What Works, What Doesn’t
The results are concrete. A 0-20 scale consistently improves metacognitive efficiency compared to the standard 0-100 format. This is the main practical finding of the work.
On the other hand, “boundary compression” - squeezing the scale by setting its endpoints closer to the center - worsens results. The authors also note that the preference for round numbers doesn’t disappear even when the scale is irregular. Models still seek out round values, even if the scale doesn’t technically highlight them.
Methodological Takeaway
The authors make one main recommendation: confidence scale design directly affects the quality of verbalized uncertainty and should be treated as a full experimental variable in language model evaluation. In other words - if you’re studying how well a model assesses its own confidence, you need to report which scale you used, because it matters.
The paper is 20 pages long. The authors don’t provide in the abstract information about which specific models they tested, what the datasets were, or detailed numbers about the improvement from using the 0-20 scale - that information isn’t in the supplied text.
Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →What This Means
In our view, this research has practical significance, though narrow. If you’re building a system where the model evaluates its own confidence and passes that assessment along - to a user or to another component - then the choice of scale isn’t a matter of aesthetics. The finding that over three-quarters of responses land on three numbers with a 0-100 scale suggests that this scale simply wastes potential and gives an illusion of precision that isn’t there. The recommendation to try a 0-20 scale is simple to implement and - if the study’s results hold up - worth testing. It’s a shame the abstract doesn’t provide access to details, because without knowing which models and data were tested, it’s hard to assess how broadly these findings can be generalized.
Source: arXiv cs.AI: Rescaling Confidence: What Scale Design Reveals About LLM Metacognition (document from 2026-09-12). This overview was based on the full text of the document, not reports from other media.
More on this from QUANTSLANT
benchmarks·PIDS-Bench: Why Prompt Injection Detectors Fail in Real UseA new benchmark reveals that detectors scoring 0.98 F1 still block one-third of legitimate queries from external sources - even when they look safe.