Short answer
A confidence score is a number, usually from 0 to 1, showing how likely an AI model thinks its own output is correct. Above a set confidence threshold the system acts; below it, the request goes to a person or gets a safer answer.- Range
- 0 to 1, or 0% to 100%
- Used for
- Deciding when AI acts and when a person takes over
- Set with
- A confidence threshold
- Watch for
- Models that are confidently wrong
A score of 0.92 on a label means the model rates that label as the right one with 92% confidence.
The word that matters is "thinks". A confidence score is the model's own estimate, not a check against the truth. A model can be confidently wrong, and a low score is not proof of a wrong answer. The score is useful when it has been compared with real outcomes and used to make a decision, not read as a promise.
How a confidence score is calculated
The method depends on the kind of model.
- Classification models. The model gives a probability to every possible label, and those probabilities add up to 1. The confidence score is the probability of the label it picked. If "order status" gets 0.88 and "returns" gets 0.09, the confidence is 0.88.
- Language models. A large language model assigns a probability to each word it writes. Those can be combined into a score for the whole answer, though the result is often less reliable than a classifier's.
- Self-rating. The model is asked to rate how sure it is. This is easy to set up and easy to fool, because the model can sound certain about something false.
- Agreement. The same question is run several times, or through several models, and the score reflects how often the answers match.
In Aide, each customer message is scored against every one of the team's topics from 0 to 1, using the topic's name, description and saved examples, and Aide tags up to three, best first, or none rather than forcing a weak match.
What a confidence threshold does
A confidence threshold is the cut-off where a system changes what it does. In customer service it usually decides between three paths.
| Confidence | What the system does |
|---|---|
| Above the threshold | Acts or answers on its own |
| Just below | Writes a draft for a person to review |
| Well below | Hands the conversation to a person with a summary |
A higher threshold means fewer mistakes and more work for the team. A lower one means more automation and more risk. The right threshold differs by request: a tracking lookup can run on a lower one than a refund.
Near misses: results just under the threshold
The results worth reading are the ones that fell just short. A threshold silently drops them, so nobody learns whether the model was nearly right or the label needs better examples.
In Aide, a customer message whose best topic falls just under that topic's threshold (80% by default) shows a near miss on the message, such as "Sales 45% · under 80%", with Aide's one-line reason. One click confirms it ("It's Sales"), which adds the message as a training example, or rejects it ("Not Sales"). Each topic lists its near misses from the last 30 days, and Aide learns only from confirmed examples, so a rejection never teaches the opposite.
Calibration: when 90% means 90%
A model is calibrated when its scores match reality: of all the answers it rated 90% confident, about 90% were right. Many models are overconfident, rating answers 95% sure that are right far less often. Calibration is checked by comparing scores with outcomes on real past conversations, then adjusting the threshold until the error rate is one the team accepts.
The practice this page rejects is treating a high score as permission. A confidence threshold decides when a system may try. It does not replace testing the answer, checking it against the business's rules, or recording what happened.
With Aide, a team sees how a procedure's trigger behaves on real traffic before trusting it: Check trigger runs it on recent conversations, lists the matches and the near misses with a reason for each, and lets the team mark "Use for this" or "Don't use for this" to add examples.
Other meanings of confidence score
The phrase is also used outside AI. In banking and lending it can describe how certain a credit or fraud model is about a decision, and in sales tools an account confidence score rates how likely a deal is to close. The idea is the same in each case: a model's estimate of how likely its call is right.
Where Aide fits
Aide, the AI platform for customer service, gives every topic it detects on a message a confidence score. A procedure can require a minimum topic confidence, anywhere from 80% to 97.5%, before it runs, so a borderline match falls back to the usual answer instead of a scripted action. Confidence is one gate among several: every reply is also checked before it is sent, and one that promises something the topic's AI level or the store's policies don't back is rewritten once or handed to the team.