Uncertainty Is Not a Bug

Uncertainty is not a defect in a model. Uncertainty is the honest part of prediction.

A model that reports no uncertainty claims knowledge it does not possess. A model that reports calibrated uncertainty tells the next decision-maker how far to trust it.

The simplest version#

A classifier outputs a distribution over classes. The distribution encodes uncertainty directly. The most probable class is the prediction. The shape of the distribution is the uncertainty report.

import numpy as np
 
def predict_with_confidence(logits):
    # Softmax converts logits to a probability distribution
    probs = np.exp(logits - np.max(logits)) / np.sum(np.exp(logits - np.max(logits)))
    predicted_class = int(np.argmax(probs))
    confidence = float(probs[predicted_class])
    return predicted_class, confidence, probs
 
def confidence_report(logits_batch, labels):
    reports = [predict_with_confidence(l) for l in logits_batch]
    correct = np.array([r[0] == y for r, y in zip(reports, labels)])
    confs = np.array([r[1] for r in reports])
    return {
        "accuracy": np.mean(correct),
        "mean_confidence": np.mean(confs),
        "gap": np.mean(confs) - np.mean(correct),
    }

The gap between mean confidence and accuracy is the first uncertainty signal. A gap of zero suggests calibration. A positive gap suggests overconfidence. A negative gap suggests underconfidence.

The code also reveals the naive failure. Softmax confidence is a function of the logits. Logits are trained for classification accuracy, not for honest probabilities. The distribution's sharpness reflects the training loss more than the true ambiguity.

Deriving the two kinds of uncertainty#

Aleatoric uncertainty comes from the data itself. Sensor noise, label ambiguity, and inherent randomness remain even with unlimited training data. Epistemic uncertainty comes from the model. Finite data, unseen cases, and missing knowledge create uncertainty that more data can reduce.

# Aleatoric: irreducible given the data-generating process
#   y = f(x) + noise,  noise ~ N(0, sigma_noise^2)
#   Even the Bayes-optimal model cannot remove sigma_noise.
 
# Epistemic: reducible with more data or capacity
#   Model variance across data resamples estimates epistemic uncertainty
#   Var(model) = mean(w^2) - mean(w)^2  across an ensemble
 
def ensemble_uncertainty(model_predictions):
    # models x samples: predictions from an ensemble
    mean_pred = np.mean(model_predictions, axis=0)
    epistemic_var = np.var(model_predictions, axis=0)
    return mean_pred, epistemic_var
 
def total_uncertainty(aleatoric_var, epistemic_var):
    # Independent sources add in variance
    return aleatoric_var + epistemic_var

The distinction drives opposite responses. Aleatoric uncertainty calls for robustness. The model must perform well despite noise that cannot be eliminated. Epistemic uncertainty calls for caution, abstention, or more evidence. The model must know where knowledge ends.

An ensemble separates the two components. Disagreement between ensemble members estimates epistemic uncertainty. Average within-member variance estimates aleatoric uncertainty. A single model cannot perform the decomposition.

Naive calibration: read the softmax literally#

The naive approach treats softmax output as a true probability. Training with cross-entropy encourages the correct class to receive high probability, but nothing in the objective controls the mapping between confidence and empirical accuracy on held-out data.

def reliability_diagram(confidences, correct, n_bins=10):
    # Bins predictions by confidence, compares to empirical accuracy
    bins = np.linspace(0, 1, n_bins + 1)
    diagram = []
    for lo, hi in zip(bins[:-1], bins[1:]):
        in_bin = (confidences >= lo) & (confidences < hi)
        if in_bin.sum() == 0:
            continue
        diagram.append({
            "mean_confidence": float(np.mean(confidences[in_bin])),
            "accuracy": float(np.mean(correct[in_bin])),
            "count": int(in_bin.sum()),
        })
    return diagram
 
def expected_calibration_error(confidences, correct, n_bins=10):
    diagram = reliability_diagram(confidences, correct, n_bins)
    total = len(confidences)
    ece = 0.0
    for b in diagram:
        ece += (b["count"] / total) * abs(b["mean_confidence"] - b["accuracy"])
    return ece

Perfect calibration follows the diagonal of the diagram. Systematic deviation shows overconfidence or underconfidence. Expected calibration error compresses the diagram into a single number by averaging the weighted gap across bins.

The matrix-style diagram already exposes the flaw. Modern networks trained with cross-entropy are systematically overconfident on held-out data. Training accuracy approaches 100 percent while validation accuracy lags. The softmax sharpness follows training fit, not generalization.

Fix one: temperature scaling#

Temperature scaling calibrates by dividing logits by a single learned scalar before softmax. The scalar is fit on a held-out validation set. Training weights never change. Calibration becomes a post-hoc correction.

def temperature_scale(logits, temperature):
    scaled = logits / temperature
    probs = np.exp(scaled - np.max(scaled, axis=1, keepdims=True))
    return probs / probs.sum(axis=1, keepdims=True)
 
def fit_temperature(logits_val, labels_val, grid=None):
    # Search T to minimize validation negative log-likelihood
    if grid is None:
        grid = np.linspace(0.1, 5.0, 50)
    best_t, best_nll = 1.0, np.inf
    for t in grid:
        probs = temperature_scale(logits_val, t)
        nll = -np.mean(np.log(probs[np.arange(len(labels_val)), labels_val] + 1e-12))
        if nll < best_nll:
            best_nll, best_t = nll, t
    return best_t

The fix succeeds because calibration and classification have different optima. Shifting sharpness with one parameter preserves argmax predictions while moving confidence toward empirical accuracy. Validation negative log-likelihood selects the temperature that the data supports.

The bottleneck is locality. A single temperature corrects average overconfidence. Overconfidence that varies by class, by region of input space, or by input difficulty survives the correction. Local calibration requires local information.

Fix two: measure with proper scoring rules#

Calibration errors describe gaps. Proper scoring rules reward honest probabilities directly. The Brier score measures mean squared error between predicted probabilities and one-hot outcomes. Log loss measures negative log-likelihood. Both are minimized only by true probabilities.

def brier_score(probs, labels):
    n, k = probs.shape
    one_hot = np.zeros_like(probs)
    one_hot[np.arange(n), labels] = 1.0
    return np.mean(np.sum((probs - one_hot) ** 2, axis=1))
 
def log_loss(probs, labels):
    n = probs.shape[0]
    return -np.mean(np.log(probs[np.arange(n), labels] + 1e-12))
 
def decompose_brier(probs, labels, n_bins=10):
    # Brier = reliability + resolution + uncertainty
    # Lower is better; the decomposition shows which term dominates
    confs = np.max(probs, axis=1)
    correct = np.argmax(probs, axis=1) == labels
    return {
        "brier": brier_score(probs, labels),
        "log_loss": log_loss(probs, labels),
        "ece": expected_calibration_error(confs, correct, n_bins),
    }

A metric that punishes confident errors removes the incentive to report extreme probabilities on uncertain cases. An overconfident wrong prediction incurs a large Brier and log loss penalty. A hedged correct prediction incurs a modest penalty. Honest uncertainty becomes the optimal strategy.

Binned metrics need care. Bin count, bin edges, and sample size change the value, and small samples make the estimate noisy. Proper scores avoid binning but conflate discrimination with calibration. A model that ranks correctly but calibrates poorly and a model that calibrates well but ranks poorly can produce similar scores.

Application one: medicine#

A medical prediction without calibrated confidence cannot guide care. A diagnosis at 90 percent certainty and a diagnosis at 55 percent certainty demand different actions.

def decision_with_confidence(probs, threshold_low=0.3, threshold_high=0.8):
    # Three-way decision: rule out, uncertain, rule in
    # The thresholds only make sense with calibrated probabilities
    decisions = []
    for p in probs:
        if p < threshold_low:
            decisions.append("rule_out")
        elif p > threshold_high:
            decisions.append("rule_in")
        else:
            decisions.append("order_more_tests")
    return decisions

Calibration lets clinicians set thresholds, allocate tests, and decide when a model output enters the workflow. Miscalibration inverts the logic: confident mistakes trigger wrong treatments, and hedged correct predictions get ignored. The stakes make the argument concrete. Calibration is not a nicety on a leaderboard; calibration is the interface between the model and the patient.

Application two: reinforcement learning#

An RL agent explores under uncertainty. Calibrated uncertainty tells the agent where exploration pays and where the current policy is trustworthy.

def exploration_bonus(epistemic_var, scale=1.0):
    # High epistemic uncertainty -> explore; low -> exploit
    # Calibrated variance turns the bonus into a decision signal
    return scale * np.sqrt(epistemic_var)
 
def risk_sensitive_value(mean_value, var_value, risk_aversion):
    # Planning under uncertainty uses the same numbers
    return mean_value - risk_aversion * np.sqrt(var_value)
 
def abstain_or_act(epistemic_var, threshold):
    # Abstention is the correct action under high epistemic uncertainty
    return ["act" if v < threshold else "abstain" for v in epistemic_var]

Overconfidence collapses exploration early and locks in a poor policy. Underconfidence wastes samples on regions already understood. Planning under uncertainty uses the same numbers. Value estimates, imagined rollouts, and risk-sensitive objectives all need a realistic gauge of model error.

What to measure#

Reliability diagrams show where calibration fails. Expected calibration error summarizes the gap. Proper scores punish dishonest probabilities. Each measurement answers a different question.

def full_calibration_report(probs, labels):
    confs = np.max(probs, axis=1)
    correct = np.argmax(probs, axis=1) == labels
    return {
        "accuracy": float(np.mean(correct)),
        "mean_confidence": float(np.mean(confs)),
        "ece": float(expected_calibration_error(confs, correct)),
        "brier": float(brier_score(probs, labels)),
        "log_loss": float(log_loss(probs, labels)),
        "temperature": None,  # Fill after fitting on validation data
    }

Checklist#

• Separate aleatoric from epistemic uncertainty. Aleatoric calls for robustness. Epistemic calls for caution or data.
• Never read softmax output as a calibrated probability without validation. Training sharpness is not calibration.
• Fit a temperature on held-out data before deployment. One scalar corrects average overconfidence.
• Report reliability diagrams alongside ECE. Scalar summaries hide local miscalibration.
• Use proper scoring rules for model selection. Only proper scores reward honest probabilities.
• Connect confidence to downstream decisions. Thresholds, exploration bonuses, and abstention rules require calibration to function.