Calibrating Confidence Thresholds for PII Redaction

Within the broader Confidence Scoring & Thresholds for PII Detection specification — itself part of the PII Extraction & Redaction Pipelines architecture — this page answers one narrow, high-stakes question: given a detector that already emits a calibrated score per candidate span, where do you set the operating threshold, and how do you defend that number? The engineers who hit this are privacy and ML platform teams running a Data Subject Request (DSR) pipeline where the two failure modes are not symmetric. A false negative — a missed PII entity that survives into an exported access package or a supposedly-erased record — is a personal-data breach reportable under GDPR Art. 33 and a violation of the confidentiality duty of GDPR Art. 5(1)(f). A false positive — over-redaction — is a real cost too: it destroys data the subject was entitled to receive under GDPR Art. 15 or corrupts a legitimate business record, but it does not leak anything. That asymmetry is the whole game. Choosing a threshold is therefore a constrained optimization: hold recall above a floor that bounds the miss rate, and only then maximize precision to limit over-redaction. The scoring stage assumes scores are already calibrated; this page is about turning those calibrated scores into a single, auditable cut point, the same discipline applied to model outputs in Evaluating NER Precision and Recall for PII.

The method is a threshold sweep over a labeled evaluation set: at every candidate cut point measure precision and recall, discard every point below the recall floor, and among the survivors keep the one with the highest precision.

Prerequisites #

The code targets Python 3.11+ (for tuple[...] / list[...] generics without from __future__ and the zoneinfo timezone stamp on the policy artifact). Install:

• pydantic (v2) — strict validation of the threshold policy artifact. The examples use v2 idioms (model_config = ConfigDict(...), field_validator, model_validate); v1 patterns (class Config, @validator, .dict()) will not work.
• numpy (optional) — vectorizes the sweep when the evaluation set runs to tens of thousands of spans; the loops below are written in plain Python so they stay readable.

Two upstream assumptions must already hold. First, scores are calibrated: a score of 0.83 means roughly an 83% posterior probability that the span is PII, per the Platt/isotonic step in the parent Confidence Scoring & Thresholds stage. Calibrating and thresholding are separate jobs — threshold-picking on uncalibrated scores gives a number that silently changes meaning across model versions. Second, you have a labeled held-out evaluation set: candidate spans each carrying the detector’s score and a gold boolean of whether the span truly is PII. That gold label is what makes precision and recall measurable rather than assumed.

Step-by-Step Implementation #

Step 1 — Assemble scored predictions with gold labels #

Every calibration run starts from an immutable snapshot of scored predictions paired with ground truth. Keep the evaluation set versioned and disjoint from anything used to tune the detector, so the threshold cannot be overfit to data the model has already seen.

from dataclasses import dataclass


@dataclass(frozen=True)
class ScoredSpan:
    """One detector candidate on the labeled evaluation set."""
    entity_type: str
    score: float          # calibrated probability in [0.0, 1.0]
    is_pii: bool          # gold label: True if the span truly is PII


def load_eval_set() -> list[ScoredSpan]:
    """Illustrative fixture; in production this is a versioned, held-out set."""
    return [
        ScoredSpan("us_ssn", 0.99, True),
        ScoredSpan("email", 0.94, True),
        ScoredSpan("person", 0.88, True),
        ScoredSpan("person", 0.71, True),
        ScoredSpan("person", 0.66, False),   # common surname / noun collision
        ScoredSpan("person", 0.52, False),
        ScoredSpan("us_ssn", 0.61, False),   # internal record ID lookalike
    ]

The gold labels are the only reason precision and recall can be computed at all; without them a threshold is just a guess dressed up as a number.

Step 2 — Sweep thresholds and compute precision/recall at each #

For a threshold t, a span is predicted PII when score >= t. That yields the confusion counts — true positives, false positives, false negatives — from which precision and recall follow. Sweeping every distinct score as a candidate cut point (plus the extremes) traces the full precision-recall curve exactly, with no arbitrary grid.

from dataclasses import dataclass


@dataclass(frozen=True)
class ThresholdPoint:
    threshold: float
    precision: float
    recall: float
    true_pos: int
    false_pos: int
    false_neg: int


def evaluate_at(threshold: float, spans: list[ScoredSpan]) -> ThresholdPoint:
    """Confusion counts and precision/recall for one cut point."""
    tp = sum(1 for s in spans if s.score >= threshold and s.is_pii)
    fp = sum(1 for s in spans if s.score >= threshold and not s.is_pii)
    fn = sum(1 for s in spans if s.score < threshold and s.is_pii)
    precision = tp / (tp + fp) if (tp + fp) else 1.0
    recall = tp / (tp + fn) if (tp + fn) else 1.0
    return ThresholdPoint(threshold, precision, recall, tp, fp, fn)


def sweep(spans: list[ScoredSpan]) -> list[ThresholdPoint]:
    """Trace the precision-recall curve over every distinct score."""
    candidates = sorted({s.score for s in spans} | {0.0, 1.0})
    return [evaluate_at(t, spans) for t in candidates]

Precision falls back to 1.0 when nothing is predicted positive (a vacuously precise but useless point) and recall falls back to 1.0 when there are no positives to catch; both edge conventions must be documented in the threshold register because they shape the curve’s endpoints.

Step 3 — Pick the threshold that meets a recall floor #

This is where the compliance asymmetry becomes code. The recall floor is a hard constraint derived from the tolerable miss rate: a missed PII entity is a breach, so the maximum acceptable false-negative rate is a policy decision, not a model output. Among every threshold that satisfies the floor, select the one with the highest precision — equivalently the highest threshold — to over-redact as little as possible.

def select_threshold(
    points: list[ThresholdPoint],
    recall_floor: float = 0.98,
    min_support: int = 200,
) -> ThresholdPoint:
    """Highest-precision cut point whose recall clears the floor.

    Raises if no threshold can satisfy the floor, so the pipeline
    fails closed to review rather than shipping an unsafe cut point.
    """
    total_pos = points[0].true_pos + points[0].false_neg
    if total_pos < min_support:
        raise ValueError(
            f"Only {total_pos} positive examples; too few to fix a threshold."
        )
    feasible = [p for p in points if p.recall >= recall_floor]
    if not feasible:
        raise ValueError(
            f"No threshold reaches recall {recall_floor}; detector recall is capped."
        )
    # Among feasible points, maximize precision; tie-break to the higher threshold.
    return max(feasible, key=lambda p: (p.precision, p.threshold))

An alternative to a fixed floor is to minimize an expected-cost objective that prices the asymmetry directly — cost = c_fn * false_neg + c_fp * false_pos with c_fn far larger than c_fp. That is useful when the floor is hard to fix a priori, but the recall-floor form is easier to defend to a regulator because the constraint reads directly as “our miss rate stays below X.”

def select_by_cost(points: list[ThresholdPoint], c_fn: float = 50.0,
                   c_fp: float = 1.0) -> ThresholdPoint:
    """Threshold minimizing asymmetric expected cost (miss >> over-redaction)."""
    return min(points, key=lambda p: c_fn * p.false_neg + c_fp * p.false_pos)

Step 4 — Freeze the decision as a Pydantic v2 ThresholdPolicy #

The chosen number is a compliance artifact, so it is validated, versioned, and immutable. The policy binds the threshold to the recall floor it satisfied, the measured precision/recall, the evaluation-set and calibration versions that produced it, and a UTC timestamp — everything an examiner needs to replay the decision. A field_validator re-asserts the invariant at construction time: if the measured recall does not clear the floor, the policy refuses to instantiate, so an unsafe cut point can never be serialized into production config.

from datetime import datetime, timezone
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator


class ThresholdPolicy(BaseModel):
    model_config = ConfigDict(extra="forbid", frozen=True)

    entity_scope: str                       # e.g. "global" or a specific entity_type
    threshold: float = Field(ge=0.0, le=1.0)
    recall_floor: float = Field(ge=0.0, le=1.0)
    measured_recall: float = Field(ge=0.0, le=1.0)
    measured_precision: float = Field(ge=0.0, le=1.0)
    eval_set_version: str
    calibration_version: str
    decided_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))

    @field_validator("decided_at")
    @classmethod
    def must_be_aware(cls, v: datetime) -> datetime:
        if v.tzinfo is None:
            raise ValueError("decided_at must be timezone-aware UTC")
        return v

    @model_validator(mode="after")
    def recall_meets_floor(self) -> "ThresholdPolicy":
        if self.measured_recall < self.recall_floor:
            raise ValueError(
                f"measured_recall {self.measured_recall} below floor {self.recall_floor}"
            )
        return self


def build_policy(spans: list[ScoredSpan], recall_floor: float,
                 eval_set_version: str, calibration_version: str) -> ThresholdPolicy:
    point = select_threshold(sweep(spans), recall_floor)
    return ThresholdPolicy(
        entity_scope="global",
        threshold=point.threshold,
        recall_floor=recall_floor,
        measured_recall=point.recall,
        measured_precision=point.precision,
        eval_set_version=eval_set_version,
        calibration_version=calibration_version,
    )

Because the model is frozen=True and extra="forbid", a promoted policy cannot be mutated in flight and schema drift becomes a load-time error — the accountability posture GDPR Art. 5(2) expects of a control you must be able to demonstrate.

Configuration Reference #

Verification #

Confirm the two invariants that matter: the selected threshold actually clears the floor on held-out data, and a policy that fails to clear the floor cannot be constructed.

import pytest


def test_selected_threshold_meets_floor():
    spans = load_eval_set()
    point = select_threshold(sweep(spans), recall_floor=0.75, min_support=1)
    assert point.recall >= 0.75
    # No feasible point may have higher precision at equal-or-better recall.
    better = [p for p in sweep(spans)
              if p.recall >= 0.75 and p.precision > point.precision]
    assert not better


def test_policy_rejects_sub_floor_recall():
    with pytest.raises(ValueError):
        ThresholdPolicy(
            entity_scope="global", threshold=0.9, recall_floor=0.98,
            measured_recall=0.90, measured_precision=0.99,
            eval_set_version="v1", calibration_version="platt-v3",
        )


def test_infeasible_floor_fails_closed():
    spans = load_eval_set()
    with pytest.raises(ValueError):
        select_threshold(sweep(spans), recall_floor=0.999, min_support=1)

Expected behaviour: the first test finds the highest-precision point at or above 0.75 recall; the second raises because measured_recall (0.90) is below the 0.98 floor; the third raises because no cut point on the fixture can reach 0.999 recall, so the pipeline escalates to review rather than shipping an unsafe threshold. Every run should also emit the selected ThresholdPolicy as JSON via policy.model_dump(mode="json") into the audit store, so the cut point is reproducible from its inputs.

Troubleshooting #

Recall decays in production while the offline number looks fine (drift). Root cause: the live data distribution has moved away from the frozen evaluation set — new document formats, a new locale, a new identifier grammar. Fix: monitor realized recall on a rolling labeled sample and re-sweep when it approaches the floor; treat a drifting borderline fraction as a leading indicator and recalibrate before the floor is breached, never after.

Precision and recall swing wildly between runs (class imbalance). Root cause: PII is rare, so a handful of positives dominates the estimate and a single mislabeled span moves the curve. Fix: enforce min_support, report a confidence interval on recall (e.g. Wilson) rather than a point estimate, and stratify the evaluation set by entity_type so a common entity cannot mask a starved rare one.

The threshold behaves differently after a model update (calibration mismatch). Root cause: the threshold was chosen against one calibration and the new model version shifted the score distribution, so 0.83 no longer means an 83% posterior. Fix: bind every ThresholdPolicy to its calibration_version, refuse to load a threshold whose calibration version does not match the live detector, and re-run the sweep as part of promoting the new version — the calibration step is upstream and non-negotiable per the parent Confidence Scoring & Thresholds stage.

The chosen threshold looks great offline but generalizes badly (overfit / leakage). Root cause: the evaluation set overlaps the detector’s training data, so measured precision/recall are optimistic. Fix: keep the labeled evaluation set strictly held out and version it, and hold at least one jurisdiction’s fixtures out of tuning entirely so the threshold cannot be silently overfit to a single regime.

No threshold can satisfy the floor at all. Root cause: the detector’s maximum achievable recall is below the floor — a model-quality ceiling, not a threshold problem. Fix: fail closed (as select_threshold does), route everything to human review, and treat it as a detector-improvement task; a threshold cannot manufacture recall the model does not have.

Frequently Asked Questions #

Related #

• Confidence Scoring & Thresholds for PII Detection — the parent stage that normalizes and calibrates the scores this page turns into an operating point.
• Evaluating NER Precision and Recall for PII — how the labeled evaluation set and its precision/recall measurements are built.
• NLP-Based Entity Recognition — the probabilistic detector whose calibrated scores feed the threshold sweep.
• Regex Pattern Libraries for PII Detection — deterministic, checksum-anchored matches that enter scoring as near-binary high-confidence signals.
• PII Extraction & Redaction Pipelines — the parent architecture the calibrated threshold governs before any masking runs.