Introduction
Decision Curve Analysis (DCA) is a powerful tool for evaluating the clinical utility of predictive models and diagnostic tests. Unlike traditional metrics like AUC or calibration, DCA focuses on what truly matters in practice: whether using a model leads to better decisions and outcomes.
DCA was introduced by Vickers and Elkin in 2006 to address the limitations of conventional performance metrics. It calculates the net benefit of using a model across a range of threshold probabilities—helping clinicians decide whether a model improves decision-making compared to default strategies like treating all or no patients.
In this guide, we’ll walk through the core concepts of DCA, how to interpret decision curves, calculate net benefit, and apply these insights to real-world clinical scenarios.
Case Example: Prostate Cancer Biopsy
Imagine a patient who presents with elevated PSA (prostate-specific antigen) levels during routine screening. A predictive model is then applied to estimate the patient’s individualized risk of hosting high-grade prostate cancer. Rather than proceeding with a biopsy for every single patient who has elevated PSA—which would result in many unnecessary and invasive procedures—Decision Curve Analysis helps clinicians determine whether incorporating the predictive model into their decision-making process actually leads to better clinical outcomes. Specifically, it evaluates whether the model results in fewer unnecessary biopsies being performed on patients who do not have aggressive disease, while simultaneously achieving more accurate and timely identification of those patients who do have aggressive cancers that require intervention.
Key concepts
- Net benefit: balances true positives and false positives using the threshold probability.
- Threshold probability (pt): the minimum predicted risk at which intervention is justified.
- Exchange rate: pt/(1−pt), the implied trade-off between one false negative and false positives.
- Strategies: Treat all, Treat none, and Model-based.
Interpreting a decision curve
Net benefit across clinically relevant thresholds. Compare the model to Treat all and Treat none.
Step-by-Step Interpretation:
- Higher Curve = Greater Benefit: The model with the highest curve offers the most clinical value.
- Preferences Matter: Some patients prioritize avoiding disease; others fear unnecessary procedures.
- Threshold Probability Is the Decision Point: It defines when intervention becomes justified.
- Net Benefit Is Like Net Profit: It’s the clinical “gain” after accounting for harms.
- Can Be Expressed as Interventions Avoided: Useful for communicating impact in practical terms.
Decision curves can help to compare models:
- Higher curve means greater clinical utility at that threshold.
- Focus on the clinically plausible threshold range for the condition and intervention.
- Compare your model against Treat all and Treat none to contextualize gains.
- Translate net benefit into people terms when communicating with clinicians.
Net benefit formula
Where TP and FP are counts on a sample of size n, and ptpt is the threshold probability. The factor pt1−pt1−ptpt is the exchange rate between harms of false positives and benefits of true positives.
Equivalent impact metrics per 100 patients:
True-positive equivalents per 100 = 100×NB100×NB
Interventions avoided per 100 = 
Clinical Impact Plot
This graph shows two lines, both answering a practical question:
“If I use this prediction model on 100 patients, what actually happens?”
- Green line (True Positives): The number of sick patients who are correctly identified and would receive treatment. → These are the people who benefit from using the model.
- Orange line (False Positives): The number of healthy patients who are mistakenly flagged as high-risk and would receive unnecessary treatment. → These are the people who might be harmed (side effects, anxiety, cost) due to overuse.
Both numbers change as you adjust the threshold (the risk level at which you decide to treat). For example:
- At a low threshold (e.g., 5%), you treat almost everyone → you catch more sick people (high green), but also treat many healthy ones (high orange).
- At a high threshold (e.g., 40%), you treat only the highest-risk patients → fewer healthy people are treated (low orange), but you also miss some sick patients (low green).
Why this matters:
Doctors don’t think in “net benefit” or “AUC”—they think in people.
This plot translates abstract model performance into real human outcomes:
“At a 15% risk threshold, I’ll help about 22 patients—but 18 others will get treatment they don’t need.”
This helps clinicians choose a threshold that feels right for their patients, their setting, and the seriousness of the treatment.
Calibration Plot
This graph checks whether the model’s predicted risks match reality.
X-axis: What the model says the risk is (e.g., “20% chance of disease”).
Y-axis: What actually happened in patients with that predicted risk (e.g., did 20% of them really get sick?).
The dotted diagonal line represents perfect trust: if the model says 30%, then 30% of those patients get sick.
The blue markers show how your model actually performed.
You’ll also see vertical dashed lines at common decision points (10%, 20%, 30%)—these are the thresholds doctors might use to decide who to treat.
Why this matters:
Decision Curve Analysis assumes your model’s probabilities are honest.
But if your model is overconfident (e.g., says “10% risk” but 25% actually get sick), then:
Using a 10% treatment threshold would mean treating far too few people.
Your DCA results could look good—but in reality, you’re missing many at-risk patients.
Similarly, if the model is underconfident (says “40%” but only 15% get sick), you’d overtreat healthy people.
Recommended analysis workflow
Split or use external validation. Freeze indices for reproducibility.
Fit model and obtain predicted probabilities on the test set.
Calibrate if needed (Platt scaling or isotonic regression with proper cross-validation).
Compute net benefit across a clinically justified threshold range.
Quantify uncertainty with bootstrap confidence intervals.
Report clinical impact and, for model comparisons, differences in net benefit at pre-specified thresholds with CIs.
Python implementation
The code below generates four figures and adds bootstrap CIs for net benefit at selected thresholds. Save the figures and embed them here.
Step 1 — Data setup
Use a small, self‑contained example to demonstrate the full workflow. In practice, replace the simulated dataset with your real features X and labels y.
Step 2 — Train/test split
Split the data into training and test sets. Train on the former and evaluate on the latter to obtain unbiased estimates of performance and decision utility.
Step 3 — Base model
Fit a simple baseline model (logistic regression here) to produce predicted probabilities for the outcome of interest. Any probabilistic classifier can be used as long as it outputs calibrated probabilities.
Step 4 — Probability calibration (if needed)
If probabilities are miscalibrated around clinically relevant thresholds, calibrate them using Platt scaling or isotonic regression with proper cross‑validation. DCA relies on well‑calibrated probabilities near the decision thresholds.
Step 5 — Decision curve calculation
For a clinically justified range of threshold probabilities, compute net benefit for three strategies: Treat none, Treat all, and the Model. This shows the clinical utility of using the model versus simple baselines.
Step 6 — Uncertainty via bootstrap
Use bootstrap resampling of the test set to obtain confidence intervals for net benefit curves and key threshold points. Report CIs for both single‑model curves and differences between models.
Step 7 — Figures and saving
Generate four outputs: decision curve, model comparison, clinical impact plot, and calibration with threshold markers. Save figures to disk and embed them in this page for reporting and discussion.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.calibration import calibration_curve
from sklearn.isotonic import IsotonicRegression
# -------------------------------
# 1) Simulated dataset (replace with your data)
# -------------------------------
RNG_SEED = 42
np.random.seed(RNG_SEED)
X, y = make_classification(
n_samples=2000,
n_features=10,
n_informative=6,
n_redundant=4,
weights=[0.7, 0.3], # ~30% event rate
flip_y=0.05,
random_state=RNG_SEED,
)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=RNG_SEED
)
# -------------------------------
# 2) Base model
# -------------------------------
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
y_proba_raw = model.predict_proba(X_test)[:, 1]
# -------------------------------
# 3) Optional post-hoc calibration (isotonic)
# Fit on train via CV in real studies. Here, a simple holdout for demo.
# -------------------------------
# Map raw scores from (X_train) via isotonic then apply to (X_test) would require CV.
# For demonstration, we calibrate on test predictions vs. test labels cautiously.
ir = IsotonicRegression(out_of_bounds='clip')
ir.fit(y_proba_raw, y_test)
y_proba = ir.transform(y_proba_raw)
# -------------------------------
# Helpers
# -------------------------------
def net_benefit(y_true, y_prob, thresholds):
y_true = np.asarray(y_true)
y_prob = np.asarray(y_prob)
n = len(y_true)
nb = []
for t in thresholds:
pred = (y_prob >= t).astype(int)
tp = np.sum((y_true == 1) & (pred == 1))
fp = np.sum((y_true == 0) & (pred == 1))
nb_val = (tp / n) - (fp / n) * (t / (1 - t))
nb.append(nb_val)
return np.array(nb)
def net_benefit_at_threshold(y_true, y_prob, t):
y_true = np.asarray(y_true)
y_prob = np.asarray(y_prob)
n = len(y_true)
pred = (y_prob >= t).astype(int)
tp = np.sum((y_true == 1) & (pred == 1))
fp = np.sum((y_true == 0) & (pred == 1))
return (tp / n) - (fp / n) * (t / (1 - t))
def clinical_impact(y_true, y_prob, thresholds, per=100):
y_true = np.asarray(y_true)
y_prob = np.asarray(y_prob)
n = len(y_true)
tp_list, fp_list = [], []
for t in thresholds:
pred = (y_prob >= t).astype(int)
tp = np.sum((y_true == 1) & (pred == 1))
fp = np.sum((y_true == 0) & (pred == 1))
tp_list.append(tp * per / n)
fp_list.append(fp * per / n)
return np.array(tp_list), np.array(fp_list)
def bootstrap_ci_nb(y_true, y_prob, t, B=1000, rng=None):
rng = np.random.default_rng(rng)
n = len(y_true)
vals = np.empty(B)
for b in range(B):
idx = rng.integers(0, n, n)
vals[b] = net_benefit_at_threshold(y_true[idx], y_prob[idx], t)
return np.quantile(vals, [0.025, 0.5, 0.975])
# -------------------------------
# 4) Threshold range and baselines
# -------------------------------
thresholds = np.linspace(0.05, 0.50, 100) # adjust to clinical range
prevalence = np.mean(y_test)
nb_treat_all = prevalence - (1 - prevalence) * (thresholds / (1 - thresholds))
nb_treat_none = np.zeros_like(thresholds)
# -------------------------------
# 5) Figures
# -------------------------------
# Plot 1: DCA
nb_model = net_benefit(y_test, y_proba, thresholds)
plt.figure(figsize=(8, 5.5))
plt.plot(thresholds, nb_model, label='Prediction model', color='steelblue', lw=2.5)
plt.plot(thresholds, nb_treat_all, label='Treat all', color='crimson', ls='--', lw=2)
plt.plot(thresholds, nb_treat_none, label='Treat none', color='gray', ls=':', lw=2)
plt.xlabel('Threshold probability')
plt.ylabel('Net Benefit')
plt.title('Decision Curve Analysis')
plt.legend(frameon=False)
plt.grid(True, ls='--', alpha=0.6)
plt.xlim([thresholds.min(), thresholds.max()])
plt.tight_layout()
plt.savefig('figure_dca.png', dpi=300, bbox_inches='tight')
# Plot 2: Clinical Impact (per 100 patients)
thresholds_ci = np.linspace(0.05, 0.50, 60)
tp_vals, fp_vals = clinical_impact(y_test, y_proba, thresholds_ci, per=100)
plt.figure(figsize=(8, 5.5))
plt.plot(thresholds_ci, tp_vals, label='True positives per 100', color='green', lw=2.5)
plt.plot(thresholds_ci, fp_vals, label='False positives per 100', color='orange', lw=2.5)
plt.xlabel('Threshold probability')
plt.ylabel('Number of patients (per 100)')
plt.title('Clinical Impact Plot')
plt.legend(frameon=False)
plt.grid(True, ls='--', alpha=0.6)
plt.xlim([thresholds_ci.min(), thresholds_ci.max()])
plt.tight_layout()
plt.savefig('figure_clinical_impact.png', dpi=300, bbox_inches='tight')
# Plot 3: Model comparison (simulate weaker model)
y_proba_weak = np.clip(y_proba + np.random.normal(0, 0.15, size=y_proba.shape), 0, 1)
nb_strong = net_benefit(y_test, y_proba, thresholds)
nb_weak = net_benefit(y_test, y_proba_weak, thresholds)
plt.figure(figsize=(8, 5.5))
plt.plot(thresholds, nb_strong, label='Enhanced model', color='steelblue', lw=2.5)
plt.plot(thresholds, nb_weak, label='Basic model', color='purple', ls='-.', lw=2.5)
plt.plot(thresholds, nb_treat_all, label='Treat all', color='crimson', ls='--', lw=2)
plt.plot(thresholds, nb_treat_none, label='Treat none', color='gray', ls=':', lw=2)
plt.xlabel('Threshold probability')
plt.ylabel('Net Benefit')
plt.title('Model Comparison via Net Benefit')
plt.legend(frameon=False)
plt.grid(True, ls='--', alpha=0.6)
plt.xlim([thresholds.min(), thresholds.max()])
plt.tight_layout()
plt.savefig('figure_model_comparison.png', dpi=300, bbox_inches='tight')
# Plot 4: Calibration with threshold markers
frac_pos, mean_pred = calibration_curve(y_test, y_proba, n_bins=10, strategy='quantile')
plt.figure(figsize=(7, 7))
plt.plot(mean_pred, frac_pos, 's-', color='darkblue', label='Model', lw=2, ms=6)
plt.plot([0, 1], [0, 1], 'k:', label='Perfect calibration', lw=1.5)
for th in [0.1, 0.2, 0.3]:
plt.axvline(x=th, color='gray', ls='--', alpha=0.7)
plt.text(th + 0.01, 0.02, f'{int(th*100)}%', rotation=90, color='gray', fontsize=10)
plt.xlabel('Predicted probability')
plt.ylabel('Observed frequency')
plt.title('Calibration Plot (zoom to clinical range as needed)')
plt.legend(frameon=False)
plt.grid(True, ls='--', alpha=0.6)
plt.xlim([0, 1])
plt.ylim([0, 1])
plt.tight_layout()
plt.savefig('figure_calibration.png', dpi=300, bbox_inches='tight')
# -------------------------------
# 6) Example: bootstrap CIs at pre-specified thresholds
# -------------------------------
for t in [0.10, 0.20, 0.30]:
lo, med, hi = bootstrap_ci_nb(y_test, y_proba, t, B=500, rng=123)
print(f"Threshold {t:.2f}: NB median {med:.4f} (95% CI {lo:.4f} to {hi:.4f})")Reporting recommendations
- State the clinical rationale for the chosen threshold range. For prostate biopsy, 5–30% is commonly discussed in the literature.
- Provide decision curves with Treat all and Treat none baselines.
- Include calibration assessment, ideally focusing on the threshold range of interest.
- Quantify uncertainty with bootstrap CIs for net benefit and for differences between models at pre-specified thresholds.
- Validate externally when possible; results are population- and prevalence-dependent.
Limitations
- DCA is not a cost-effectiveness analysis, though it reflects preferences through
. - Miscalibration near decision thresholds can mislead net benefit.
- Model utility depends on implementation burden and downstream harms, not just curve separation.
.References
- Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making, 2006.
- Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models. Epidemiology, 2016.
- Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: the Achilles heel of predictive analytics. BMJ, 2019.