Skip to content

09. Calibration

Jump to bottom
Nao Yamamoto edited this page Jul 1, 2026 · 2 revisions

This page describes how to calibrate disease parameters in MIGHTI so that simulated prevalence (and optionally cause-specific death rates) align with observed data.

Source code: mighti/calibration/

What We Calibrate

The primary workflow fits acquisition probability for each health condition:

  • p_acquire_female / p_acquire_male — sex-specific acquisition multipliers applied on top of seed values in the parameter CSV
  • p_death (optional) — joint fit against cause-specific death targets when {region}_death_rates.csv is available

HIV transmission parameters are calibrated separately (see HIV calibration below).

Calibration ensures model outputs reflect region-specific prevalence patterns. Incidence is not read directly from input files — it is inferred by matching simulated prevalence over time.

Scripts

Script Purpose
calibration_diseases_ver2.py Current workflow. Optuna search per condition for p_acquire (and optionally p_death)
diseases_for_calibration.py Calibration-specific disease class definitions (p_acquire=1 so the search fits a multiplier)
calibration_diseases.py Legacy calibration script (superseded by ver2 for new work)
calibration_hiv.py Calibrate HIV transmission betas against observed HIV prevalence

How ver2 Works

For each condition, calibration_diseases_ver2.py:

  1. Builds a simulation with HIV + one target condition (annual dt=1)
  2. Initializes prevalence from data/processed/{region}_prevalence.csv and {region}_prevalence_hiv.csv
  3. Runs an Optuna study minimizing a weighted objective:
    • Prevalence mismatch (by age/sex vs observed)
    • Death-rate mismatch (when {region}_death_rates.csv exists and --fit-pdeath is enabled)
  4. Writes best-fit parameters and diagnostic plots to a results folder

Conditions are skipped automatically when:

  • No observed prevalence data exist for the relevant sex
  • The disease class is missing from diseases_for_calibration.py
  • Prevalence is all zero (no calibration signal)

Non-fatal conditions

These conditions use p_death = 0 and skip the p_death search dimension:

AnxietyDisorder, BipolarDisorder, ChronicPain, Hyperlipidemia, Hypertension, Obesity, TobaccoUse

Running Calibration

From the repository root (requires optuna; install via pip install -r requirements.txt):

# Smoke test (5 trials, one condition)
python mighti/calibration/calibration_diseases_ver2.py \
    --conditions Type2Diabetes \
    --smoke

# Full run for selected conditions
python mighti/calibration/calibration_diseases_ver2.py \
    --conditions Type2Diabetes Hypertension COPD \
    --total-trials 500 \
    --results-dir mighti/calibration/results/my_run

# All conditions in the parameter CSV (default)
python mighti/calibration/calibration_diseases_ver2.py \
    --total-trials 500 \
    --results-dir mighti/calibration/results/calibration_ver2_eswatini

Useful CLI flags

Flag Description
--conditions One or more condition names (default: all in parameter CSV)
--total-trials Optuna trials per condition (default: 500)
--smoke Short run with 5 trials
--weight-prev Objective weight for prevalence fit
--weight-death Objective weight for death-rate fit
--fit-pdeath / --no-fit-pdeath Include or exclude p_death in the search
--pdeath-bound-mult Search bounds as multiples of seed p_death (default: 10×)
--results-dir Output directory; reuse the same path to accumulate all conditions into one CSV

Environment variables (optional): MIGHTI_CALIB_WEIGHT_PREV, MIGHTI_CALIB_WEIGHT_DEATH, MIGHTI_CALIB_FIT_PDEATH, MIGHTI_CALIB_PDEATH_BOUND_MULT.

Outputs

Results are written under --results-dir (default: mighti/calibration/results/calibration_ver2_{region}_{timestamp}/).

File Description
calibrated_p_acquire.csv Best-fit p_acquire_female, p_acquire_male, optional p_death_best, fit metrics
calibrated_p_acquire_aligned.csv Same values merged into parameter-file condition order (handoff table)
calibration_results_<Condition>.txt Per-condition summary
diagnostics/<Condition>/ Optuna plots (plot_optimization_history.png, plot_param_importances.png, etc.)

See mighti/calibration/README.md for how to interpret diagnostic plots.

Example aligned output columns:

condition,p_acquire_female,p_acquire_male,p_death
Type2Diabetes,0.00332,0.00205,0.000762

After Calibration

Update your region parameter file — not individual files in mighti/diseases/:

  1. Copy calibrated values from calibrated_p_acquire_aligned.csv into data/processed/{region}_parameters.csv
  2. Add or update p_acquire_female, p_acquire_male, and p_death columns as needed
  3. Re-run your full simulation (mighti_main.py or project script) and check prevalence with PrevalenceAnalyzer_HIV + plotting helpers

The production disease classes in mighti/diseases/ read parameters from the CSV at runtime; you do not need to hard-code calibrated values in Python unless you prefer that workflow.

For reproducible downstream studies, freeze a results folder and record a checksum (see FREEZE_MANIFEST.txt in example result directories).

HIV Calibration

HIV transmission (beta_m2f, beta_m2c) is calibrated separately:

python mighti/calibration/calibration_hiv.py \
    --region eswatini \
    --start 1990 \
    --stop 2023 \
    --n-agents 10000 \
    --trials 200

Inputs: {region}_prevalence_hiv.csv, {region}_asfr.csv, {region}_mortality_rates.csv.

Fixed HIV betas used during disease calibration are embedded in calibration_diseases_ver2.py (HIV_BETA_M2F, HIV_BETA_M2C).

Required Input Files

All under data/processed/ (or tests/test_data/ as fallback):

File Used for
{region}_parameters.csv Seed parameters and condition list
{region}_prevalence.csv Observed non-HIV prevalence targets
{region}_prevalence_hiv.csv HIV prevalence initialization
{region}_asfr.csv Fertility
{region}_mortality_rates.csv Background mortality
{region}_death_rates.csv Optional cause-specific death targets for joint p_death fit
rel_sus.csv Interaction matrix (used in full sims, not per-condition calibration sim)

See Parameter Preparation for file formats.

See Also

  • Parameter Preparation — how seed p_death and dur_condition are prepared
  • Health Conditions — disease classes used after calibration
  • Analyzers — PrevalenceAnalyzer_HIV, CauseDeathRateAnalyzer used during calibration
  • mighti/calibration/README.md — Optuna diagnostic interpretation

Clone this wiki locally