DPVis supports clinical researchers in exploring disease progression pathways from longitudinal observational health records using Hidden Markov Models (HMMs). The workflow enables researchers to configure and evaluate HMMs that discover latent disease states from observed biomarkers, interpret state characteristics, explore state transition patterns across patient populations, iteratively build and compare patient subgroups, and derive clinical insights about disease etiology and heterogeneous progression trajectories.
The input consists of longitudinal observational data from clinical studies: patient records with multiple visits at irregular time points, each visit carrying observed variables (e.g., autoantibody measurements: GADA, IAA, IA2A) and static variables (e.g., genetic profiles, gender, family history, breastfeeding status). Clinical researchers select output variables for the HMM (those believed to manifest disease progression), specify the number of hidden states to discover, and set constraints on state transitions (e.g., no-backward transitions for chronic diseases where damage is irreversible).
The system trains HMMs with a range of state counts (e.g., 2–20 states). Each trained model infers a hidden state per patient visit based on the observed variables, producing posterior probability distributions over states for every visit. Cross-validation is performed to compute log-likelihood fitness scores. The researcher evaluates candidate models — inspecting whether the discovered states are clinically meaningful and whether the number of states provides adequate resolution without over-fragmentation — and selects the best-performing model (e.g., an 11-state autoantibody model).
The researcher inspects the Feature Matrix (mean of each observed variable per state, colour-coded on a diverging blue-red scale) and the Feature Distribution (mean, standard deviation, histogram, and missing-data indicators per cell) to understand what each discovered state represents clinically. For example, states with all-blue cells indicate no autoantibodies (pre-autoimmunity), while states with red in specific rows indicate particular antibody combinations. The researcher categorises states by clinical meaning (e.g., 0-AAB, 1-AAB, 2-AAB, 3-AAB states) and assesses whether the model's states correspond to clinically recognized disease stages.
The researcher explores how patients transition between states over time using multiple coordinated pathway views: Pathway Waterfall (beeswarm plots with force-edge-bundled trajectory lines showing individual progression paths over age), Pathway over Observation (Sankey diagram aggregating transitions over visits), Pathway by Time Unit (stacked bars aggregating states per age unit), State Transition Chord Diagram (radial network showing overall transition frequencies), and State Transition Pattern List (BIDE-mined frequent sequential patterns ranked by subject count). From these views, the researcher identifies major trajectory groups (e.g., "Multi-AAB First", "IAA First", "GADA First") and observes differences in progression speed and timing.
The researcher iteratively constructs patient subgroups using multiple mechanisms: filtering by state transitions in Pathway Waterfall (clicking transition points), building complex queries in State Sequence Finder (drag-and-drop node-link query with constraints on time, visit position, and posterior probability), filtering by static variables (clicking bar chart segments), and selecting frequent patterns from the Pattern List. For each subgroup, they inspect Dual Kernel Densities (comparing distributions of diagnosis age and seroconversion age between subgroup and population), Subject List View (individual state sequences with posterior pie charts), and Static Variables View (genetic profiles, demographics). They compare subgroups to identify associations between progression trajectories and health outcomes or genetic factors. This cycle repeats — building subgroups, comparing distributions, refining criteria, discovering anomalies — until the researcher has characterised all clinically meaningful trajectory groups and their distinguishing factors.
The researcher synthesises clinical insights: confirmed or novel trajectory groups (e.g., early IAA progressors), associations between genetic profiles and progression speed, anomalous patients requiring further investigation (e.g., diagnosed patients with no autoantibody state assignments suggesting measurement error or novel endotypes), and implications for early intervention and precision medicine.