The workflow supports model developers in iteratively understanding, diagnosing, and refining deep learning models using explainable AI (XAI) methods. Through an XAI pipeline instantiated in TensorBoard, analysts apply diverse explainer modules — ranging from high-abstraction model-agnostic methods (LIME, LRP, Saliency) to low-abstraction model-specific methods (weight statistics, activation trends) — to generate visual explanations that reveal model decision-making. Identified issues trigger refinement (architecture changes, hyperparameter tuning, data augmentation), producing new model states that are re-diagnosed and compared until the model is satisfactory.
The input consists of a trained neural network model (its computational graph, learned weights, and architecture), a training dataset, and a test dataset. The analyst inspects the model graph in an interactive node-link visualisation to understand the architecture: layer types, connections, activation functions, parameter counts, and data flow. Hovering or clicking graph nodes reveals wiki-style educational information and external references explaining each component's function. This understanding phase establishes the mental model needed for subsequent diagnosis — the analyst identifies which layers, parameters, and operations are present and how data flows through the network.
The analyst selects an input sample (e.g., an image from the test set) and applies high-abstraction explainers to understand the model's decision for that sample. LIME generates a linear surrogate model highlighting feature importance; LRP propagates relevance scores back to input pixels; Saliency and gradient-based methods visualise which input regions most influence the prediction. Simultaneously, the analyst applies low-abstraction explainers to individual graph nodes: HistoTrend shows weight distribution evolution over training, MinMax reveals value extremes, DeadWeight highlights unused parameters, and SaturatedWeight identifies over-saturated activations. The combination of high- and low-abstraction explanations reveals both what the model focuses on (decision explanation) and where internal issues lie (structural diagnosis). Findings are saved as provenance cards documenting evidence of problems.
Based on diagnosed issues (e.g., model focuses on irrelevant features, has dead weights, or misclassifies specific inputs), the system recommends refinements — transition functions that would modify the model state. Recommendations include adding convolutional layers for better spatial feature extraction, changing hyperparameters, adding training data, or modifying the loss function. The analyst selects and applies a refinement, producing a new model state (retrained network). They then re-apply explainers to the new state to verify improvement: for example, after adding convolutional layers to a digit classifier, LRP and Saliency should show the model now correctly focuses on the digit's shape rather than background noise. Multi-model explainers enable comparative analysis between the old and new states. This cycle — diagnose, refine, verify — repeats until the analyst judges the model satisfactory based on both performance metrics and explanation quality.
Throughout the process, the analyst accumulates provenance cards in a persistent cross-dashboard clipboard. In the reporting phase, they arrange, annotate, and structure these cards into a narrative documenting the model development history: initial issues found, refinements applied, improvements verified, and final model justification. The report serves multiple purposes: trust building (demonstrating the model focuses on relevant features), reproducibility (documenting the path through model state space), and communication (sharing with colleagues or stakeholders).