## ATWL Workflow Review Report **Paper Title:** SOMFlow: Guided Exploratory Cluster Analysis with Self-Organizing Maps and Analytic Provenance **Workflow ID:** SOMFlow --- ### Overall Assessment **Summary:** This is a very well-constructed ATWL representation. The template matches the body exactly across a long, non-trivial 15-step loop; every feature, visualisation, model, knowledge, and specification artifact is correctly typed with no `representation form`-substitution errors anywhere; the Pattern-A loop closes its feedback correctly; and the workflow's loop `purpose` field explicitly and accurately mirrors the paper's own four abstract exploration tasks (analyze, adapt, partition, reflect). The main gap is a representational one rather than a structural bug: no `contextualise` transform or `arrangement` artifact is used to capture the paper's explicit, switchable cell/graph layout mechanisms, even though every other reviewed workflow (and the paper itself) treats this as a distinct analytical step. **Recommendation:** - [ ] Approve as-is - [x] Approve with minor revisions - [ ] Requires revision - [ ] Reject - major rework needed --- ### Strengths - Template-to-body correspondence is exact across all 15 loop-body steps plus the two pre-loop and one post-loop transform — an unusually long and precisely matched sequence. - Every single feature artifact uses `value structure` (+ optional `value type`/`representation form`) correctly; every visualisation uses `layout`/`form`/`encoding` correctly. Not one instance of the common `representation form`-as-substitute anti-pattern occurs anywhere in this workflow — a notable achievement given its size. - `T_assign_items_to_cells` is correctly typed as `characterise` (not `define-unit`), precisely matching the "apply an existing model to assign labels/membership" rule, exactly analogous to the correct precedent set in DPVis's `T_infer_visit_states`. - `D_analysis_graph` is a textbook-correct application of the single-entity `formation` pattern (no `embedment`, since it represents one graph), matching the precedent set by `D_aggregated_network` in the NodeTrix review. - The loop's `purpose` field ("Iteratively analyse cluster-map results, adapt computations, create finer-grained data partitions, and preserve an overview of alternative analytical branches") maps almost word-for-word onto the paper's four abstract exploration tasks (analyze/adapt/partition/reflect, Section 4, Fig. 2) — an excellent, precise paper-alignment choice. - `F_cell_topology` is correctly typed `relational configuration` with a clarifying `representation form` ("neighbourhood relations between map cells"), appropriately capturing the u-matrix/force-directed-topology concept (Fig. 4C–D) without naming the specific technique. - Pattern A is implemented correctly: the `then` branch contains three substantive transforms and a complete `assign` block closing the feedback loop for all three loop-carried artifacts (`S_exploration_configuration`, `D_current_subset`, `D_analysis_graph`); the `else` branch correctly contains `exit loop`. - No algorithm names (SOM, Kohonen, k-means, prefuse), library names, or numeric thresholds (qeT, simT) leak into `manner`, `model type`, or `description` fields — excellent, disciplined abstraction level throughout a paper that is rich in such implementation detail. --- ### Critical Issues (Must Fix) None identified. All transform inputs/outputs reference declared artifacts, the artifact dependency graph is acyclic outside the sanctioned loop assignments, every non-given artifact has a producer, and no dead-end artifacts occur inside the loop conditional. --- ### Recommended Improvements (Should Fix) #### Improvement 1: Missing `contextualise` transform and `arrangement` artifact for cell/graph layout **Location:** `T_define_map_cells` → `D_map_cells`; `T_visualise_cluster_map`; `T_visualise_analysis_provenance` **Suggestion:** The paper explicitly describes the SOM grid layout as switchable between the native grid positions and a separately computed force-directed layout ("we allow the user to switch to a force-directed graph layout that is using the distances of cell neighbors as spring forces," Section 4.1, Fig. 4D), and likewise uses a force-directed layout (via prefuse) for the provenance flow graph itself (Section 6). Both are genuinely *computed* contexts in the ATWL sense — exactly the "data-driven grid" example given in the specification (`contextualise` producing `D_proj_space, A_projection`). Currently, `D_map_cells` carries `embedment: space` directly with no accompanying `arrangement`, and `V_cluster_map`/`V_analysis_flow` derive layout positions with no referenced arrangement artifact at all — unlike every other reviewed workflow (UTOPIAN, EventThread, NodeTrix, HITL, cluster-calendar), all of which pair a `contextualise` transform with an explicit `arrangement` before the corresponding `visualise` step. ``` # SUGGESTED ADDITION (inside the loop, after T_define_map_cells): transform T_layout_map_cells : intent: contextualise manner: "grid-based or force-directed cell layout" input: D_map_cells, F_cell_topology, S_exploration_configuration output: D_cell_layout_space, A_map_cell_layout actor: machine description: "Position map cells within a two-dimensional display space, either at their native grid coordinates or via a force-directed layout based on inter-cell topology" artifact D_cell_layout_space : entities internal structure: elementary features: - id: dimensions value structure: atomic value type: numeric description: "Number of display dimensions" description: "Computed two-dimensional display space for the cluster-map overview" artifact A_map_cell_layout : arrangement(D_map_cells) context: D_cell_layout_space principle: "native grid position or force-directed layout based on cell topology" description: "Placement of map cells either in their default grid positions or rearranged by inter-cell similarity" # T_visualise_cluster_map should then take A_map_cell_layout as input # instead of deriving "cell position from cluster-map topology" ad hoc. ``` A parallel (optional) addition could introduce `T_layout_analysis_graph` for `D_analysis_graph`'s force-directed node positions, consumed by `T_visualise_analysis_provenance`. **Why:** `contextualise` is entirely absent from the template despite the paper describing a genuinely separate, switchable layout-computation step; adding it brings this workflow in line with the layout-modelling convention established across all other reviewed workflows and makes the switchable-layout contribution explicit and comparable. #### Improvement 2: Manual annotation scope omits cell-level labelling **Location:** `transform T_annotate_current_result`, `artifact F_manual_annotations` **Suggestion:** The paper explicitly supports labelling at the **cell** level, not just the item level: "Analysts can assign user-defined labels to each cell" (Section 4.2), and Figure 1–C shows "manual annotations" applied directly to selected SOM cells to filter measurement noise. The current transform takes `D_map_cells` as input but only outputs `F_manual_annotations : feature(D_current_subset)`, omitting a cell-level annotation feature entirely. ``` # SUGGESTED: transform T_annotate_current_result : intent: characterise manner: "manual annotation of cells and items" input: D_current_subset, D_map_cells, V_cluster_map, P_cluster_patterns output: F_manual_annotations, F_cell_annotations actor: human description: "Assign analyst-defined labels to selected cells or time-series items for documenting interpretations, data issues, or analytical relevance" artifact F_cell_annotations : feature(D_map_cells) value structure: list value type: categorical description: "Analyst-defined labels marking cells as noisy, relevant, or otherwise notable" ``` **Why:** This is an explicitly described, dedicated interaction (not an implementation detail) that is otherwise entirely unrepresented at the cell level. #### Improvement 3: `T_compute_guidance_features` bundles conceptually dependent outputs into one transform **Location:** `transform T_compute_guidance_features` **Suggestion:** `F_partition_cues` is, per the paper, *derived from* the other three outputs (`F_cell_quality`-type qe thresholding, `F_metadata_association` interestingness ranking, and meta-clustering — Section 5.2's "three different types of actions"), not computed independently and in parallel with them. Consider splitting into a `characterise` transform producing `F_cell_topology`, `F_cell_quality`, `F_metadata_association`, followed by a separate transform (still `characterise`, or arguably `generate-knowledge`/`assess`-adjacent given its recommendation nature) that derives `F_partition_cues` from these three: ``` transform T_characterise_cell_guidance : intent: characterise input: D_map_cells, D_current_subset, F_cell_profile, F_cell_density, F_cell_variation, F_item_cell_membership output: F_cell_topology, F_cell_quality, F_metadata_association transform T_compute_partition_cues : intent: characterise manner: "rank candidate refinement and partitioning actions" input: F_cell_quality, F_metadata_association, D_map_cells output: F_partition_cues ``` **Why:** Making this dependency explicit in the dataflow improves clarity of how recommendations are derived and better matches the paper's own staged description of the recommendation pipeline (Section 5.1 → 5.2). --- ### Minor Suggestions (Optional) - `F_cell_variation` ("within-cell profile variation indicating local aggregation precision") and `F_cell_quality` ("indicators of aggregation precision and separation from neighbouring cells") overlap in describing "aggregation precision" twice. Consider rephrasing `F_cell_quality`'s description to focus solely on the separation/u-matrix aspect, since within-cell precision is already captured by `F_cell_variation`. - `T_initialise_analysis_subset` takes only `D_time_series` as input (no specification) yet its manner ("select initial analysis collection") implies some selection logic. Since the initial subset is simply the entire dataset (per Section 4, "The analyst is presented with a visualization of the entire dataset in the beginning"), consider either simplifying this to a pre-loop `assign: D_current_subset := D_time_series`, or clarifying in the description that no actual filtering occurs at this stage. - Consider adding a brief note (e.g., in `D_analysis_graph`'s description or `T_visualise_analysis_provenance`'s description) clarifying that cross-SOM comparisons (shared-item highlighting, Fig. 5–A) and back-propagated annotations (Fig. 8–C) rely on state snapshots embedded within each registered graph node across iterations, not solely on the current iteration's `F_item_cell_membership`/`F_manual_annotations` — this is implicit but not currently made explicit. - The workflow models iterative refinement as a single evolving `D_current_subset`/`D_analysis_graph` pair rather than explicitly preserving multiple simultaneously-active exploration branches for later revisitation (a capability central to the paper's "reflect" task and Fig. 10's multi-branch graph). This is a reasonable ATWL-level simplification given the loop construct's linear nature, and `D_analysis_graph`'s cumulative structure does partially preserve this information, but it is worth noting as an inherent limitation of representing branching, non-linear exploration within a single Pattern-A loop. --- ### Alignment with Paper Captured from paper: - [pass] Main methodology phases (series preparation → SOM fitting → cell/membership characterisation → guidance-feature computation → provenance registration → coordinated visualisation → interpretation → annotation → assessment → partitioning) - [partial] Iterative refinement procedures (the four abstract exploration tasks — analyze, adapt, partition, reflect — are precisely captured in the loop `purpose`; the switchable grid/force-directed layout mechanism described in Sections 4.1 and 6 is not represented — see Improvement 1) - [pass] Key methodological contributions (SOM-based cell characterisation, quality/interestingness-based guidance cues, partition-based divide-and-conquer exploration, and analytic-provenance flow graph are all represented at an appropriate abstraction level) - [pass] Domain terminology (cluster map, cells, partitions, analysis graph — consistent with the paper's own vocabulary while remaining generic) Missing or incorrect: - Switchable grid/force-directed layout computation for cells and provenance graph nodes (Improvement 1) - Cell-level manual annotation capability (Improvement 2) --- ### Conclusion This SOMFlow workflow is among the strongest representations reviewed to date: its artifact typing is essentially flawless, its template-body correspondence is exact despite considerable length, and its loop structure precisely mirrors the paper's own four-task conceptual framing. The principal area for improvement is the absence of an explicit `contextualise`/`arrangement` pairing for the paper's switchable cell and graph layout mechanisms — a gap that is easily addressed with the additions suggested above without requiring any restructuring of the existing, well-formed loop. With these revisions, this workflow would be an exemplary ATWL representation.