Supplementary · Experiment Appendix

The Scaffold Experiment

A controlled 2×2 study of how an AI assistant designs a visual-analytics workflow, with and without ATWL scaffolds, across two tasks.

2×2

factorial design

independent tasks

scaffold conditions each

model, parallel sessions

This appendix documents the controlled experiment behind the paper’s section Validating the Scaffold Effect. We asked a single frontier model to design a visual-analytics workflow under four starting conditions, crossing the presence of the ATWL language definition with the presence of an ATWL library of worked examples. We ran the experiment for two unrelated tasks, then tested multi-pass prompting strategies.

·The four conditions

Variant	ATWL language definition	ATWL library of examples
V1 — none	–	–
V2 — language	in context	–
V3 — library	–	in context
V4 — both	in context	in context

·The two tasks

SoftSky (apartment MCDA). Design a visual-analytics workflow for multi-criteria decision analysis over ~12,800 Leipzig rental offers: spatial focusing, user-defined utilities, a tolerant Pareto frontier, and grouping of survivors into kinds. COVID-19 pandemic behaviour. Design a workflow to analyse daily European COVID-19 time series — levels, trends, episodes, country grouping, and policy association.

·What the comparison found

The same pattern held in both tasks:

No scaffold (V1) — most creative, least formal. Produced usage scenarios, method-comparison tables, and explicit formulas that no scaffolded variant included, but with no typed structure.
Language definition (V2) — most rigorous specification. The grammar disciplined the output into fully typed artifacts and transforms, at the cost of conceptual exploration.
Library (V3) — best conceptual architecture. Worked examples prompted pattern-level thinking (e.g. the dual-decomposition idea), yielding the most sophisticated pipeline.
Both (V4) — competent but template-filling. Balanced and well-formed, yet not the best on any single dimension we assessed; attention split between two attractors.
Displacement. Outputs were roughly equal in length, so a formal template in context displaces harder-to-generate analytical content. Introducing scaffolds sequentially, after a creative first pass, recovers it; a two-pass strategy was the most efficient.

How to read these results. The conditions ran on a single model with parallel sessions, and the assessments are the authors’ qualitative judgments along three dimensions (coverage, formal correctness, architectural sophistication) — not a blind, multi-rater measurement, and we are the authors of ATWL. The full transcripts are provided here precisely so readers can form their own judgment.

·Pages

SoftSky — apartment MCDAShared brief, the 4-variant comparison, and the four variant transcripts (V1–V4). COVID-19 pandemic behaviourShared brief, the 4-variant comparison, and the four variant transcripts (V1–V4). Multi-pass & cross-study synthesisThe displacement analysis, cross-task synthesis, and the three- and two-pass results.

·Files included

File	Task	Contents
`SoftSky-v1 … v4`	SoftSky	Single-pass output under each condition
`SoftSky-cmp`	SoftSky	Cross-variant comparison + summary tables
`covid-v1 … v4`	COVID-19	Single-pass output under each condition
`covid-cmp`	COVID-19	Cross-variant comparison + summary table
`SoftSky-twoPrompts`, `covid-twoPrompts`	both	Two-pass runs (unconstrained → refine + formalize)
`twoUseCases-cmp…`	cross-task	Displacement analysis, cross-study and 3-/2-pass synthesis

For brevity, near-duplicate incremental runs (*-v1_lib, *-v1_lib_lang, SoftSky-cmp1) are not rendered separately; their content is represented by the two-pass runs and the synthesis page.