Analyses of the designs produced in the 2×2×7 experiment. The designs that were written in ATWL are analysed with features computed from the ATWL text and with the ATWL syntax checker; the prose designs (C1 and C4) are coded under a fixed protocol, so that conditions with and without ATWL can be compared. The comparison of the library papers and methods used by C3, C4 and C5 is on a separate page.
ATWL-side analysis
25 formal features, ATWL syntax checker and descriptors for the 20 ATWL designs (C2, C3, C6, C6++, C7); comparison with the library and between conditions, models and tasks.
Coding of the prose designs
Protocol, inputs and outputs of the two LLM coders for the 8 prose designs (C1, C4), with data and script.
Comparisons enabled by the coding
C4 against C1, model and task, the 2×2 across prose and ATWL designs, the lineage from C4 to C6, C6++ and C7, and the explicitness of prose and ATWL.
Reliability of the coding
Validation of the coder outputs, agreement between the coders and the development of the protocol.