Overview
20responses included
14used the web demonstrator
6did not use the web demonstrator
14completed the comparison questions for both tools
5.35 / 6mean total score of the three comprehension probes (maximum 6)
The page reports counts and distributions. It uses no significance tests and makes no claims about the effects of the notebook or the web demonstrator.
A – Background
Self-reported background of the 20 respondents.
| Item | Answer | Count | Share |
| A3 Applied a clustering algorithm outside a tutorial | Yes | 8 | 40% |
| No | 12 | 60% |
| A4 Before, ran the same method with several parameter settings on the same data | Yes | 4 | 20% |
| No | 16 | 80% |
| A9 Attended the lecture on clustering | Yes | 20 | 100% |
12345
| Item | Distribution | n | Mean | Median |
| A5 Familiarity with Python / scikit-learn (1 none, 5 fluent) | | 20 | 3.20 | 3 |
| A6 Familiarity with clustering validity indices (1 none, 5 fluent) | | 20 | 2.75 | 3 |
| A7 Experience with Jupyter or Colab notebooks (1 none, 5 daily use) | | 20 | 3.45 | 3 |
| A8 Experience with interactive web-based visualization tools (1 none, 5 daily use) | | 20 | 2.70 | 3 |
B – Notebook
Asked of all 20 respondents. Usefulness (B4) is rated from 1 (not useful) to 5 (very useful). Agreement items use 1 (strongly disagree) to 5 (strongly agree). Mental demand (B15) uses 1 (very low) to 5 (very high).
| Item | Answer | Count | Share |
| B1 How far did you get in class? | Part 1 only | 6 | 30% |
| Part 1 and part of Part 2 | 12 | 60% |
| Parts 1 and 2 | 2 | 10% |
| B2 Continued with the notebook after the class? | No | 11 | 55% |
| Yes, Part 2 | 9 | 45% |
| B3 The time in class was sufficient. | Yes | 6 | 30% |
| Partly | 8 | 40% |
| No | 6 | 30% |
Usefulness of components (B4)
| Item | Distribution | n | Mean | Median |
| B4a Sweep plots | | 20 | 4.40 | 5 |
| B4b Maps and space–time cubes | | 20 | 3.60 | 3.5 |
| B4c Persistence and transition displays | | 20 | 4.15 | 4 |
| B4d Assignment confidence (silhouette) plots | | 20 | 3.40 | 3.5 |
| B4e Calendar and profile plots | | 20 | 3.30 | 3 |
| B4f Interactive sliders | | 20 | 3.40 | 3.5 |
| B4g Prediction checkpoints | | 20 | 3.40 | 3 |
| B4h The notes “What this shows” | | 20 | 4.20 | 4.5 |
| B4i Audit exercise (Part 3) | | 20 | 3.40 | 3 |
Complexity, guidance and interaction (B5–B17)
Bars show the responses as given, from 1 (strongly disagree) to 5 (strongly agree), also for negatively worded and reverse-keyed items. For these items, high values are the unfavourable answer.
| Item | Distribution | n | Mean | Median |
| B5 I knew what to do next. | | 20 | 4.15 | 4.5 |
| B6 There was too much information at once. (negatively worded) | | 20 | 3.00 | 3 |
| B7 The available options were manageable. | | 20 | 2.90 | 3 |
| B8 The notes helped me understand what to look at. | | 20 | 4.00 | 4 |
| B9 I needed more guidance on how to interpret the plots. (negatively worded) | | 20 | 3.00 | 3 |
| B10 I could compare results from different parameter settings. | | 20 | 3.95 | 4 |
| B11 I could see how groups changed at neighbouring parameter values. | | 20 | 2.95 | 3 |
| B12 The interface responded quickly enough when I changed parameters. | | 20 | 3.95 | 4 |
| B13 The prediction checkpoints helped me check my understanding. | | 20 | 3.75 | 4 |
| B14 I could use the notebook without reading the code. | | 20 | 3.00 | 3 |
| B15 Mental demand of working with the notebook (1 very low, 5 very high). | | 20 | 3.55 | 4 |
| B16 I understood what persistence and transitions showed. | | 20 | 3.50 | 3.5 |
| B17 I understood what assignment confidence showed. | | 20 | 3.30 | 3 |
C – Web demonstrator
Fourteen respondents used the web demonstrator. Six did not.
| Item | Answer | Count | Share |
| C3 Total time with the web demonstrator | < 15 min | 4 | 29% |
| 15–30 min | 3 | 21% |
| 30–60 min | 1 | 7% |
| > 60 | 6 | 43% |
| C4 What students did (several answers possible) | Looked at precomputed Flickr sweeps | 14 | 100% |
| Ran own sweeps (Explore) | 9 | 64% |
| Looked at other example data sets | 5 | 36% |
Reasons for not using it
| Item | Answer | Count | Share |
| C2 Reason for not using it | Focused on the notebook | 3 | 50% |
| Not interested | 2 | 33% |
| Did not know how to start | 1 | 17% |
Usefulness of components (C5), 14 users
| Item | Distribution | n | Mean | Median |
| C5a Quality-metric charts | | 14 | 4.00 | 4 |
| C5b Transition (Sankey) view | | 14 | 4.00 | 4 |
| C5c Confidence violin plots | | 14 | 3.29 | 3 |
| C5d Group embedding | | 14 | 3.86 | 4 |
| C5e Map and space–time cube | | 14 | 3.57 | 3 |
| C5f Calendar and profile views | | 14 | 3.64 | 4 |
| C5g Controls to change parameters | | 14 | 3.71 | 4 |
| C5h Explore mode | | 14 | 3.86 | 4 |
| C5i Workflow guide | | 14 | 3.71 | 4 |
Complexity, guidance and interaction (C6–C16), 14 users
Bars show the responses as given, from 1 (strongly disagree) to 5 (strongly agree), also for negatively worded and reverse-keyed items. For these items, high values are the unfavourable answer.
| Item | Distribution | n | Mean | Median |
| C6 I knew what to do next in the web demonstrator. | | 14 | 3.86 | 4 |
| C7 There was too much information at once in the web demonstrator. (negatively worded) | | 14 | 4.43 | 4.5 |
| C8 The available options in the web demonstrator were manageable. | | 14 | 3.64 | 4 |
| C9 The web-demonstrator notes helped me understand what to look at. | | 14 | 4.00 | 4 |
| C10 I needed more guidance on how to interpret the web-demonstrator plots. (negatively worded) | | 14 | 4.00 | 4 |
| C11 I could compare results from different parameter settings in the web demonstrator. | | 14 | 3.86 | 4 |
| C12 I could see how groups changed at neighbouring parameter values in the web demonstrator. | | 14 | 3.71 | 3 |
| C13 The web demonstrator responded quickly enough when I changed parameters. | | 14 | 3.79 | 4 |
| C14 The workflow guide helped me decide what to look at next. | | 14 | 4.14 | 4 |
| C15 I could start without help. | | 14 | 3.29 | 3 |
| C16 Mental demand of working with the web demonstrator (1 very low, 5 very high). | | 14 | 3.14 | 3 |
D – Preferences of students who used both tools
Answered by the 14 students who used both tools. All of them used the notebook first, so the answers are preferences in this sequence and not an isolated comparison of the tools.
NotebookNo differenceWeb demonstrator
| Question | Distribution | Notebook | No difference | Web demonstrator |
| D1 Which helped you more to understand how the result changes with the parameters? | | 7 | 1 | 6 |
| D2 Which helped you more to understand assignment confidence? | | 7 | 2 | 5 |
| D3 Which gave you better guidance on what to do next? | | 5 | 3 | 6 |
| D4 Which made it easier to try your own ideas? | | 4 | 4 | 6 |
| D5 Which would you prefer for learning about clustering in a course? | | 3 | 3 | 8 |
| D6 Which would you prefer for analysing your own data? | | 3 | 3 | 8 |
E – Concepts and comprehension
E1–E9 use 1 (strongly disagree) to 5 (strongly agree). E1 and E4 are reverse-keyed: agreement reflects a single-run or single-optimum view. E7–E9 are self-reports of intended change.
Bars show the responses as given, from 1 (strongly disagree) to 5 (strongly agree), also for negatively worded and reverse-keyed items. For these items, high values are the unfavourable answer.
| Item | Distribution | n | Mean | Median |
| E1 A clustering result is best judged by computing a validity index and taking the highest value. (reverse-keyed) | | 20 | 2.00 | 2 |
| E2 Two parameter settings that produce similar validity scores can support meaningfully different conclusions about the data. | | 20 | 4.05 | 4 |
| E3 If a cluster structure appears only for one parameter setting and disappears for neighbouring settings, that is evidence against relying on it. | | 20 | 4.00 | 4 |
| E4 Once the optimal number of clusters has been determined, examining other numbers adds little information. (reverse-keyed) | | 20 | 1.75 | 2 |
| E5 Knowing how confidently each individual item is assigned to its cluster is important for interpreting the result. | | 20 | 4.50 | 5 |
| E6 It can be a valid analytical outcome to conclude that none of the tested configurations should be used. | | 20 | 4.20 | 4 |
| E7 Because of the class, I would now run the same method with several parameter settings before choosing one. | | 20 | 4.00 | 4 |
| E8 Because of the class, I would now compare the groups across settings and not only the scores. | | 20 | 3.90 | 4 |
| E9 Because of the class, I would now check how confidently the items are assigned to their groups. | | 20 | 4.10 | 4 |
Comprehension probes (E10–E12)
Each probe was answered by selecting one of several answer options. Each option has a score of 0, 1 or 2. The table lists the options that respondents chose.
| Probe and answer chosen | Score | Count |
| E10 Advice before relying on one DBSCAN result (mean score 1.70) |
| I would check neighbouring values and use the transition display to see whether the group persists. I would also inspect its contents. | 2 | 8 |
| One run is not enough. I would vary epsilon and min_samples and see whether similar groups recur; then inspect the data context before making a claim. | 2 | 4 |
| I would rerun DBSCAN with several parameter settings, compare the clusters across nearby settings, and inspect whether they make sense on the map. I would also check noise and assignment confidence. | 2 | 2 |
| I would try a few settings and compare the silhouette scores before choosing. | 1 | 6 |
| E11 Meaning of a group that occurs at only one setting (mean score 2.00) |
| It probably means the group is not stable. I would check the neighbouring runs and whether its members make sense together. | 2 | 12 |
| If the group occurs only at K=6, it may be specific to that setting. I would compare K=5 and K=7 and inspect the transitions. | 2 | 8 |
| E12 Meaning of low assignment confidence (mean score 1.65) |
| Low confidence means some points are close to other clusters or the group may be heterogeneous. I would inspect them and test other settings. | 2 | 8 |
| The group may overlap with another group or contain mixed items; I would inspect the points and consider changing parameters. | 2 | 5 |
| It means the algorithm is unsure which group is correct. | 1 | 7 |
| Total score across the three probes | Distribution | Mean | Median |
|---|
| Total score, 0–6 | | 5.35 | 5.5 |
Reading and limits
- Responding was voluntary. Respondents are probably more engaged than the whole group, and their ratings are likely to be more favourable than those of non-respondents.
- 20 responses are too few for significance tests. The page shows counts and distributions only.
- There was one group, no control group and no pre-test. The page does not show learning gains or effects of the tools.
- The notebook came first and the web demonstrator second. The two differ in guidance, and these differences are part of what was rated.
- Ratings and the E7–E9 items are opinions, not performance. The comprehension probes were answered by selecting from fixed options, which is easier than producing an answer.
- The implementations use different libraries, random seeds and preprocessing. Results seen in the notebook and in the web demonstrator are comparable but not identical.