Understanding Dimensions of Student Readiness from a Calculus Baseline Assessment Through Semi-Automatic Text Analysis and Clustering

In this article, we introduce a diagnostic tool that is intended to gauge the level of preparedness for students who are beginning their first undergraduate calculus course. The results gathered by this tool form a multi-dimensional view of student readiness through the qualitative coding of “explain your reasoning” prompts that are paired with multiple choice questions (MCQs). The codes are categorized into a taxonomy that gauge and observe various student skill sets. Manually coded responses are used as a training set for the fitting of a gradient boosting machine (GBM) model, which automatically codes responses at a fixed cost. Compared against a manual coder’s assessment of a held-out validation, the automatic coder averaged over 80% matching accuracy. Using these qualitative codes as vector dimensions, k-clustering of student demographic allows for the visualization of distinct snapshots of diverse student skill sets that exist within a large first-year undergraduate math class. These visualizations are generated as spider plots.