Background and context. Concept inventories (CIs) are a widely used tool in STEM education that can help instructors identify specific misconceptions students hold about key concepts. Over the past several years, much research has been published contributing to CIs in computer science education.
Objectives. This research aimed to investigate the state of CIs in computing education through a systematic literature review built around two research questions: 1) What do we know about CIs in computer science today, and 2) Have researchers been able to effectively tackle the 4 challenges specific to computer science CIs outlined by Taylor, et al. in their previous 2014 literature review of this topic?
Method. We conducted a systematic literature review focusing on CIs in computing education. We included research from any country provided it was published in English between 1990 and 2022. We searched with key terms across four databases (Compendex, Inspec, Education Source, and ERIC) and collated a final list of 65 papers for analysis from 175 initial results.
Findings. From these 65 papers, we identified 33 total CIs for computer science, 12 of which are currently validated. The majority of studies used established techniques for choosing concepts, identifying misconceptions, and/or developing questions, but a few (n = 4) reported on successful alternative techniques. We found that although no papers explicitly articulated an aim to tackle the aforementioned challenges, they are a recurring theme in the literature and have been addressed in various ways.
Implications. At the time of the previous literature review there were few computer science CIs. Our findings promisingly suggest that researchers have found effective ways to tackle Taylor, et al.’s challenges and that the number of CIs across computer science is gradually increasing. Future work is warranted regarding the alternative techniques researchers proposed for developing CIs.