Piloting a Diagnostic Tool to Measure AP CS Principles Teachers' Knowledge Against CSTA Teacher Standard 1

Problem. Understanding computer science (CS) teacher CS knowledge primarily relies on self-reported data from participants to understand the learning impact on teachers and improve teacher growth. There is currently a lack of quality instruments to determine where CS teachers need to improve their knowledge. Research Question. Our research question for this project was: What are the preliminary psychometric properties of a developed measure for AP CS Principles teachers? Methodology. We developed and piloted a diagnostic tool for high school teachers (n=18) who have been or will be engaged in teaching AP Computer Science Principles (AP CSP). We then administered the diagnostic with two groups of teachers at the CSTA PD week in 2023, and analyzed the results. Findings. The full 22-item measure demonstrated acceptable reliability (alpha = 0.74) in the present sample. However, four items were identified as "low performing"based on low discrimination values. Further, despite the adequate reliability of the full scale, reliability of the individual subscales was lower (0.29-0.54). This may have been caused by low sample size and/or the lower number of items included at the subscale level. Implications. As a pilot, our analysis of the diagnostic indicated that reliability can be improved by revising certain items. We will use this information to revise these items, then pilot the diagnostic again next year with a larger set of teachers. This diagnostic can then be used by CSTA and the broader CS education community for their repertoire of tools designed to inform professional development topics and practices.