Confirmatory factor analysis of the framing agency survey

In this research paper, we investigate the structure and validity of survey data related to students' framing agency. In order to promote increased opportunities for students to engage in and learn to frame design problems that are innovative and empathetic, there is a need for instruments that can provide information about student progress and the quality of learning experiences. This is a complex problem because, compared to problem solving, design problem framing is less studied and harder to predict due to the higher levels of student agency involved. To address this issue, we developed a survey to measure framing agency, which is defined as opportunities to frame and reframe design problems and learn in the process. This study extends past research which focused on the construct of framing agency and developing an instrument to measure it following best practices in survey design, including using exploratory factor analysis of pilot data, which recovered six factors related to shared and individual consequentiality, problem structure and constrainedness, and learning. However, as a pilot, the sample limited generalizability; the current study addresses this limitation. We used a national cohort that included multiple engineering disciplines (biomedical, mechanical, chemical, electrical, computer, aerospace), types of formal design projects (e.g., first-year, design-spine, senior capstone) and institution types, including private religious; Hispanic-serving; public land-grant; and research flagship institutions (N=449). We report sample characteristics and used confirmatory factor analysis (CFA) to provide validity evidence, reporting the chi-square and standardized root mean square residual as estimates of fit. We report Cronbach's alpha as a measure of internal consistency. We found that overall, the CFA aligned with the prior exploratory results, in this case, recovering four factors, measured on a seven-point scale: shared consequentiality (the extent to which the student identifies that their understanding of the problem changed as result of a teammate's decision, M = 6.15; SD = 1.13); learning as consequentiality (the extent to which the student identifies learning as the result of decisions, M = 5.88; SD = 0.98); constrainedness (the extent to which the student reports the ability to make decisions despite design constraints, M = 4.95; SD = 1.49); and shared tentativeness (the extent to which the student identifies uncertainty about the problem and solution, M = 4.02; SD = 1.76). This suggests the survey can provide valid data for instructional decisions and further research into how students learn to frame engineering design problems and what role framing plays in their professional formation.