Development and validation of a 21st Century Skills Assessment: Using an iterative multimethod approach

In response to the call for more rigorously validated educational assessments, this study used an iterative multimethod validation process to develop and validate outcomes from the 21st Century Skills Assessment global rating scale. Qualitative and quantitative data sources were used to inform four types of validity evidence: content, response process, internal structure, and consequential. STEM experts and typical assessment users provided support for content, response process, and consequential validity evidence through focus group interviews and cognitive interviews. Psychometric findings suggest strong internal validity evidence for the assessment through high measures of internal consistency reliability, intra-rater agreement, and test–retest reliability measures.