Cognitive interviewing evidence on DIF in polytomous items of the student questionnaire of PISA

Abstract

Research on Differential Item Functioning (DIF) in international and cross-cultural assessments has focused much more on developing statistics than on explaining DIF results. Qualitative evidence from cognitive interviewing may be helpful in explaining DIF results. This study is part of a major research project aimed at investigating the causes of DIF by means of cognitive interviewing. Starting from DIF results obtained by analyzing US and Spanish versions of the Student Questionnaire of the Program for International Student Assessment (PISA, OECD, 2006), 25 cognitive interviews in the US and 25 in Spain are being conducted. Interview protocols were developed taking expert appraisal evidence into account. Interviewees respond to general and follow-up probes after answering each Student Questionnaire scale. Preliminary cognitive interviewing evidence on the 8 items flagged with large DIF is presented. Lastly, arguments for extending the use of cognitive interviewing to explain DIF results will be pointed out.