Using rubrics to scaffold learning. How the integration of criterion-referenced descriptors enhances student-centred formative assessment
MetadataShow full item record
AuthorRobinson Fryer, Bryan John; Olvera Lobo, María Dolores; Escabias-Machuca, Manuel; Gutiérrez Artacho, Juncal
Universidade Federal de Minas Gerais
Translator trainingRubricsFormative assessmentSelf-assessmentPeer-assessment
Robinson, B.J.; et al. Using rubrics to scaffold learning. How the integration of criterion-referenced descriptors enhances student-centred formative assessment. In: IATIS 5th International Conference: "Innovation paths in translation and intercultural studies". Belo Horizonte (Brazil), 7-10 July, 2015. [http://hdl.handle.net/10481/47434]
SponsorshipResearch group "Acceso y evaluación de la información científica" (HUM466), coordinated by Mª Dolores Olvera-Lobo.; University of Granada. Department of Translation and Interpreting
Context: Criterion-referenced descriptors offer a transparent approach to translator training that promotes the development of higher-order cognitive skills (Bloom 1973) - in particular analysis and evaluation, which are crucial to professional translators - and develops the interpersonal competences essential to efficient teamwork and specified in European tertiary education since the initiation of the Bologna process (Pagani 2002) but often ignored by academics. Descriptors in the form of rubrics provide students with scaffolding (Kiraly 1999, 2000, 2003) that supports and directs their development through essentially social constructivist activities (Robinson et al 2008), undertaken both in- and out-of-class, when the criteria they embody are in harmony with curricular objectives and the activities themselves allow for structured incremental growth in learning (Vygotsky 1978). Objectives: Our hypotheses are that (1) rubrics provide learners with tools they can learn to use and apply with substantial certainty that the grades awarded will gradually coincide with the tutor-set "standard" grades; that (2) the application of rubrics in team- and individual self- and peer-assessment activities will enhance the quality of their learning processes by developing higher-order cognitive skills; and that (3) the use of self- and peer-assessment of collaborative teamwork competences can broaden the learning experience at the tertiary level bringing actual learning closer to the aims of the Bologna process by including transverse competences. Method: In the present communication, we describe the use of rubrics as formative tools that provide valuable feedback in the context of our approach to their use in the classroom (Olvera-Lobo et al 2007; Robinson et al 2006; Robinson et al In press [a]). We draw on extensive data in order to measure their success in providing feedback during translation quality self- and peer-assessment workshops. Participants: Our sample consists of three consecutive generations of final year students (2010-11 n1=73; 2011-12 n2=73; 2012-13 n3=92) using a single rubric for self- and peer-, team and individual assessment (Robinson 1998). Furthermore, we present initial results on the use of a pilot rubric developed for the individual self- and peer-assessment of collaborative processes in team-based activities (Robinson In press [b]) with data drawn from the 2012-13 cohort (n3=92). Statistical analysis: We use the Shapiro-Wilks test to assess the normality of the grades awarded by individuals, teams and the tutor, ANOVA (for normally distributed grades), and the Kruskal-Wallis test and Friedman test (for non-normal distributions) to compare the average grade assigned by the different sources of variability (individual and team [self- and peer- awarded] and tutor) and detect possible differences between them. Finally, we use Cohen's kappa coefficient and the intraclass correlation coefficient, to compare interrater agreement in grades assigned by participants self, team and tutor. All statistical analysis is with R software. Conclusions: We believe our results will confirm the reliability of this approach and encourage the wider application of rubrics and the consequent collection of data from other contexts that will shed further light on their value in translator training.