Based on my experience working with language teachers, assessing spoken language often poses a number of serious challenges for L2 teachers. Should the assessment focus be on fluency or accuracy? Does poor grammar equate to poor content? Should students automatically fail a speaking task if they have intelligibility problems even though the content might be quite good? How and to what extent should pronunciation be considered in oral assessment? During my time as ISEP coordinator, in an attempt to address some of these questions, I designed criterion-based rubrics (i.e. analytic rubrics) for teachers to use in their speaking classes (see below for an example rubric). The rubrics standardized and hopefully increased the reliability and validity of assessment practices in the program; nonetheless, they didn’t account for one particular factor that I generally find difficult to control: accent familiarity. Having lived in Japan, for example, I am quite familiar with the way Japanese speak English, and research provides ample evidence that familiarity with speakers' accented speech (and with their cultural and linguistic background) does indeed influence listener/rater comprehension of spoken language (e.g. Derwing & Munro, 1997; Major, Fitzmaurice, Bunta, & Balasubramanian, 2002; Munro & Derwing, 1995). What can be done to minimize or control this particular variable? Videotaping speaking tasks to assess student performances is something I have seen teachers use. I must admit, I haven't experimented with this, but the fact that teachers can replay a recording several times seems to me an interesting, albeit somewhat time-consuming, solution. Does anybody have any other suggestions?
Derwing, T. M., & Munro, M. J. (1997). Accent, intelligibility and comprehensibility: Evidence from four L1s. Studies in Second Language Acquisition, 20, 1-16.
Major, R. C., Fitzmaurice, S. F., Bunta, F., & Balasubramanian, C. (2002). The effects of nonnative accents on listening comprehension: Implications for ESL assessment. TESOL Quarterly, 36(2), 173-190.
Munro, M. J., & Derwing, T. M. (1995). Foreign accent, comprehensibility and intelligibility in the speech of second language learners. Language Learning, 45(1), 73-97.
I am a Lecturer in TESOL at the University of Wollongong in Australia. I blog about L2 learning, L2 teaching, L2 teacher education, and research.