Linking an English Proficiency Test to the CEFR: Setting Valid Cut Scores
Main Article Content
Abstract
The Common European Framework of Reference for Languages (CEFR) is widely used as a reference in EFL English education, and therefore linking English proficiency tests to the CEFR is imperative if test scores are interpreted according to the CEFR levels. This study aimed to align a FRELE-TH-based test, an English proficiency test developed by Chiang Mai Rajabhat University in Thailand, with the CEFR using the Yes/No Angoff method to derive cut scores. The participants were nine university English lecturers purposively selected as judges. These judges made three rounds of consideration regarding the possibility that a borderline test-taker of each CEFR level would correctly answer the test questions. Their judgments, 1 for ‘Yes’ and 0 for ‘No’, were then calculated for the cut scores for A1, A2, B1, B2 and C1 which were 22, 36, 57, 80 and 105, respectively. The test scores can now be interpreted in relation to the CEFR, and this meaningful interpretation is useful for further enhancement of students’ English abilities. For further study, triangulation of data using a different but appropriate standard setting method should be undertaken to increase validity of the derived cut scores.
Article Details
References
Athiworakun, C., Vathanalaoha, K., Thongprayoon, T., Rajprasit, K., & Yaemtui, W. (2018). SWU-SET as a CEFR standardized English test. Journal of Language Teaching and Research, 9(2), 261-267. doi: http://dx.doi.org/10.17507/jltr0902.06
Cizek, G.J. (2012). An introduction to contemporary standard setting: Concepts, characteristics, and contexts. In G.J. Cizek, (Ed.). Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 3-14). NY: Routledge.
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment—Structured overview of all CEFR scales. Retrieved from https://rm.coe.int/168045b15e
Council of Europe. (2020). Common European Framework of Reference for Languages: The CEFR Levels. Retrieved from https://www.coe.int/en/web/common-european-framework-reference-languages/level-descriptions
Foley, J.A. (2019). Adapting CEFR for English Language Education in ASEAN, Japan and China. The New English Teacher, 13(2), 101-117. Retrieved from http://www.assumptionjournal.au.edu/index.php/newEnglishTeacher/article/view/3879
Fulcher, G. (2010). Practical language testing. London: Hodder Education.
Hiranburana, K. (2020). FRELE-TH: Springboard for holistic English educational reform. LEARN Journal, 13(1), 62-75.
Hiranburana, K. Subphadoongchone, P, Tangkiengsirisin, S., Phoochaeoensil, S., Gainey, J., Thogsongsri, J., Sumonsriworakun, P., Somphong, M., Sappapan, P. & Taylor, P. (2018). Framework of reference for English language education in Thailand—(FRELE-TH) based on the CEFR: Revisited in the English educational reform. Pasaa Paritat, 33, 51-91. Retrieved from http://www.culi.chula.ac.th/publicationsonline/files/article2/vP11nmOMYxMon45327.pdf
Hoge, R.D. & Coladarci, T. (1989). Teacher-based judgments of academic achievement: A review of literature. Review of Educational Research, 59(3), 297-313. Retrieved from http://www.jstor.org/stable/1170184
Impara, J. C. & Plake, B. S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34(4), 353-366. Retrieved from https://onlinelibrary.wiley.com/doi/epdf/10.1111/j.1745-3984.1997.tb00523.x
Kaftandjieva, F. (2010). Methods for setting cut scores in criterion-referenced achievement tests: A comparative analysis of six recent methods with an application to tests of reading in EFL. Arnhem, The Netherlands: CITO. Retrieved from https://www.ealta.eu.org/documents/resources/FK_second_doctorate.pdf
Livingston, S. A. & Zieky, J. M. (1982). Passing scores: A manual for setting standards of performanceon educational and occupational tests. Princeton, NJ: Educational Service. Retrieved from https://www.ets.org.Media/Research/pdf/passing_scores.pdf
MacCann, R. G. & Gordon, S. (2004). Estimating the Standard Error of the Judging in a modified-Angoff Standards Setting Procedure. Practical Assessment, Research, and Evaluation, 9, 1-9. https://doi.org/10.7275/n78q-6g60
Sireci, S.G., Robin, F. & Potelis, T. (1999). Using cluster analysis to facilitate standard setting. Applied Measurement in Education, 12(3), 301-325, DOI: 10.1207/S15324818AME1203_5
Tannenbaum, R. J., & Baron, P. A. (2011). Mapping TOEFL ITP scores onto the Common European Framework of Reference. Princeton, NJ: Educational Testing Service. Retrieved from http://www.ets.org/Media/Research/pdf/RM-11-133.pdf
Tannenbaum, R. J., & Baron, P. A. (2015). Mapping scores from the TOEFL Junior® Comprehensive test onto the Common European Framework of Reference (CEFR). Princeton, NJ: Educational Testing Service. Retrieved from http://www.ets.org/Media/Research/pdf/RM-15-13.pdf
Tannenbaum, R. J. & Wylie, E. C. (2008). Linking English-language test scores onto the Common European Framework of Reference: An application of standard-setting methodology. Princeton. NJ: ETS. Retrieved from https://www.ets.org/RR-08-34.pdf
Tannenbaum, R. J. & Wylie, E. C. (2019). Mapping the TOEIC® Tests on the Common European Framework of Reference. Retrieved from https://www.ets.org/s/toeic/pdf-cefr-flyer.pdf
Wudthayagorn, J. (2018). Mapping the CU-TEP to the Common European Framework of Reference (CEFR). LEARN Journal, 11(2), 163-180. Retrieved from http://www.so04.tci-thaijo.org/index.php/LEARN/article/view/161641