TY - JOUR
T1 - An investigation of integrative and independent listening test tasks in a computerised academic English test
AU - Wei, Wei
AU - Zheng, Ying
N1 - Publisher Copyright:
© 2017 Informa UK Limited, trading as Taylor & Francis Group.
PY - 2017/11/17
Y1 - 2017/11/17
N2 - This research provided a comprehensive evaluation and validation of the listening section of a newly introduced computerised test, Pearson Test of English Academic (PTE Academic). PTE Academic contains 11 item types assessing academic listening skills either alone or in combination with other skills. First, task analysis helped identify skills important for listening comprehension in academic settings. Aspects analysed included the purpose of assessment tasks, skills/constructs assessed, and task stimuli employed in PTE Academic. The findings indicated that modern technologies enabled PTE Academic, a computer-based test, to assess students’ academic listening abilities in real time using the integration of multi-modal sources. The statistical validation consisted of two stages. Exploratory factor analysis was performed first with a sample of over 5000 students who took PTE Academic, to examine the underlying listening constructs as measured by the scores on the 11 item types item on different listening skills; these scores were subjected to Rasch analysis using CONQUEST. Second, the difficulties of the item types were estimated and the effectiveness of these item types was evaluated by calculating the information function by item type. The study has implications for test developers and test users regarding the interpretation of student performance on listening assessments.
AB - This research provided a comprehensive evaluation and validation of the listening section of a newly introduced computerised test, Pearson Test of English Academic (PTE Academic). PTE Academic contains 11 item types assessing academic listening skills either alone or in combination with other skills. First, task analysis helped identify skills important for listening comprehension in academic settings. Aspects analysed included the purpose of assessment tasks, skills/constructs assessed, and task stimuli employed in PTE Academic. The findings indicated that modern technologies enabled PTE Academic, a computer-based test, to assess students’ academic listening abilities in real time using the integration of multi-modal sources. The statistical validation consisted of two stages. Exploratory factor analysis was performed first with a sample of over 5000 students who took PTE Academic, to examine the underlying listening constructs as measured by the scores on the 11 item types item on different listening skills; these scores were subjected to Rasch analysis using CONQUEST. Second, the difficulties of the item types were estimated and the effectiveness of these item types was evaluated by calculating the information function by item type. The study has implications for test developers and test users regarding the interpretation of student performance on listening assessments.
KW - Computerised test
KW - academic listening
KW - construct validity
KW - integrative and independent tasks
UR - http://www.scopus.com/inward/record.url?scp=85029418186&partnerID=8YFLogxK
U2 - 10.1080/09588221.2017.1373131
DO - 10.1080/09588221.2017.1373131
M3 - Article
AN - SCOPUS:85029418186
SN - 0958-8221
VL - 30
SP - 864
EP - 883
JO - Computer Assisted Language Learning
JF - Computer Assisted Language Learning
IS - 8
ER -