„Validating integrated writing tasks – A mixed-method approach to investigating the construct of summarisation“ (Working title)
Supervised by Claudia Harsch at the University of Bremen.
The PhD project is designed as a validation study within the context of a large-scale digital language test intended for admission purposes in Germany. Taking an argument-based approach to validation, it aims at validating the use and interpretation of a specific integrated writing task – a summarisation task. In particular, to the project investigates the extent to which the performances on the summarisation task permit inferences to the writing competence in the target language use domain. The PhD project uses a mixed-methods research design adopting three different perspectives: (1) a process-oriented perspective, (2) a product-oriented perspective, and (3) an evaluation and scoring perspective.
In a first step, data from eye-tracking and stimulated recall are used to yield insight into the cognitive processes of test takers while working on the integrated writing task. When viewed together, the quantitative and qualitative data provide information about the reading-writing-relations in this specific task type, and thus substantiate the link between the cognitive processes and existing writing and/or text processing models. Data from an additional typing speed test allow to examine possible effects of test-takers’ typing skills on the writing process.
The analysis of the written products focuses on the ability of test-takers to transform the language used in the source material. More specifically, it focuses on the linguistic range and complexity that test-takers manifest when summarising relevant information in their own words, taking their overall language competence into account.
Looking at the evaluation and scoring of the summaries, ratings of the written performances are used to investigate to what extent the integrated writing task measures writing competence. Key issues refer to the reliability of rating integrated writing performances and to the effect of reading competence on task performance.
In a last step, data from the distinct perspectives are linked to each other in order to examine how cognitive processes might have an impact on the written products and their evaluation. Finally, the mixed-methods and multiple-perspective approach provides a rationale and justification for the use of an integrated writing task within a high-stakes assessment context.