The issues of algorithmic fairness and bias have recently featured prominently in many publications highlighting thefact that training the algorithms for maximum performance may often result in predictions that are biased against various groups. Educational applications based on NLP and speech processing technologies often combine multiple complex machine learning algorithms and are thus vulnerable to the same sources of bias as other AI systems. Yet such systems when used in assessment can have high impact on people's lives. In this talk I will discuss the issues of fairness and bias in educational applications using as an example the case of automated scoring of non-native English spontaneous speech.
Anastassia Loukina is a research scientist in the Research and Development division at Educational Testing Service (ETS) in Princeton NJ. ETS develops, administers and scores more than 50 million tests annually in more than 180 countries at more than 9,000 locations worldwide. The NLP & Speech group at ETS develops technology for automated scoring of openended items, classroom support tools that teachers, and tools that can aid in the test development process. Since joining ETS, Anastassia has led the research to improve the validity, reliability and fairness of speech-based educational application and made key contribution to successful launch of several automated scorings ystem. She published more than 40 papers and book chapters, holds several patents and frequently attends international conferences and workshops.