TY - GEN
T1 - Bring it on! Challenges encountered while building a comprehensive tutoring system using ReaderBench
AU - Panaite, Marilena
AU - Dascalu, Mihai
AU - Johnson, Amy
AU - Balyan, Renu
AU - Dai, Jianmin
AU - McNamara, Danielle S.
AU - Trausan-Matu, Stefan
N1 - Publisher Copyright: © Springer International Publishing AG, part of Springer Nature 2018.
PY - 2018
Y1 - 2018
N2 - Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self-explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa =.43).
AB - Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self-explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa =.43).
KW - Intelligent tutoring systems
KW - Natural language processing
KW - ReaderBench
KW - Self-explanations
KW - Support vector machines
UR - https://www.scopus.com/pages/publications/85049369683
U2 - 10.1007/978-3-319-93843-1_30
DO - 10.1007/978-3-319-93843-1_30
M3 - Conference contribution
SN - 9783319938424
T3 - Lecture Notes in Computer Science
SP - 409
EP - 419
BT - Artificial Intelligence in Education - 19th International Conference, AIED 2018, Proceedings
A2 - Penstein Rosé, Carolyn
A2 - McLaren, Bruce
A2 - Martinez-Maldonado, Roberto
A2 - Hoppe, H. Ulrich
A2 - Luckin, Rose
A2 - Porayska-Pomsta, Kaska
A2 - du Boulay, Benedict
A2 - Mavrikis, Manolis
PB - Springer Verlag
T2 - 19th International Conference on Artificial Intelligence in Education, AIED 2018
Y2 - 27 June 2018 through 30 June 2018
ER -