Automated assessment and feedback generation should support teachers in the learning process by minimizing the time and effort required, especially for repetitive and simple corrections. The goal of this project is to address writing assignments, particularly essays, in the mid-level writing concerns, such as readability and cohesion, so teachers’ attention can go towards higher-level concerns, like overall argumentation or audience.
About this project
The main goal of this work is to minimize the time spent by teachers assessing and providing feedback for essays. There are widely used tools that assist students for the lower-order concerns when writing, for instance spelling or basic grammar and punctuation, they often do not require an explanation or feedback, they just have to be corrected. On the other hand, higher-order concerns still require a complex understanding of both writing quality elements and the particular assignment, the topic, or the objective. These require interpretation, and the feedback should be detailed and specific. It is the middle ground, where characteristics such as the formality or cohesion of each sentence, can be addressed by current technology. It is beneficial for students to practice and have a few iterations when revising their own written work, without teachers intervening just yet. The revision coach provides several levels of support for students upon request, by first by showing them which sentences have the most opportunity for improvement and in which field, then providing advice on how to revise them, and finally showing an automatically generated example sentence.
Approach & Outcome
The revision coach finds the sentences with most improvement opportunity by revising each sentence and calculating the change in score for the total essay on each of the 4 characteristics selected: formality, cohesion, coherence and readability. The system uses different approaches for revision, such as paraphrasing, text generation and synonyms.
Some paraphrasing techniques include round-trip translation, translating a sentence to German and then back to English; and neural network-based language models with paraphrasing capability, such as Google’s T5[1] and OpenAI’s GPT-2[2]. These language models also have text generation capabilities, instead of revising a given sentence like paraphrasing, they can take the previous and following sentences and generate a completely new one that fits in between them. Finally, synonym revision consists of finding the nouns in the sentence, and replacing them by their synonyms, effectively changing the sentence while keeping its meaning.
The sentences and revision techniques that have the highest positive change in score will be highlighted for students. By revising the sentences with different approaches, the system can also provide different advice to students on how to improve their essay.
[1] Transfer Learning with T5 - https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html
[2] Better Language Models and Their Implications - https://openai.com/blog/better-language-models/
Partner: Feedback Fruits
