Assisted grading with Gradescope

How Gradescope improves our grading workflow.

Federica Zoe Ricci https://twitter.com/ZoeRicci , Mine Dogucu https://minedogucu.com
08-12-2021

With higher course enrollments in data science courses, there is more grading to do and less time to provide meaningful feedback. Also, active learning activities often involve the use of additional formative assessment, like short quizzes, that instructors may be hesitant to introduce due to the need of then devoting time/resources to grade them.

In this blog post, we will present some of the features thanks to which Gradescope , an AI-assisted tool, allowed us to increase efficiency and fairness in grading.

We will cover:

  1. how Gradescope can make grading easier
  2. how it can increase the quality of grading
  3. how it can improve learners’ experience of feedback
  4. additional strengths and limitations of Gradescope

We hope that instructors can still take away tips on grading workflow even if they do not plan on using Gradescope.

1. Easier grading

The ways in which Gradescope has simplified our grading routine essentially hinge on three characteristics:

1.1. AI-assisted and manual question grouping

Perhaps our most loved feature of Gradescope is the possibility of forming and grading answer groups rather than grading answers individually. This feature is currently available only for fixed-length assignments (Create Assignment > Homework / Problem Set > Submission Type: Templated (Fixed Length) or Create Assignment > Exam / Quiz) but we hope that it will soon be usable with all types of assignment.

Provided that a question has been answered by all students in the same space in their submissions (e.g. at the beginning of page 3), Gradescope allows forming answer groups. These are groups of similar answers to which the grader intends to assign the same mark and feedback. To form these groups, Gradescope displays the answers of all students to the same question next to each other and allows the grader to browse and group homogeneous answers. After groups have been formed, the grader can assign specific feedback to the answer group, which is then automatically applied to all answers that were grouped together.

For multiple-choice or short, numeric-answer questions, the grader can be assisted in grouping answers by an AI tool. This is a kind of supervised auto-grading system that uses image recognition to group those answers that are estimated to be identical with high confidence while leaving ungrouped the ones on which there was higher uncertainty. The work of the graders is then greatly simplified, as they will only need to confirm that the grouping done by the AI system is correct and group the (usually few) answers that the autograder left ungrouped.

The video shows an example of how the AI-assisted grouping in Gradescope works. In the example, the AI tool automatically groups 6 answers into 3 groups and leaves 2 answers ungrouped. The grader browses the groups that were formed and confirms that the grouping was done correctly. Then the grader groups the 2 answers that were left ungrouped. Now 8 answers have been divided into 4 groups. The grader only needs to grade the 4 groups, and automatically all answers are given the feedback and mark corresponding to their group.

1.2 Dynamic rubrics

The second aspect that greatly facilitated our work at grading is the way that grades and feedback can be assigned in Gradescope. For each question, the grader can assign a mark by selecting possibly multiple items from a rubric that can be modified as grading proceeds. For each item in the rubric, the grader specifies how many points should be added/subtracted and what feedback should be given:

Screenshot from Gradescope's website showing a rubric with two items selected, the associated points and feedback.

We find that this method is more convenient than having to pre-specify a rubric before seeing students’ actual answers. And, very importantly, the points and feedback assigned to a rubric item can be changed at any time by the grader, and the change will automatically apply to all individual answers/answer groups for which the modified rubric items were selected. This makes it very simple to make improvements or fix mistakes, even after grades have been distributed. For example, the instructor may decide that something for which the grader removed 1 point should actually not have been penalized. Then, instead of having to go through all submissions and fix the grading mistake in each, the grader can simply increase from “-1” to “0” the points assigned to the rubric item involved and re-publish the grades.

1.3 Grading by question

Having collected all students’ submissions of an assignment made of multiple questions, one may often tend to organize their grading “by submission” - that is, going through all questions in one student’s submission and then moving to another student’s submission. Instead of organizing grading by individual student’s submissions, Gradescope arranges together answers to the same questions across all students’ submissions:

Screenshot from Gradescope's website showing their Grade Submissions page, where you can select what question you want to grade.

Such arrangement predisposes the grader towards working by grading all answers to the same question and then moving on to a different question. This workflow lends itself towards a more effective way of:

While a grader and multiple graders can obviously find ways to organize their work also when grading by submission, focusing on grading all answers to the same question before moving on to a different question and/or assigning all answers to a question to the same grader can help to make grading both faster and more consistent.

2. Higher-quality grading

Besides making grading simpler, the grading system implemented in Gradescope can also have positive outcomes on the quality and fairness of grading:

Screenshot from Gradescope's website showing several examples of personalized feedback, as well as a list of previously assigned feedback among which to possibly select.

3. Better feedback experience

Several features of Gradescope also have the potential to improve students’ experience of feedback.

As previously discussed, answer groups and dynamic rubrics mean that an extensive feedback needs only to be written once to then be received by all students with similar answers. Moreover, students can be encouraged and motivated by receiving personalized feedback, that the graders may have more time to write if they use a grading tool that simplifies and speeds up other parts of the grading.

Another important way in which Gradescope can ameliorate students’ relationship with grading concerns is regrade requests. If enabled, after grades have been published, students can easily ask for a regrade of any of their answers, supplying an explanation of the motivation for their request. Simplifying this process can increase students’ perception of grading fairness, and can make them more likely to ask for explanations when they have not understood the reason for their score, thus potentially improving their satisfaction with the instruction.

Screenshot from Gradescope's website showing a regrade request being handled.

Finally, if the process of grading is expedited, students can receive feedback more quickly. This is also likely to improve their satisfaction, and especially it provides them with better opportunities to learn from the feedback received on formative assessments.

4. Other strengths and limitations

We conclude this post with an overview of other worth-mentioning strengths and limitations of Gradescope.

Strengths

Limitations


Disclaimer: the materials used for illustration only includes activities of fictitious students’.