White Paper: Impact of Online Discussion Platform and Pedagogy on Student Outcomes
This study explores the impact of the use of Packback on grade outcomes, student engagement, and discussion quality. 10 institutions participated in an IRB-approved comparative study of the use of Packback versus the Learning Management System discussion.
Finding 1: Increased Student Engagement
More Posts (Median)
Students in the Packback treatment group had a 1.96X higher median post rate, despite both the treatment and control group having equivalent quantitative posting requirements.
In Spring 2020 (during the COVID-19 pandemic), the Packback treatment group, there were 1.83X as many ‘Weekly Active Users’ than in the control group. Weekly Active Users are defined by a student logging in and submitting at least one post.
Finding 2: Increased Discussion Rigor and Quality
More Posts with a Citation
Posts in the Packback treatment group were over 2X more likely to contain a source citation than posts in the treatment group.
📝 This finding was also independently replicated by University of North Texas in a 2019 Study, “How AI is Shaping Online Discussion Quality.”
Higher Median Wordcount
The posts from the Packback group are centered at a higher median word count; in the Spring 2020 term, median wordcount in the treatment group was 5.88 percentage points above the control group.
Finding 3: Improved Final Grade Outcomes
Compared to both control group students and historical data, students using Packback were significantly more likely to receive A’s in the course.
Compared to both control group students and historical data, students using Packback were significantly less likely to receive F’s in the course. Compared to control data, students on Packback were also significantly less likely to receive D’s in the course.
Finding 4: Positive Student and Faculty Satisfaction
of Faculty Agree
84% of faculty participating in this study agreed or strongly agreed with the statement, “I find my students’ questions and conversations more dynamic and genuine this term (on Packback).“
Positive Student Sentiment
83% of students in treatment groups responded with positive sentiment to the statement, “Based on the discussion you experienced did you find Packback more effective than traditional online discussion boards?“
Summary: In Fall 2019 and Spring 2020, 10 institutions participated in a research study with Packback, an inquiry-based discussion platform powered by AI. This paper details the outcomes of the study in order to deepen our understanding of online discussion and how to maximize it ultimately improving success rates along with the faculty and student experience.