White Paper: Impact of Online Discussion Platform and Pedagogy on Student Outcomes

This study explores the impact of the use of Packback on grade outcomes, student engagement, and discussion quality. 10 institutions participated in an IRB-approved* comparative study of the use of Packback versus the Learning Management System discussion.

Download the white paper

*Study reviewed and approved by each participating institution’s IRB office

Top half of image: Cover Image of "Impact of Online Discussio Platform and Pedagogy on Student Outcomes." Bottom half of image: Logos of colleges who participated in the study

Finding 1: Increased Student Engagement

Figure 1- Table of results comparing number of discussion posts per student in Fall 2019 and Spring 2020

1.96X

More Posts (Median)

Students in the Packback treatment group had a 1.96X higher median post rate, despite both the treatment and control group having equivalent quantitative posting requirements.

1.83X

More Active

In Spring 2020 (during the COVID-19 pandemic), the Packback treatment group, there were 1.83X as many ‘Weekly Active Users’ than in the control group. Weekly Active Users are defined by a student logging in and submitting at least one post.

Finding 2: Increased Discussion Rigor and Quality

Figure 2- Bar graph comparing percent of student discussion posts with linked sources in treatment versus control group

2X

More Posts with a Citation

Posts in the Packback treatment group were over 2X more likely to contain a source citation than posts in the treatment group.

? This finding was also independently replicated by University of North Texas in a 2019 Study, “How AI is Shaping Online Discussion Quality.”

5.88 %

Higher Median Wordcount

The posts from the Packback group are centered at a higher median word count; in the Spring 2020 term, median wordcount in the treatment group was 5.88 percentage points above the control group.

Finding 3: Improved Final Grade Outcomes

Figure 3- Bar graph comparing student grades between treatment and control groups.

Significantly More

A’s

Compared to both control group students and historical data, students using Packback were significantly more likely to receive A’s in the course.

Significantly Fewer

F’s

Compared to both control group students and historical data, students using Packback were significantly less likely to receive F’s in the course. Compared to control data, students on Packback were also significantly less likely to receive D’s in the course.

Finding 4: Positive Student and Faculty Satisfaction

Figure 4- Two bar graphs, each showing percentage of professors who agree with positive sentiments about using Packback. Indicators range from "strongly disagree" to "strongly agree"

84%

of Faculty Agree

84% of faculty participating in this study agreed or strongly agreed with the statement, “I find my students’ questions and conversations more dynamic and genuine this term (on Packback).“

83%

Positive Student Sentiment

83% of students in treatment groups responded with positive sentiment to the statement, “Based on the discussion you experienced did you find Packback more effective than traditional online discussion boards?“

Cover image of "Impact of Online Discussion Platform and Pedagogy on Student Outcomes"

Summary: In Fall 2019 and Spring 2020, 10 institutions participated in a research study with Packback, an inquiry-based discussion platform powered by AI. This paper details the outcomes of the study in order to deepen our understanding of online discussion and how to maximize it ultimately improving success rates along with the faculty and student experience.

Get a copy of the paper