Designing an AI to grade for curiosity isn’t easy; psychologists have been trying to boil curiosity down to a definable metric for generations. At Packback, our research team has been working for the past year to develop an algorithm that measures the quality of the posts on the Packback platform. While we’re not quite ready to announce that the Packback Curiosity Score is the definitive metric to objectively measure curiosity (that would be BIG news!). We are very excited about the accuracy with which it can score effort, presentation, and credibility of every question and response on Packback!
What is a Packback Curiosity Score?
Every post on Packback earns a unique point score. Those post level points get totaled up in conjunction with points for specific behaviors that are correlated with highly active, highly curious, highly driven members of the community to calculate a per-community score for each Packback member. On a higher level, these points are summed across all of a user’s communities to calculate a user’s Curiosity Score and rank across the entire 15,000+ active users on Packback.
The algorithm used to calculate these scores was created with deep testing and refinement against the “human” grading that our R&D team members assigned to thousands of posts. From this data, we distilled the commonalities amongst high-quality posts and the behaviors of top users, we were able to develop a concise algorithm that grades every piece of curious content on the Packback platform within a tight degree of accuracy to how a human grader would score each post and rank each user.
How are Packback Curiosity Scores Calculated?
The Packback Curiosity Scoring Algorithm currently scores individual posts based on 3 main criteria; Presentation, Credibility, and Effort.
In our research, we discovered that the legibility and presentation of a post were highly correlated to the quality of posts and the value that they brought to the community’s readers. To grade for presentation, the Packback Curiosity Scoring Algorithm considers legibility, formatting, the inclusion of images, videos and supplemental materials within questions and responses that make the post easier, more interesting and more insightful to read.
The inclusion of materials that provide evidence to support the argument in an individual’s response or provide context to a question elevates a posts’ quality and unique value in the community. To grade for credibility, our algorithm studies whether or not posts contain relevant, reliable sources to defend ideas and arguments. The algorithm also considers the behaviors that correlate with credibility; including frequency of posting, depth of post and time of post.
The Packback Curiosity Scoring Algorithm studies the content of the questions and responses on the platform to score the effort that the writer put into each individual post. The effort is one of the strongest indicators of a quality response on Packback; because most of the questions on Packback are open-ended and do not have a single right answer, a well documented, well articulated and well-supported response is far more valuable than one that it simply provides a cut-and-dry answer. The effort is graded by studying whether the writer simply gave the most straightforward answer to the question, or if they added new and unique value to the community and the readers of their post by explaining their argument fully, sharing a story, including videos or images to illustrate the point, and more.
About Packback Curiosity Scores
While the Packback Curiosity Scoring Algorithm is currently used primarily to measure the quality of posts within the Packback platform, the long-term goal of this algorithm is to continue to push the limits of what Packback members can do in their communities.
The Packback Curiosity Score drives friendly competition between members, helps to make professors’ lives a little easier when grading, and gives students feedback about their curious questions and insightful responses. It is a living algorithm that will continue to be refined, researched and improved upon as our members continue to redefine what it means to be a curious mind and show the world what students can accomplish when they put their minds together.
Learn more about the Packback Curiosity Scoring System
Want to see a live Packback community and learn how you can increase student engagement and critical thinking in your course?