In the past year, Packback’s team of Community Managers have moderated more than 350,000 questions and responses and maintained the health of all of the Curiosity Communities on Packback Questions.
How do our moderators manage the post volume?
Through careful manual moderating and coaching around the Packback Community Guidelines, augmented by Packback’s proprietary Community Health Algorithm (that notifies our Community Managers of all posts that may not meet the Community Guidelines).
We moderate the questions and answers in Packback to create an environment that lets curiosity to thrive and allows great discussions to be built.
Our Community Managers are not policemen; their goal is not to punish students for posting something that doesn’t fit the guidelines. They are builders that work tirelessly to craft and protect a safe, supportive environment in every community that allows curiosity to thrive.
Lazy posts, cheating or hurtful language can spoil the trust in a community and make it hard for everyone to be truly open and curious. But great questions lead to more great questions and make the community a safe space to be bold, think big, and make discoveries.
How posts are flagged and moderated?
Our Community Managers receive flags from three sources: their own manual bi-weekly review of every community, Crowdsourced Flags from students and professors flagging posts within their own community, and our Community Health Algorithm which automatically flags posts which may be violating the Community Guidelines.
Just because a post is “flagged” does not mean that it will be removed! But it does mean that it will get reviewed by our moderation team within 24 hours of being “flagged”, guaranteeing that every “questionable” question gets a second look by a human moderator who cares deeply about the health of the community.
Flagging Source: The Community Health Algorithm
The Community Health Algorithm helps multiply the impact and reach of our Community Managers and make it possible for them to effectively review and manage the thousands of questions and responses that are posted on Packback each week. Our Community Health Algorithm is extremely sensitive, and currently has a 44% dismissal rate by our Community Managers, but only misses 2.1% of posts that our Community Managers remove. It is an incredibly powerful tool that lets our team catch potential problems in a community before they even happen!
Our Community Health Algorithm detects posts that may be violating one of the Packback Community guidelines and “auto-flags” them to bring them to our Community Managers’ attention. It “Auto-Flags” based on the following criteria which are modeled off of our Community Guidelines:
It is a statement, not a question: Algorithm “auto-flags” questions lacking question marks, ending in a period or exclamation, lacking a “question starter word”
It is a closed-ended question: Algorithm “auto-flags” questions with “question starters” that indicate closed-endedness, like “What is the definition of”, etc. which don’t allow for open-ended discussion
It contained profanity: Algorithm “auto-flags” posts that contain profane or harmful language)
It is related to class logistics or homework, instead of concepts: Algorithm “auto-flags” posts that contain certain “class-specific” trigger words)
Flagging Source: “Crowdsourced” Community Flagging
The students and professors in each Curiosity Community on Packback Questions know their classes better than our Community Managers ever could. The members of each community support our Community Managers by flagging posts which they know to be incorrect, cheating or totally off-topic from the content of the course. When a professor “flags” a post, our moderators remove it without question.
Our Community Managers’ Process
Our team reviews every “flag” and dismisses “flags” which were too sensitive, edits questions whenever possible (in the case of a question missing a question mark, or a misspelled word), or deletes the posts that are true violations of the Community Guidelines and that would cause a breakdown of quality and academic, curious discussion for other students.
In the case that a question or answer is removed, the student who posted it receives a friendly email explaining why their post was removed with a link to easily re-post. Our team’s goal is never to punish students for asking a question. Our goal is to raise up students’ curiosity, inspire them about the amazing value of great questions and empower them to keep asking more challenging questions!
Pedagogical Basis for the Packback Community Guidelines
Packback’s pedagogy can be broken down into two elements: the Curiosity Curriculum (a carefully curated list of recommendations around structure and implementation into class) and our Community Guidelines (how we moderate communities, detect “featured” questions, and provide coaching to students.
The basis for our moderation guidelines is “Bloom’s Taxonomy of Cognition”. The posts that are encouraged on Packback push students into the levels of Analyzing, Applying, Evaluating, and Creating; Open-Ended questions with more than 1 right answer, that push students to make connections between subjects and topics and apply their learning to real-world scenarios. The posts that are discouraged on Packback are closed-ended questions (questions with only one right answer). These questions comprise the base levels of Bloom’s Taxonomy: Remembering and Understanding.
We detect and moderate based on how students phrase their questions, looking for those “remembering” and “understanding” level questions based on question keywords.
The impact prioritizing Community Health has on curiosity
We have found that the most important factor in HOW a student asks a question is what other questions they were able to see on the main feed at the time that they asked the own question.
The quality in a community, or lack thereof, is compounding.
Communities full of great questions that represent the higher levels of Bloom’s taxonomy, beget more and more thought-provoking questions. While we do allow professors to opt out of our moderating services, communities on our platform which choose to not have our moderators moderate the community tend to quickly fill with questions about specific homework problems.
To create an environment that is conducive to asking open-ended, high-level questions that employ combinatory thinking and analysis, we have found in our research that it is critical to remove questions that are closed-ended, asking for homework help, or are of poor effort or quality. The efforts of our Community Management team have improved the experience that students have on the Packback platform significantly.
We survey before the semester and at the end of the semester using proven methods of measuring a students’ State and Trait curiosity and do micro-check-ins within the app as students post to see how curious they are to receive an answer.
Students that used Packback in 2015-2016 self-reported that Packback inspired their curiosity throughout the semester, self-reported that they often logged in to Packback to read other students posts because they were curious, and self-reported learning new things on Packback that they wouldn’t have asked in class.
Our purpose is to awaken fearless, relentless curiosity in every student, and our Moderation services help guarantee that we can deliver on that purpose for every community and every class.
