Content moderation is critical to the success of the digital world: ensuring online communities are safe, inclusive and achieve growth goals. However, moderating content is a complex task. This blog shares 6 best practices all online communities should adhere to in order to safeguard users and revenue.
Why is content moderation important?
In the wake of the 2020 presidential election, social platform Parler enjoyed explosive growth, gaining 4 million followers in the first two weeks. Parler was supported by conservatives as an alternative to mainstream social platforms like Facebook or Twitter, which were accused of liberal bias. Parler was touted as the ‘free speech alternative’ platform, with a permissive content moderation policy.
Instead of traditional content moderation, Parler relies on user complaints, which are then reviewed by platform moderators. However, the weak content moderation system employed by Parler made it a target for pornography purveyors, with tens of thousands of pornographic images and videos posted to the feeds of regular users over just a few days1. This disrupted user experience, and slowed the momentum of platform growth.
When users, willingly or unknowingly, create content that violate guidelines, platforms come under fire from media and government regulators. Having illegal, disturbing, or offensive content on your site erodes the user experience, increasing churn and impacting revenues. Effective content moderation also prevents damage to brand reputation and messaging.
What types of content moderation are available?
Many platforms hire people (either directly as full time employees or through large outsourcing firms) to review user-generated content. A person can identify banned behaviors and interpret the context, helping to execute the correct response to a situation.
The drawbacks to using a human content moderation team is that it is slower than an automated solution, unable to review and respond to UGC in real time. It can also be inconsistent, as moderators are expected to have the ability to manage a high volume of content and apply guidelines consistently.
One of the biggest drawbacks though, is that reviewing violent, illegal, and otherwise disturbing content for hours on end is terrible for the mental health of content moderators. Many develop PTSD, as well as heightened levels of stress, anxiety and depression as the result of their jobs. Hiring people to moderate content is also inefficient, as it is difficult to keep up with the enormous amount of user-generated content that is posted daily; and it can be expensive.
Keyword / RegEx lists.
Many platforms choose to use a filtering system that references lists of banned words, expressions, IP addresses or emails. Content matching filtering system is removed or flagged for moderation. This may work as support for human moderators, lightening their workload to a point, but they are not an effective solution for several reasons:
- Difficult to maintain. The lists must be updated, reviewed and managed manually.
- Easy to circumvent. Simply replace the letter ‘O’ with a zero, and the filter will ignore.
- Unable to read context. Banned behaviors can appear without using banned words or expressions – because an action like cyberbullying or sexual harassment is dependent on the context of the action as much – or more – than the content itself.
Setting up a system for users to report toxic behaviors they encounter on your platform is like crowdsourcing content moderation: it gives your platform a resource for identifying banned behaviors. At the same time, it provides users with a sense of agency – allowing them to take action when they have viewed something negative, and hopefully diluting the effects of the experience.
Artificial Intelligence / Machine Learning.
The latest in content moderation involves artificial intelligence that can read contextual cues, to accurately identify and respond to banned behaviors in real time, across languages. This solution improves on filters because it is more difficult to circumvent and able to read context. It improves on human moderators because it is fast, efficient, and does not cause the stress of repeated viewings on employees.
Even more importantly, it improves on itself: as data is gathered and fed back into the algorithm, it becomes better at identifying and responding to toxic behaviors.
Best practices for content moderation
To build a content moderation system on your platform, or improve the content moderation process you have in place, consider the following:
1. Find the method or mix that matches your needs.
Depending on the volume of UGC on your platform and the strictness of your community guidelines, a different content moderation solution may be required. Perhaps a keyword filter in combination with a content moderation team, or user reporting that is then reviewed by moderators will work best for your platform.
2. Create and publish community guidelines.
Expectations should be clear, comprehensive, and accessible to all users. The more clearly you describe the actions that are encouraged and discouraged on your platform – with examples, if possible – the better everyone will understand behavioral expectations.
An example of great community guidelines can be found at GitHub, a community for software developers. Their guidelines include a statement of purpose:
“The primary purpose of the GitHub community is to collaborate on software projects. We want people to work better together. Although we maintain the site, this is a community we build together, and we need your help to make it the best it can be.”
It also includes examples of good behaviors (mutual respect, open-mindedness, empathetic communication), bad behaviors (violence, discrimination, bullying), and the process for reporting banned behaviors.
3. Cover all languages.
Make sure that community guidelines and other forms of content moderation cover the languages that are used on your platform. It is difficult to communicate consequences, appeals, and other actions outside of the home language, so for clarity and transparency, make sure that multiple languages are supported.
4. Incentivize positive behavior too.
The community guidelines should include examples of negative behavior but there should be examples of positive behavior too. And just as toxic behaviors have negative consequences, look for opportunities to reward positive interactions. Perhaps the platform could distribute a leveling system of badges for participation or longevity, or number of posts by a specific user.
5. Consider all types of content.
UGC isn’t just written comments or other content, it can also include video, live chat, images, etc. Content moderation must be planned to create a safe, inclusive user experience across all interactions – no matter what the type of content.
6. Safety is everyone's responsibility.
Content moderation cannot occur in a silo - it requires buy-in and effort from key stakeholders across the business. Trust & Safety teams should have support and input from product, marketing, advertising, and the executive suite, to create successful, comprehensive moderation plans.
7. Build transparency into the system.
A content moderation plan must have rules and guidelines and consequences for exceeding those boundaries. And the platform will be required to ensure that those consequences are applied fairly and equitably, without undue influence on any group of marginalized individuals. Creating a content moderation process with a foundation of transparency that includes regular review and reporting is critical to the sustainability of the moderation process.
Many leading platforms publish regular transparency reports: disclosing the actions that are taken with regard to the content on their platforms. Twitter, for example, hosts a web-based transparency center that includes current and historical reports with data on rules enforcement, government requests for information, and platform manipulation. Google publishes transparency reports as well, covering security and privacy, content removal, and political advertising reports.
Because every platform and every audience is different, they will each have unique guidelines and specifications for acceptable behavior by their users. A one-size-fits-all solution can’t address the complexities of content moderation – even automated solutions must be customizable to the needs of each platform.
When you are looking to implement or upgrade content moderation for your digital environment, consider an automated solution that works in real time, across multiple languages, and can evaluate the context of content to ensure accurate, effective, customizable moderation.
Spectrum Labs was created to answer these questions, meet these challenges, and help platforms create and enforce appropriate UGC. Our Contextual AI solution evaluates user-generated content in real time, over multiple languages, deciphering context and adapting to a changing environment. Accurate, automated and reliable: Contextual AI relieves employees of the burden of content moderation, allowing you to channel those resources to higher-level, strategic objectives. To learn more about Spectrum Labs, contact our team today!