Social media platforms and the algorithms that power them shape our culture and identities, influence our perspectives on the world, and control the media, news, and advertising that we see daily. The need for new content moderation policies and technologies has rapidly surpassed the current capabilities of many of the world’s most popular platforms.
What if content moderation fails to improve, and who decides that it has? Currently, the debate over content moderation and how to enhance trust and safety is happening in the form of outrage without any consistent data points to back up opinions.
Platforms must take action now, step up, and script out what they’re going to do without being told what is right by outside forces. Innovation is critical during this stage to avoid government regulation that could hinder the growth of platforms. In other words, platforms must move beyond simple profanity filters and consider more advanced technologies to support the health and safety of communities.
Platforms also risk the loss of user trust, giving way for the next generation of platforms that will be built with robust moderation policies and technologies. They will bake in privacy, safety, and DEI (diversity, equity, inclusion) as part of their DNA.
To brace for the future, platforms must observe what is happening in the world right now. Explore recent trust and safety trends that are currently shaping the social media platform landscape:
1. A growing threat to user experience: Sophisticated spam farms and machine-generated content
Machine-generated content is growing more sophisticated1 and more common, creating new challenges for platforms. Any nefarious entity that can get past a platform’s account verification system can begin to spew content into the platform and hurt the user experience. Take spam SMS messages, for example. Spammers use adversarial machine learning (AML) to target the security mechanisms that determine if an SMS message is spam or not, allowing spam to bypass detectors.2
An overwhelming amount of ongoing new threats require an innovative and holistic approach. Platforms require a solution to moderate all content, regardless of where it is coming from; a legitimate user, a spam bot, or AI. The content source doesn’t matter; if it hurts the user experience, it needs to go.
2. The legal battle over content moderation
On March 8, Twitter filed a lawsuit against Texas Attorney General Ken Paxton for investigating its moderation practices. The platform says it was exercising its First Amendment rights by refusing to publish Trump’s speech and that Paxton opened the investigation to “intimidate, harass, and target Twitter” in retaliation for banning former President Donald Trump.3
According to the suit, “Public disclosure of all Twitter’s internal content moderation procedures would, among other things, provide a roadmap for bad-faith actors to design their content to carefully evade Twitter’s scrutiny, undermining the company’s ability to remove content that negatively affects the security and integrity of the platform and the health of the conversation on the platform.”4
Platforms should disclose information via an exhaustive transparency report that gives visibility to policies and methods without providing ammunition to bad actors. These reports inform the public of a platform’s moderation policies and might help curb new investigations. Facebook5, Twitter6, and LinkedIn7 publish regular reports that shed light on how they enforce policies, respond to data requests, protect intellectual property, handle copyright and trademark notices, and much more.
Related Reading: 7 Best Practices for Content Moderation
3. Health misinformation proliferates
False claims about COVID-19 vaccines are spreading throughout Facebook and its subsidiary, Instagram. Clever influencers, “wellness coaches,” and other bad actors employ tactics to evade out-of-date algorithms and profanity filters. This never-ending game of whack-a-mole allows new hashtags like #healthsovereignty and #wearethecontrol to be created on the fly to spread misinformation under the radar of content moderation tools.
Combined with human moderators who monitor trends and observe bad actors, platforms require sophisticated moderation tools that can empower platforms to fight changing evasion tactics in a timely fashion. A platform must also assess its recommendation algorithms to ensure that it isn’t spreading misinformation amongst its own users.
Trust and Safety teams should periodically evaluate the tools and solutions to support community guideline enforcement with transparency and consistency. Whether you are looking to safeguard your audiences, increase brand loyalty and user engagement, or maximize your moderators’ productivity, Spectrum Labs can help make your community a better place. If you’d like to learn more, download the Contextual AI Solution Guide from Spectrum Labs.
