One thing that makes marketplace platforms so attractive to people are social features like comments, chats, etc. Unfortunately, these social features on e-commerce platforms are often exploited by spammers, creating a bad experience for users.
While spam can sometimes be lumped into a general bucket of “harmful behaviors” to squash on a platform, it is unique for a few reasons. First, spammers have an underlying intent to defraud people and make money. Second, those who create spam tend to come from organized crime rings that are incentivized to expand their reach as much as possible.
Research shows that spam affiliate programs can generate as much as $12-92 million over the course of their operations.
Because there is a financial drive behind spam, there is also an adversarial nature to spammer behaviors - they will do whatever they can to get around any sort of detection or content moderation practices they perceive to be in place. This means that spammer behavior is going to be ever-changing, always adapting to counter measures. Spam detection solutions must be able to learn faster than the spammers, otherwise this becomes a never-ending, expensive game of whack-a-mole.
An Anti-Spam Framework
Since spam behavior is ever changing in nature it’s important that any detection methods used to identify spam look across multiple features to truly capture context and see the full picture. This means looking at things like the content itself, where the content is being posted, and attributes related to the user. Let’s break that down:
1. Look at Content
Within the actual piece of user generated content in question you’ll want to look to see if there is clear intent to move someone off the platform. This is often in the form of providing an alternative contact method such as an email or phone number. Since spammers are trying to get around filtering they’ll probably provide this contact info in a non-standard way. I.e. `name@gmail.com` becomes `name @ gma1l . com`. You can also look at the context of the request — for example, is it someone’s nieces birthday and she has to have this item right away but she can only communicate via SMS?
2. Where the Content is Posted
If your platform is designed for selling items, there are some item categories that are more prone to spam than others. If your platform is more of a matchmaking platform, things with lower barriers to entry have higher risk — i.e. applications to an entry level job vs. one that requires years of experience.
3. Who Posted the Content
You’ll want to look at attributes such as when did the user sign up for the platform? Was it less than an hour ago and they’re already posting a high volume of content? What kind of behavior have they participated in previously? How wide have they set their preferences? How seriously are they taking their account — have they verified their email address? Enabled 2fa? Etc. Is the user coming from an IP address that is known to be problematic?
There are some detection methods that just focus on the IP address, others that focus on just volume and frequency, and even others that just detect emails. These leave a lot on the table and have a high risk of false positives. It’s important to look at all of the different factors to make the most accurate determinations which is possible through contextual AI that ingests metadata along with raw content.
Related Reading: Why Use AI for Content Moderation
Spam is Everyone’s Problem
However, detection is just one piece of the puzzle. It’s also important to design the product and user experience to be in line with your anti-spam goals. In order to do this you need cross-team alignment on what those exact goals are, especially as they can be in competition with user growth goals. We’ll get into suggested success criteria in a bit. But first, I want to dive into what are some of the product specifics you can think through to encourage positive community engagement while minimizing spam.
First up, the sign up process. What sort of information does a user need to provide in order to sign up for your platform? Are there any verification steps? While each additional step you introduce can slow down the entry process a bit, it also increases the quality of users on your platform. And introduces friction for spammers.
Now that a user has signed up for your platform, what sort of behavior is reasonable for them to be allowed to do? It’s best to do an analysis on what “good” community members typically do within the first 24 hours or so and set limits so bad actors don’t abuse the social components. For example, do you need to allow more than 10-20 comments/applications/matches/etc. on that first day? Think through the lens of both good actors and potential spammers in order to understand and mitigate risk.
You Caught a Spammer! Now What?
Next up, what happens when you do detect a spammer? How do you respond in a way that removes the harmful behavior and protects your community without feeding spammers information about how your detection works so they can start coming up with new tactics to get around it? What you don’t want to do is immediately ban the user so they have that real time feedback. Instead, here are some creative ideas we’ve seen work well in practice:
- Spammer Island - mark users as spammers and only let them communicate with each other.
- State of “Purgatory” - create a holding area so spammers think their messages have posted but no one else can see them.
Talking about understanding and mitigating risk, it’s also important to think through the user experience effects if you get this detection wrong and accidentally punish a good community member who is not a spammer. You don’t want to accidentally put a susceptible user in spammer island, and you don’t want to kick out solid community members who are driving revenue. Make sure you have a review process in place that makes sense for your use case. If automating everything, perhaps you pull a random sample to review monthly to understand how you’re performing. Otherwise, choose what kinds of cases should go to a human to review ahead of issuing an automated punishment.
The other thing to consider with spammers is that they keep coming back because it ultimately works; some people do fall for spammer tactics and end up getting defrauded. What sort of education opportunities are there to help your users understand how to safely interact with your platform and not fall for these harmful tactics?
Related Reading: Reducing Fraud and Minimizing Risk at Upwork and eBay
Measure Your Efforts
Okay, now I promised we’d touch on success metrics so here we go! In a world where your product is designed to bring people together and give them UGC channels for communication there is also going to be an element of spam. So how do you minimize the game of whack-a-mole and define success against your goals? Often, detection performance is measured through precision and recall, which is one metric, but it’s also important to see how that detection is adding value against the actual goals.
Other things to look at include:
Impact
How many people are being exposed to spam on your platform? (Remember - this can be affected by how you’re responding to spammers)
Time to Remove
This is a bit of a game of whack-a-mole so how fast can you adjust the hammer to the new tactics and hit that mole?
Quality of Content
What percentage of the overall UGC is spam vs. not spam?
Quality of Users
What percentage of overall users, or new users, are spammers vs. revenue producing, quality community members? (This is a great way to combine user growth and spam goals.)
Spectrum Labs configurable AI spam solution detects the most disruptive problems first and helps you take real-time action to redact messages and shadow-ban accounts.
Sources
1https://ppcprotect.com/resources/CharacterizingLargeScaleClickFraudinZeroAccess.pdf
