The Brief: Reducing fraud and minimizing risk at Upwork and eBay

We talk about trust and safety at peer-to-peer marketplaces with Jeff Chen. He is the former Vice President of Trust & Safety at Upwork, the world’s largest marketplace for online work. Prior to that, Jeff was Director of Managed Marketplaces at peer-to-peer marketplace juggernaut eBay.

Jeff is an expert in improving the outcomes of company-wide trust and safety initiatives. He specializes in online marketplace growth and managing risk. Throughout his time at Upwork and eBay, he was active in reducing fraud, risk, account takeovers, spam, and scams.

Featuring:
- Jeff Chen, Former Trust & Safety Executive at Upwork & eBay (LinkedIn)
- Hosted by: Meredith Reed

Watch the live video recording of this episode on YouTube.

Full Transcript:

Meredith Reed  3:20  

Hello, and welcome to this episode of The Brief. I'm your host, Meredith Reed. I'm here today with Jeff Chen. He is the former VP of Trust and Safety at Upwork, the world's largest marketplace for online work. Prior to that, Jeff was Director of Managed Marketplaces at eBay, the O.G. of peer-to-peer marketplaces. 

Jeff specializes in online marketplace growth and managing risk. He is an expert at improving trust and safety outcomes and reducing fraud, risk, account takeovers, spam, and scams. Welcome, Jeff.

Jeff Chen  4:02  

Thanks. Thanks for having me.

Meredith Reed  4:04  

We're glad to have you. So let's start off by talking about the peer-to-peer marketplaces that you've worked for. You've worked for two of the biggest ones on earth: eBay and Upwork. 

Anytime you connect to strangers online for some type of transaction, there are going to be risks involved. Can you tell us about some of the most prevalent trust and safety issues you've seen at peer-to-peer marketplaces like eBay and Upwork?

Jeff Chen  4:37  

Firstly, I think if we step back and look at the origination of online marketplaces, they really started as an intermediary. So they would connect two parties who wanted something and then they would kind of back away. And that worked for a while, because [...] “people are basically good” was eBay's mantra, and that was how the company was founded. And for the most part, that is the case. However, when things happen poorly, they start holding the platform accountable. 

And so if people weren't getting their items [from] eBay or [their item] wasn't arriving as described, they would blame eBay. And they would say, “I had a poor experience on eBay”. Those types of experiences multiply over time. 

It could be something as simple as not getting your item, or it was a little bit damaged when it arrived. Then you fast forward, you know, 10, 20 years later. And we have so many different marketplaces. Whether it's an online work marketplace or ride-sharing marketplace or a hospitality marketplace, you can imagine the things that can happen have also gotten a lot worse. 

It's covering the areas of physical harm now, it's covering election interference, it's covering spam and scams where people are losing their money, their hard-earned money, on scams that are on the platform. 

So it's been super important for trust and safety to really be able to protect those customers and continue to grow the marketplaces.

Meredith Reed  5:59  

So, I'm going off of what you just said. Can you talk a little bit more about the changes that you've seen over time on peer-to-peer platforms like Upwork and eBay as technology and sophistication has increased?

Jeff Chen  6:19  

I think there was a time when eBay was in the beginning. We didn't have access to the machine ID -- that was new technology, phone verification was new technology, ID verification was new technology. Data science at the time was new technology. 

And so, as all these evolve, it makes us more capable of stopping a lot of these people who are either doing no good, or it helps us create an environment where people know the best people perform better. 

Originally, trust and safety was really just around making sure policies are followed and stopping all the bad actors on the side. And then it evolved more into helping good people succeed and helping [to] improve outcomes. That's where I think really trust and safety and that competitive advantage took off. 

And you can tell the marketplaces that really invest into growing the marketplace by having a trusted marketplace versus ones that just stop fraud. It's very different. And I think that some of those technologies have really made it possible. We can now connect. We can connect through video, we can connect through limitless information available on the internet, and just through the device you use to access the site.

Meredith Reed  7:26  

So what are some technologies that can be used to keep people safe on peer-to-peer marketplaces?

Jeff Chen  7:32  

So there aretwo ways of looking at that. There are technologies that the company itself can use. And then there are technologies that the consumer can use. 

The consumer, for instance, they can do Google searches, they can search online. There are Yelp reviews. There's limitless information they can find if they're buying from someone or working with someone. There are LinkedIn profiles. 

If it's a company, I think that's where the more of the crux [of the issue] is. It's “How do you find that information in a scalable way?” 

There are technologies like phone verification. You can find a lot of information: about the telephone number, is it voice, where it's located, sometimes the geolocation, how long it's been in service. And so, generally you know when you find a landline that's been in service for 20 years, it's typically pretty good. Or at least it’s some really, really patient fraudsters, you know, waiting to deceive someone.

Generally, it's going to be someone pretty legitimate. There's a device ID that people can capture and really understand if people are coming back over and over when they've been suspended. 

There's document ID verification these days -- where you can actually scan your driver's license and passport to see if it's legitimate. The new technology is also comparing the face. So, not only the driver's license, but it compares my face to that driver's license to make sure that I am who I say I am. 

So there's one of those technologies that have really kind of upped the bar in terms of helping people identify when things aren't going well.

Meredith Reed  9:00  

You mentioned that trust and safety could be connected to a platform’s growth. Could you give some examples of how trust and safety has been connected to growth at the platforms you've worked at?

Jeff Chen  9:15  

Yeah. One of the most common questions that usually come up in the trust and safety working groups is, “How do I sell my idea?” Because the inherent perception of trust and safety of the initiatives is that it's adding friction, and it's shrinking the business and it's, you know, creating an environment that's more difficult to transact.

And while that may be true at face value, there's a deeper kind of question you'd have to ask: What is the ultimate goal of the company? To have a single, short-term transaction that potentially could end poorly and the customer will never come back? Or, is it one that you want them to have a delightful experience, something they enjoy, they're going to tell their friends and family about? They're going to come back over and over.

In most cases, It's always going to be the latter. They want that enjoyable experience. 

So it's important for teams in trust and safety and even in the business to really identify: What are the metrics? And what are the objectives and outcomes that you want in terms of how to create that environment? The companies I've been at, and what I've seen in the industry is that if you focus on {...} -- whether it's NPS, whether it's customer success, whether it's term rates -- those are metrics that most people in the company will care about. 

And if you can focus on those, and optimize around that, then you can ultimately gain success. 

One of the things that we used at Upwork was, obviously, we wanted good outcomes, and people knew we wanted good outcomes, but it was difficult to find correlations of a good outcome today. How does it lead to revenue tomorrow, because we're all on quarterly cycles where we're measuring P&Ls. So we innovated this idea of, “Why don't we focus on Customer Success?” 

We know that, let's say, top-rated or the best-performing freelancers on the site are going to deliver great outcomes. So if we focus on getting them more jobs, those freelancers are more successful, and they also deliver better outcomes. And that's a very immediate metric that you can look at to determine success. So, those kind of short-term metrics that are long-term indicators have been extremely helpful in predicting and encouraging and enabling success over time.

Meredith Reed  11:29  

So, when you're trying to maximize the success of the outcomes on your site over time, are there any challenges that you run into with the trust and safety measures you want to use when dealing with people internationally versus just domestically?

Jeff Chen  11:47  

I think generally, {...} the primary metric is usually pretty understandable. So it's usually some form of more dollars in the pockets of the customer, or better outcomes for the customer or reduced fraud overall. So I haven't seen too much inconsistency with respect to that we usually can agree on the primary metric. It's usually the constraint metrics that vary geography to geography. 

You know, at eBay, they had [...]  20-something different countries all over the world. And the different business units in those different countries would have different criterias for tolerance. So some markets are growing a lot faster, so they had a bigger tolerance for risk. And some markets were more mature, and they had a lower tolerance for risk. And so it's [about] being able to meet those restrictions and criteria. 

That's where it became a bit more challenging: because some of the markets needed to onboard more people, and others had enough supply and they needed to focus more on quality. So it was usually [about] optimizing around those constraints, which were the more nuanced approaches.

Meredith Reed  12:49  

Well, I would also think something like verifying someone's identity. And when you're dealing with someone in a different country versus a driver's license in the US, [that]  could be a challenge. 

I used to sell on eBay for years and years. And at least back when I was selling, there was an issue of international shipments. You couldn't track them and have delivery confirmation the same way as you could if you were shipping domestically. 

So a lot of sellers would opt to not ship internationally, because then you would have the problem of people saying that they never received it, and you have no way to prove it. And then it's like, Is it the buyer’s or the seller’s fault? Yeah, you take the loss there. Have you seen a lot of issues like that with having an international market?

Jeff Chen  13:38  

Yeah, I think that is probably one of the areas I don't know. I haven't seen it solved very elegantly yet. I think there's challenges. At Upwork, you also ran into the same challenge of: if I wanted to hire a developer, I either went to the absolute best and cost is no object, and I'm willing to pay top dollar, and then there are also people who want value. They want decent strong developers, but they wanted a reasonable cost. 

And it's really difficult to tell {...} when you're trying to identify with what the buyer wants or the client wants, which one they're optimizing for. And so, you know, we've tried asking them, do you want value? Or do you want quality? And people may say one thing, but we find that they actually want the other. 

It's really hard when they say “I want top quality” and you rank five top-quality people, and one is half the price. You just naturally, as a human, go to that value. So those have always been challenging, and I think of ways that we've attempted to solve that. 

Where I've seen people trying to solve it is (1) just being transparent on the profile or on the matching page, trying to show, you know, is this person a bit cheaper, but they're a bit lower quality? Or if you're buying an item {...}, if you're in the US and you buy it from China, it will be much cheaper, but it will also take many weeks for it to be delivered or arrive. So, a lot of that has just been [about]  transparency to the customer and making sure that they understand what choice they're making. And most times, they'll be satisfied and the ones that are a little bit careless, those are the more difficult ones you have to worry about.

Meredith Reed  15:12  

Yeah, I think also the piece of educating the consumer with how the platform works. I also find a lot of people don't take the time to actually read the listing and or maybe they expect it to be like a normal big box store where you can just return things, and then they get upset about that. So clearly, I've had plenty of issues.

So this is similar to this issue, [but] for people who are vendors; who are selling a product or service on these sites. How do you manage user redemption? How do you first penalize users for engaging in behaviors that don't align with the protocol you have in place? And then, when do you let someone come back, versus blacklisting them completely?

Jeff Chen  16:05  

Yeah, I think it varies on different sites. What I've typically done in my approach is obviously, there's the black and white of, “Are you committing fraud?” If you're committing fraud, obviously no one's gonna ask a second question about terminating them forever. 

When it's a legitimate customer who has intent to do something good or use the platform as it's designed, I usually ask, “Is this person able to acknowledge what they did was a mistake? And it was a learning opportunity, and will they never do it again?” 

And if the answer is yes, they'll learn and they'll not do it again, then it's somewhere in a situation where you want to encourage them to come back on the platform, because there's always this gray area. And it's easy to make decisions when they're on the extremes, but it's that gray area where they could be an opportunist or they could be a scumbag and it's kind of difficult to tell. 

In business, some of the most successful businesses skirt that gray area. And so you don't necessarily want to turn those away, because that's that's actually a lot of growth. And potentially, arguably, it could be even some of the highest growth customers in those areas because they understand the laws, they know the rules, they know how to skirt them but not break them. 

And so for them, it's, “How do you ensure that they understand that what they did was wrong?” And so whether it's acknowledging the terms of service, whether it's signing something that says, “I will not do it again”, whether it's dropping a seven-day suspension...so they feel a bit of the penalty and understand that we take this seriously. And obviously, you know, if they do it again, then they're gone.

Meredith Reed  17:47  

That makes sense. I'm sure it's probably very challenging to figure out with a lot of these transactions who did what, and trying to keep a paper trail, so to speak, on either “no, that doesn't really exist in this particular case, what happened?” and how you can keep people feeling good about your site, but also make sure that you're regulating certain behaviors that are not productive to these platforms. 

So, how do you ensure that people in trust and safety are working collaboratively? 

Jeff Chen  18:28  

I see a million dollar question. I think in any world, not just trust and safety, it really comes down to the understanding of what the objectives are. With trust and safety, I've seen it sit in so many different organizations. I've seen it sit in marketing, I’ve seen it sit in engineering and product and the business unit’s operations. To me, [where it sits is] less important. It's not really critical where it sits because trust and safety is so cross-functional. 

In order for trust and safety to be successful, you have to touch every area of the business -- from registration through the transaction, all the way through payment, or maybe withdrawal. 

And so for me, it's it's really important to gather a very diverse cross-functional team of different stakeholders to address the problem you're trying to agree on, the problem you're trying to address, early on. 

So, they all know what problems are we trying to solve for? Oftentimes, what I've seen is that people come to those teams as more of an approval check at the last minute to say, “I want to do X. Do I have your blessing? Yes or no?” And the first reaction is like, “Whoa, why are you adding friction? Why are you trying to shrink my business or harm my metrics?” 

And so, involving them early on really shows that you're open to different ideas, you're willing to brainstorm, you're willing to address different opportunities, and maybe even find a middle ground where it actually works for both teams and then move forward with that. What I’ve often found is that people tend to be very embracing with that. They tend to be supportive. They understand what you're doing. And then when it comes time for the final sign off, they're not surprised -- they've been involved throughout the process. 

And so it's making sure that Legal is okay with a solution, Marketing understands how to speak about it, Product knows how to design it. 

Look it Upwork. What we found is that we want to do ID verification because there's a group of fraudsters that were misrepresenting themselves. And so we wanted to scan their driver's license and also have them show up on video so we can compare the face. And at face value, that sounds like a lot of friction. And rather than develop the solution and go to the Marketing and Sales teams afterwards, we actually went to them early -- and they were actually very encouraging because the Sales team was saying, you know, “My clients, my customers, actually want to know if the customer has been ID verified. So this is actually an advantage of competitive advantage for you to sell.” 

You know, in Marketing, they could actually put out articles and content that says, “We verify our freelancers. We take security and safety very seriously.” 

And so, being able to be able to spin it, being able to explain why you're doing [what you’re doing through] a good person's lens and how it can actually grow the business has been really helpful in achieving a lot of the goals on both sides: stopping bad people, enabling good people to succeed, and also just making sure people trust your marketplace a lot more.

Meredith Reed  21:09  

Thank you. That was really helpful. And I think as a consumer, as someone who is going on these platforms, I want to know that they are ID verified. I think, at this point in time, most people know that there's some sort of risk with any one of these peer-to-peer platforms. So I don't think you would be planting the seed in [the customer’s] head that there might be a [seller] with not-so-great intentions.

I think that's probably the way to go is just, you know, having the synergy of all your departments and making sure that you're taking those preventative measures.

Jeff Chen  21:48  

We've even found that people tend to trust the internet even more so than in-person [transactions]. Sometimes what I find is [that] they buy from a big site like eBay or they work at a job on Upwork. They just somehow inherently assume they're going to get this retail or world-class Fortune 500 company experience. 

And, you know, I'm sure half the people realize this is just an intermediary platform. And the other half of people actually expect that this is like going to Walmart or going to Target and buying something. 

When we were doing ID verification, probably half the customers actually thought we already did it, but then the other half didn't. It's one of those things where you do have to address the diversity of people's understanding and knowledge or familiarity with the internet and how risky it can be. 

The people that are really worried, you probably don't have to worry about them as much. It's really the people that are a bit naive or just trust the platform. Those are the ones you really need to make sure you do your due diligence to make sure that they get the outcomes that they expect.

Meredith Reed  22:46  

That's very true. And you know, after I said that, I was thinking, the worst thing you can do is assume that the way you see the world is the way that everyone sees it. And there's certainly a wide spectrum of people’s understanding of these platforms. I've seen that a lot in my personal experience as well. 

It's currently April 2020. So, we have to talk about Coronavirus and the pandemic and how this has had an impact on every single industry out there, essentially. How has Coronavirus had an impact on peer-to-peer marketplaces? And how do these marketplaces respond to crises like the one that we're in now?

Jeff Chen  23:33  

It's an unfortunate and definitely an unplanned event that's happening. I think I would bucket this in more the kind of the macro environmental or natural disasters that can happen. It's something that you know is possible. It's inevitable that something like this is going to happen, but you never really expect it to happen tomorrow. I I think Coronavirus kind of snuck up on all of us. 

What I've seen with marketplaces is It's really the time that you can really distinguish which companies are prepared and which ones are really willing to stand with their values. 

If you go with the extreme, Coronavirus, COVID-19 -- this is definitely a huge global pandemic. And it created this area where, again, it could be in that light gray area of opportunities. You know, people stocking up on supplies and hand sanitizer, toilet paper, paper towels, and then trying to sell it for a premium. And obviously, it's something that's against the law and it's just not great for the human race. It's not another thing that you really want to be taking advantage of and profiting from. 

And so you find companies like eBay and Amazon [are] launching your anti-price gouging initiatives, where they are stopping the price gougers and not allowing them to sell for multiples higher. It's a good testament to people who can prepare and then ultimately try to execute on that. It's obviously very difficult when there probably aren’t SKUs for all these types of items and there's different ways of trying to find people to sell these on marketplaces. 

There were times when other pandemics happened -- or not “pandemics”, but just global natural disaster events -- like there was a volcano in the Philippines that happened last year and people lost internet internet access. They lost power. They were forced to evacuate. 

And one thing Upwork did is we actually sent emails [to all of their clients] saying, “If you're working with a freelancer in the Philippines, they may have to evacuate. So if there's a delay in communication or work, we just wanted to let you know this is why. Please be patient and wait for them. They'll be back online as soon as they can.” 

And the clients really appreciated that because not all clients follow Philippino news and they're not all aware that there was a volcano eruption. The freelancers also were very appreciative because we had their backs. We were looking out for them. Their feedback and reputation is very important, and the last thing you want is for them to get penalized for something completely out of their control. 

So those are some of the types of initiatives that companies can do to really help out in situations like this -- whether it's offering free hospitality stays or donating money to the World Health Organization to help find cures and treatment. These different companies have different opportunities or different resources to help out as best as they can. 

So far, it seems like they've been really positive in this time of crisis. And so that that makes me happy -- that people are always trying to do some good in the world.

Meredith Reed  26:30  

Yeah, me too. I've really been so encouraged by seeing how many online businesses have been there for consumers putting out free upgrades and free content. Just like what you were saying -- sending out notifications about [what] has happened and [to please] be patient and having the best interest of the people on [their] platform in mind. I think everybody has really appreciated that in this particular crisis, and I'm sure, in any crisis. 

So, I think that concludes our interview conversation. So I just want to say thank you so much for being here.

Jeff Chen  27:13  

Thanks for having me. It's been great.

Meredith Reed  27:16  

This is Jeff Chen. We will link to him in the show notes. And that's it for this episode. So we will see you next time on the free Okay. Thank you. Okay. Thanks.