Practical Fraud Prevention

Fraud and AML Analytics for Fintech and eCommerce, using SQL and Python

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

Gilit Saporta and Shoshanan Maraney

Practical Fraud Prevention

by Gilit Saporta and Shoshana Maraney

Copyright © 2022 Gilit Saporta and Shoshana Maraney. All rights reserved.

Printed in the United States of America.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

- Acquisitions Editor: Michelle Smith

- Development Editor: Corbin Collins

- Production Editor: Kate Galloway

- Interior Designer: David Futato

- Cover Designer: Karen Montgomery

- Illustrator: Kate Dullea

- April 2022: First Edition

Revision History for the Early Release

- 2021-03-05: First Release

- 2021-06-22: Second Release

See http://oreilly.com/catalog/errata.csp?isbn=9781492093329 for release details.

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Practical Fraud Prevention, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.

The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-492-09325-1

Chapter 1. Fraudster Archetypes

A NOTE FOR EARLY RELEASE READERS

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 2nd chapter of the final book.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at ccollins@oreily.com.

If you were trying to pin down the main difference between an analyst who works extensively with data, and a data scientist, you’d likely touch on the concept of domain expertise as a key value that analysts bring to the table. Data is enormously valuable, but it won’t get the results you’re looking for without a deep understanding of the context. This is particularly important with fraud fighting, since you’re fighting an enemy that actively fights back, changing its techniques and patterns to evade detection and trying to reverse engineer and thus avoid the traps you’ve created to catch them. You don’t want to wait until you have a lot of data about a new attack type or new fraud ring; you want to catch it before that.

Fraud domain experts are often the ones to provide the “secret sauce”, those impactful variables that really boost the performance of a fraud prediction model. If we had to narrow down that “secret sauce,” that domain expertise that every fraud analyst should hone to perfection, we’d say that it ultimately comes down to fraudster profiling. A good analyst helps her team understand that not all fraudsters are created equal. Some of them are amateurs; others are pros. Some of them are native speakers of the language of their victims; others rely on auto-translate. Some are very tech-savvy, so trying to beat them with checkout barriers would be futile (e.g. if they’re using brute force bots, captcha would be counter-productive).

Therefore, it’s essential to learn how to group fraud attacks based on the guesstimated profile of the human being behind them. It’s worth mentioning that user profiling in general is a powerful analytic practice. Many departments seek to better understand their target audience by breaking down the group they’re speaking to into approximate groups, each one of which has specific characteristics, behaviors and pain points. Discussing the groupings they’re using with marketing, sales or product teams can actually be a good, regular touchpoint for fraud teams to get together with other departments and build lasting, mutually beneficial relationships. It can also help your team work to avoid false positives.

That said, a fraud team’s key target audience is rather different. A fraud analyst is primarily focused on fraudsters. Breaking down the types of fraudsters your team is facing, and which characteristics each is likely to show, helps the team to identify and block each type, working out which ones most regularly attack your business. This should contribute to your strategy for how to prevent fraud both now and in the future, may help you work out where to invest most research, resources or new tools, and can even lead to insights about how and why your business is most vulnerable to fraud - and, therefore, what you could do to make yourself a less attractive target.

In this chapter, we cover some of the most common fraudster archetypes. We hope that this will provide you with a useful mental framework for understanding the online criminals you work against, as well as the practical strategic help as described above. Some will be more or less relevant to you depending on your industry (banks are more likely to be targeted by Psychological Fraudsters than by Amateur Fraudsters, for example) but there are no hard and fast rules for this, and we encourage you to be aware of all of these archetypes so you can protect against them should they attack you.

It is important to bear in mind, however, that these are archetypes, not actual people, that we’re discussing; so don’t expect every single fraudster to conform to type in every single way, every time. In particular, you need to remember that these days fraudsters rarely work alone, so you may often have more than one archetype in play as part of a large coordinated attack. Even amateur fraudsters work in groups or spend time on online forums learning from and sharing experiences with other fraudsters-in-training, and they benefit from guides and burnt cards made available by more experienced fraudsters. Mechanical Turk style fraudsters are only really a threat because they operate as cogs in a larger fraud machine, guided by managers and fueled with data stolen by other criminals.

Many fraudsters, particularly those for whom fraud is their profession, work as part of an organized criminal initiative, with colleagues who specialize in areas such as botnets, phishing campaigns or design (to create convincing fake websites and ads), to name just a few. Larger organizations boast their own financial and even HR officers. There are also consultants who can be hired. Online fraud is a big business.

Amateur Fraudsters

The most common type of fraudster, and the easiest to spot, is the amateur fraudster. This is either someone who has only recently entered the world of online fraud, or someone who engages in fraud as a side job or hobby rather than a profession. This can include young people attempting a bit of fraud without really understanding the criminal nature of the enterprise (it’s something they’ve seen on TV) or attempting to steal items currently popular among their cohort which are out of their price range.

The amateur approach, by nature, tends to be somewhat slapdash and this type of fraudster will rarely think through all the implications of their online appearance. For example, they may not try to match their apparent IP to the victim’s IP, or if they do, they may not remember to ensure that other details of their profile match that location. On the other hand, amateur fraudsters can come from anywhere, and if they’re local to your country or even area they have the advantage of local knowledge; they can easily take advantage of specific holidays, and are less likely to make mistakes in language, time zone and so forth. This can make it easier for them to slip by if you’re not careful.

Even fairly simple rules are usually enough to catch this kind of fraudster out - but make sure that you do have a mechanism in place to catch them, because if you don’t, even an amateur fraudster can cost you money. This is particularly the case because amateurs share information and tips constantly on forums, social media and messaging apps. The better the tips, the higher the street cred of the tipster. It’s a good way to start building up a reputation online. So if one does find a vulnerability, you might be in for a flood of fraudsters coming your way.

Amateur fraudsters are also sometimes used as a cat’s paw by more sophisticated fraudsters. For instance, a fraudster carrying out targeted, carefully concealed, high-value attacks using a stolen credit card might also post this credit card on the darknet as a “burnt” card for free or at very low cost. The resulting low quality attacks using the card then conceal the work of the experienced actor. So it’s worth making sure your system is designed to correlate this sort of data, in case a flood of amateur fraud has a pattern that can lead you to more sophisticated and damaging fraud elsewhere.

The other interesting thing about this persona is that some amateurs go on to polish their method and master the finer points of fraud. If you’ve managed to get a really good lock on their location, IP, profile and so on while they’re new and careless, it’ll be far easier to re-identify and stop them when they’re more advanced. So don’t dismiss this category as irrelevant to your efforts; think of it as part of your long-term investment in fighting fraud.

Mechanical Turk Fraudsters

The original Mechanical Turk was itself a fraud - a fake chess-playing “automaton” which appeared to be able to play by itself but was actually controlled by a hidden operator. There’s a similar situation here, with Mechanical Turk fraudsters. Think of this as a sweatshop-style setup. Typically found in developing countries, this is a situation where many poorly paid individuals are paid to sit packed into small spaces, with a laptop each, and run through lists of designated sites and stolen data which they are given by their manager.

You won’t see the idiosyncratic aspects that make other fraudsters stand out for fraud teams with this type of fraudster. Effectively, these are brute force attacks at scale - but with the human element built-in, avoiding the traps that actually automated attacks can fall into because they’re machines.

Each fraudster essentially follows a “script” of predefined actions each time, and repeats them until they succeed, at which point control may be handed to a more experienced fraudster - so, as with amateur fraudsters, be sensitive to this pattern, which can help stop more sophisticated attacks as well. Often, apps or programs are used to make actions like cleaning the browser or device, or swapping to an appropriate IP, automatic.

It’s these sort of attempts that sometimes make fraud teams want to block entire regions or even countries, simply due to the sheer scale of the attacks involved. For this reason it’s important to retain perspective; the attacks are rarely that sophisticated, and the scale is in reality due to the sweatshop setup and not the population of the surrounding area. Don’t throw the baby out with the bath water!

Psychological Fraudster

You might also think of this type of fraudster as a social engineer. The psychological fraudster is a very different personality and mentality to those we’ve described so far. This is the fraudster who is behind Victim Assisted Fraud schemes, where victims are tricked out of their money by a plausible stranger with a lie tailored to their situation.

Think of elderly people scared into believing that their bank account has been hacked, and they have to set up a new one at once - they just need to give the nice young man on the phone all their login and security information and he’ll take care of that right away. Authorized Push Payment fraud is a matter under much debate in the banking industry and beyond, and this is the fraudster archetype behind it.

Another common attack by a psychological fraudster is romance scams, where a fraudster pretends to be romantically interested in a victim and then persuades them to send him money in gift card form, to meet some imagined urgent need. In retail or travel, they might be offering a deal that’s almost too good to be true, with some rational explanation for the great price - and a reason you need to decide immediately, rather than take time to think. They may even have set up ads for similar products, so that they know they’re targeting the right audience.

Hitting your business or financial institution itself, the psychological fraudster might send phishing emails which purport to come from suppliers you actually use, using almost identical email designs and website addresses and appearance. Or they might pretend to be your own IT support, calling at a particularly busy time of day to fix an urgent problem - you just need to give him remote access to your computer and he’ll take care of it all in the background.

In contrast to the archetypes described so far, this fraudster takes time over their schemes. Often considerable research goes into targeting an individual in the way that will leverage their particular weak points, and which uses supporting evidence to make them sound more legitimate.

These attacks are harder to catch, but worth watching out for, because the attacks are often carried out for high value, and unfortunately the techniques are highly repeatable, and you’ll want to catch any patterns before they become seriously problematic. Once you know what you’re facing, you can invest in steps against them.

For instance, if it’s Victim Assisted Fraud (VAF) that’s draining your business, you can put efforts into knowing your customers and matching current behaviors to past typical behaviors. Inconsistencies would raise a flag for you to investigate. If there are duplicates of your app or store being used for nefarious purposes, get them shut down, communicate with your customers, and keep an eye out in the future for repeats.

In general, combating psychological fraudsters requires fraud teams to take a wider view of their role, almost stepping into a trust and safety mindset. Collaboration with other departments may be necessary in blocking the vulnerabilities you unearth. It’s worthwhile keeping this in mind - because when these criminals are successful, the results often flow downstream to become the fraud team’s problem.

Product-savvy Fraudster

Product-savvy fraudsters are among the few people in the world who actually bother to read the fine print of your user agreement. They know all the ins and outs of your returns and refunds policies. They sometimes even take the time to get to know your customer service reps individually, so as to target whoever picks up the phone most effectively.

In essence, the superpower of this fraudster is a deep working knowledge of your business, how you relate to customers, and your products. If you have different policies for different types of product, or different price ranges, they’ll know.

Some product-savvy fraudsters focus on a specific business; there are those who specialize in defrauding a particular retailer, marketplace or payments company. Others target a specific track common to many businesses - for example, there are fraudsters who master refund scams, others who are kings of coupon abuse, yet others who are experts in leveraging stolen loyalty points, and so on.

WARNING

Product-savvy fraudsters may target your call centers, your online chat or the customer service you run by email. Many have preferences as to mode of communication, but equally many do not. Similarly, while many product-savvy fraudsters operate at a remove, some product-savvy fraudsters commit attacks using the buy online pick up in store/return in store (BOPIS/BORIS) trends, picking up or returning goods either themselves, in person, or using a mule.

This diversity of approach can lead fraud teams to treat each channel separately, as a different type of attack. This is a mistake. The value of seeing fraudsters according to their archetype is that you can see the patterns of attack across channels.

Fraudsters do not distinguish by channel, except when they’re pinpointing vulnerabilities. They will exploit weaknesses wherever they find them. They are channel-agnostic. You need a similarly neutral perspective when you take a big picture view of who’s targeting your business, and how.

Some fraud teams dismiss product-savvy fraudsters as “a customer support problem” or something which “can be solved with policies.” While this is understandable, we feel it is short-sighted and can leave your business vulnerable.

Firstly, as with the Psychological Fraudster, these are problems which ultimately impact the fraud prevention team. Even when there’s no chargeback, loyalty, coupon and refund fraud cause significant loss to the business and when questions are asked about fraud, they’ll typically be asked of the fraud team. Secondly, these fraudsters may well work in more direct forms of theft as well, and if you can’t make the connections, you’ll find it harder to identify them accurately.

Thirdly, the risk posed by a malicious actor who knows your business so well is simply too high to ignore. A personal example from Gilit’s experience seems the best way to illustrate this point to doubters.

This story features a product-savvy fraudster who had a little technical knowledge as well. They had worked out that a certain retailer allowed all types of special characters in the “customer name” field. Au fait with every aspect of the retailer’s customer service practices, this fraudster also knew that when a Customer Support representative was asked to delete an account, the standard practice was for the rep to copy and paste the account details, including the “customer name” field, to a request form.

Putting these two things together, they signed up to a new account under a name which included the SQL drop tables syntax command. (Possibly this fraudster was a fan of XKCD.) The logic, of course, was that the details of the request form would likely be automatically copied and pasted into a backend script, which would execute the “drop tables” command. Now, this attack is of the malicious cyber variety rather than fraud specifically, likely as part of an attempted blackmail attempt. But with a slight alteration, a similar sort of attack could equally be carried out as part of a fraud campaign as well (though we’ll leave the details to your imagination, since we haven’t seen this in the wild yet and don’t want to give anyone ideas). Fraud teams need to have this broader awareness as well, just in case.

Luckily, this attack was caught in time, thanks to a wonderfully paranoid developer who had already thought of this possibility, and had guarded against it. But not every company has developers who think like criminals! The point of this story is that product-savvy fraudsters have the potential to cause real and lasting harm to your business. That’s inherent in their in-depth knowledge of how you operate. So watch out for them.

Tech-Savvy Fraudster

Tech-savvy fraudsters are behind the automated attacks that have been such a painful feature in fraud fighters’ lives since the mid-2010s. We’ll split this fraudster archetype into two, based on the objective of the criminal concerned, and look at both the Bot Generator and the Hacker type of Tech-savvy fraudster.

Bot Generator

This fraudster is one with decent scripting abilities, able to generate lots of reasonably convincing transactions on your system. Sometimes the only “tell” might be something like the fact that the emails are composed of random dictionary words. (It’s automated; the content needs to come from somewhere.) Watch out for these attacks because they come fast and furious.

The fact that the attack is scripted means that playing “cat and mouse” with this type of fraudster can escalate quickly (they can fix their mistakes far faster than a manual fraudster, if they’re clever enough, and once they’ve crafted the perfect attack they can do it instantly at scale.

It’s best to find “honeypot” based defenses against this fraudster, so that they are initially drawn in rather than instantly bouncing off to look for alternatives vulnerabilities - e.g. instead of declining orders too soon, try to make the fraudster burn through their resources by asking for 2FA. If your process takes too long, and commands too many resources, they’ll likely look for an easier target. It’s all about ROI.

WARNING

It can be tempting to assume that, because bots can generate a lot of volume in fraudulent orders, AI-based models are the best defense, as they’re the best able to keep up. This is a dangerous approach.

Keep in mind that any attempt to use machine learning when the training set is heavily tilted towards bots will result in wild over-fitting. For instance, to give a simple example, your model might learn that only 3am payments should be deemed to be fraud, simply because the tech-savvy fraudster wrote a script that submits bad orders every day at 3am. If you train a model to tackle this attack, you’ll be playing into the hands of the fraudster, who will simply make a small adjustment to their script, leaving you with a model that needs lengthy retraining.

Don’t think fraudsters don’t plan that far ahead; they do. You can see discussions about exactly this kind of many-staged attack on fraudster forums. They’re not just trying to defraud you, they’re trying to outwit you. Don’t play into their hands.

Hacker

This category reaches beyond a fraud team’s standard parameters, but we think it’s important for fraud analysts to bear it in mind - and not only because an appreciation of the bigger picture is useful. Sometimes, the consequences of their work will affect your job as well.

In the Hacker category we’re including offenders whose aim is to access and, in some cases, take over your servers. Some of them hope to blackmail you or force you to pay a ransomware fee for the recovery of your data. Others aim to collect a bounty from a competitor who would benefit from a DDOS attack on your operation. (Not a nice thought, but it happens.) Many of them aim to obtain your own customers’ PII, which they then sell to the highest bidder.

Certainly, these hackers are mostly the concern of your CISO, and these types of attacks, while extremely serious, do not fall under a fraud team’s remit. What is relevant to a fraud team, however, is that sometimes these attacks will include an influx of fake transactions, which might blind your volatility-based or velocity-based fraud detection models/heuristics. This is why preparing for hacker attacks should not be just a CISO’s job. A fraud analyst’s responsibility here does not end with good online security hygiene and practices, making sure not to fall for phishing scams or click on dubious links. It should also be your responsibility as a fraud manager to prepare for malware-induced spikes in traffic, which will otherwise throw off your detection processes.

Organized Fraud Crime

Organized fraud crime is an international affair, and such organizations may be based in any country in the world - or, indeed, in several. Simply put, fraud has become an appealing way for people with either few good options or a love of the illicit (or both) to ensure a good and reliable income.

The difficulty of pursuing crime across borders, the “fuzziness” over legal jurisdiction for online crimes and the relatively low amounts of money often involved in specific instances of crime all combine to make online fraud a fairly “safe” industry for criminals. Moreover certain types of fraud, such as adtech fraud, remain in a grey area, where it is not yet universally agreed what is criminal and what is not. Similarly, bear in mind that Initial Coin Offering (ICO) scams were legal until 2018.

The law is always far behind technology, and fraud is no exception. On the contrary - fraudsters, for whom “don’t care about the law” is practically in their job description, have a natural advantage over companies which take care to adhere to regulations. Their creativity has no boundaries. Within this context, it’s no wonder that some fraudster operations have become more and more institutionalized.

Some common forms of organized fintech fraud are mafia operations trafficking PII, forcing or paying shell companies / shell people to open bank accounts in order to launder dirty funds by buying crypto, or precious metals, or make peer-to-peer (P2P) payments for made up services, and so on and so forth. These groups are of most relevance to banking and Anti-Money Laundering (AML) fraud fighters, but they may also be seen attacking e-commerce or marketplaces.

Distinction Between Organized Crime and Mechanical Turk Fraudsters

In our categorization we are distinguishing between organized fraud crime setups and Mechanical Turk sweatshops, even though the latter are often a part of the former - an early stage in the lifecycle of a wider organization. For fraud analysts, however, the distinction is important, because they work differently and the identification process is necessarily different as well.

In particular, note that organized crime operations tend to be well-funded, and therefore can afford top-notch tech. For example, the VPN or hosting services they use can be easily and automatically matched to the location of the stolen credit card being leveraged. Of course, that can become a weakness for them, if you become able to spot telltale signs that that’s happening.

Similarly, these groups can afford to operate at scale, which can also affect their impact - for example, adtech fraud fighters need to bear that in mind when analyzing click farm fraud, or fraud analysts more generally should consider it when balancing the risks of sim farms.

Small But Organized Crime

There is “small but organized crime” which also deserves a mention here. This type of fraud is less well-funded, though generally meticulously planned and quite well staffed. Examples here would be work-from-home schemes, where 2-6 mules are (often unwittingly) commanded by a single arch-fraudster. Travel agencies fit the same pattern, with agents often unaware that the spreadsheet of customer details they’re converting into flight ticket orders are using stolen cards. Sim farms and click farms may also fall into this category, when they operate on a smaller scale. Here the reason behind the scam is important: for instance, is the click farm being used to monetize incentivized browsing, or to sustain fake social media profiles that will be used to fuel fake news?

Fraud analysts are more likely to be able to spot the patterns here, since smaller groups like this often have a set way of working - particularly necessary, for the manager, when mules are unaware of what they’re really doing. You need to be able to spot patterns even when multiple individuals are involved. For example, trying to identify the same obfuscated IP won’t help you, but being able to identify a set pattern of actions or movements will.

Friendly Fraudsters

You’ll see that friendly fraudsters are not a primary focus in this book, except in areas where they’re specifically relevant such as here in the archetypes chapter, or in the next chapter where we discuss crises, or in the chapter dedicated to this topic where we discuss friendly fraud and refund fraud. That’s certainly not because there isn’t enough to say about friendly fraud - we could write an entire book on this topic alone.

Rather, it’s because to an extent friendly fraudsters don’t fit the model we’re discussing here. Firstly, a friendly fraudster is typically someone using their own genuine identity, credit card etc., who then submits a chargeback, claiming fraud, rather than honestly paying for the goods. Some do move to more sophisticated (though still basic) levels, setting up throwaway electronic payments accounts and using proxies and so forth, and we would classify them as shifting to the Amaetur category.

Secondly, and relatedly, the methods you use to combat other kinds of fraud won’t work against friendly fraud. It won’t help you to uncover their real identity and confirm that their name, address, email and phone belong together. Even if you have what seems to be cast-iron proof that this real person made the purchase, you may not be able to fight the chargeback; the customer is always right.

Thirdly, not all fraud teams are responsible for fighting friendly fraud. It may be classed as a service, customer support or chargebacks problem. You might be asked to provide evidence so that your business can contest the chargeback, but you may well not be expected to prevent these instances. It is, after all, very difficult to know the intention of a real buyer at the time of transaction.

That said, all fraud chargebacks count towards your company’s fraud chargeback limit, so you may well be looking for ways to stop this kind of fraud. Working with other retailers or financial organizations, especially in your own industry, may be the most effective way to identify friendly fraudsters - at least before they’ve run up many chargebacks on your site - since this is often a repeat offence across multiple sites. We’re not suggesting sharing personal user information in the form of lists, which may have data privacy implications, particularly in the EU, but you can employ a form of Privacy Enhancing Computation as part of the collaborative effort to ensure that no personal data is shared.

Despite all this, we are including Friendly Fraudsters in this chapter, and elsewhere in the book where they are relevant, because of the time in which we’re writing this book. Friendly fraud has been on the scene for over a decade, and has been steadily growing as a concern during that time, but 2020 saw a notable increase - the result of the economic situation and general uncertainty and stress.

The friendly fraudster is either genuinely mistaken (which can happen, especially when people are under stress), acting on the infamous Buyer’s Remorse (which increases in time of crisis, because people find they have less money than they had anticipated), reacting angrily to receiving the wrong product or receiving the goods late and incorrectly filing a fraud rather than a service chargeback, because that’s a more reliable way to get the refund (which again, increases in a crisis which impacts ship time and when people are more desperate than usual for refunds), or had always intended to try and get the goods for free (which also increases in crises, because people want items, to cheer themselves up, but can’t afford them). Underlying all this is a sense of betrayal, since people are suddenly unable to afford or to receive things they had come to expect as a normal part of their lives - and this makes them less considerate about the difference between service and fraud chargebacks, or what it might mean to a retailer.

The economic and psychological factors behind this trend are likely to be with us for some time, and it’s for this reason that we are including Friendly Fraudsters as a category here, with some discussion of the factors influencing Friendly Fraudsters’ actions and decisions. Consumers who learned how easy it is to file a fraud chargeback as a result of the pandemic period may well be more likely to do so again, even after the crisis has passed.

Different businesses have different levels of tolerance for this behavior, with policies to match, and some differentiate between cases where this behavior was planned, and become repetitive, and cases where a buyer originally had legitimate intentions but acts on “buyer’s remorse” when the bill comes through. In some ways, this is more a matter for policy than for fraud analysis, but at a time when friendly fraud has grown significantly and is likely to stay with us for some time to come, it’s worth working with your policy decision makers and ensuring you’re part of that loop, because the more serious this trend becomes, the more likely it is to affect your department.

Pop Quiz

Do you think the following set of payments was the work of a single bot, a random group of amateurs, or an organized operation of Mechanical Turk fraudsters?

Name on Card

Time of Day (EST)

Email

IP

Amount in US $

Shoshana Maraney

01:12:03

shomar1993@gmail.com

102.129.249.120

30

Gilit Saporta

02:26:03

gilsap83@gmail.com

102.129.249.120

30

Amelia Blevins

09:05:59

ameblev23@gmail.com

102.129.249.120

30

Winnie T. Pooh

16:45:14

winpoo1926@gmail.com

102.129.249.120

30

The honest answer is, obviously, that one would be reluctant to make a decision solely based on 5 pieces of data - and we strongly recommend that you work off far more than this! That said, the very simplification of this example highlights something important: Even with this limited data set, you probably have an intuition about the set of payments.

This is a very valuable ability, not to be overlooked - because it’s based on your substantial experience of past transactions and interactions, both fraudulent and legitimate. Sometimes, when you have only seconds to make a decision, this intuition can be one of the most valuable tools you have. More than that, as Alon Shemesh pointed out, “good analyst intuition will tell you not only which questions to ask but, even before that, when there’s a question that needs to be asked in the first place.”1

Here are a couple of pointers to help when you want to test your intuition - and these are relevant even when you have far more data to work off.

First, repetition in IP, amount and email domain naturally point to a single fraudulent entity behind all purchases. The IP is most likely a proxy (or VPN, or manipulated traffic, etc.). This points us to the fraudster profiles that like to work en masse: Mechanical Turk, organized crime or tech savvy.

Secondly, you can break down this apparent identity into data points, and analyze each one to work out which persona its characteristics suggest. For example, to narrow it down even further, consider the following comparison in Table 1-1.

Table 1-1.

Mechanical Turk (low grade paying jobs for up to ~10 mules/juniors guided by a single fraudster, usually 3rd world)

Organized crime (operations of dozens of employees, usually well-funded, not just 3rd world)

Tech-savvy fraud (scripted bot, designed to automatically complete checkout/signup)

IP 102.129.249.120 - US IP, free proxy

The low grade proxy matches the low budget of most Mechanical Turk style operations

Amount $30

The low ticket price might match the low expectations of some Mechanical Turk style operations

The low ticket price might match the tech-savvy fraudster, since it’s fairly easy for this fraudster to generate a large volume of small orders. He may also imagine that low-ticket orders might be more likely to pass unexamined.

Winnie T. Pooh

A fake name might match the trait of shell identities, which is typical for organized crime

A fake name might match the trait of beta testing, which many tech-savvy fraudsters attempt before they launch a final script

Gmail

Might match this sweatshop style of working; one of the group may be detailed to create such accounts, or it might simply be part of a “script” they all follow.

Gmail accounts are somewhat harder to create automatically, so they are less suitable for most script attacks. However, the emails might be bought/scraped from an external source.

Partially repetitive email pattern

The email usernames show a pattern of 3 letters of first name+last name + digits. Some of the emails show alleged year of birth, though using inconsistent date format.

This partial consistency suggests that different people submitted the orders and that while they were instructed to stick to a uniform format, they weren’t professional enough to do so properly.

Time frames

The orders are mostly submitted during off-hours for US, which might indicate that they are coming from elsewhere

Pace

The orders are submitted in changing intervals, which is usually not the work of a script

Summary

In this chapter, we’ve explored the distinctions between different fraudster archetypes, and discussed how to use this framework in practical fraud identification and prevention, as well as within a wider framework of understanding the fraudster fraternity and the threat they pose to businesses.

In the next chapter, we’ll look at things which all these archetypes have in common - such as an results-driven mindset, creative willingness to exploit any weakness, including crises, and a holistic view of the customer journey when it comes to attacking it.

1   Conversation between the authors and Alon Shemesh, November 4, 2020

Chapter 2. Fraud Analysis Fundamentals

A NOTE FOR EARLY RELEASE READERS

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 3rd chapter of the final book.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at ccollins@oreily.com.

In this chapter we’ll cover the fundamentals of fraud analysis; elements which are relevant to every fraud fighting team, regardless of industry. These topics - foremost among them thinking like a fraudster, but including others such as the distinction between account and transaction in fraud prevention, practical anomaly detection and crisis planning and response - are important to understand when combating a wide variety of types of fraud, and also to bear in mind when developing a wider strategy of fraud prevention for your organization.

We won’t be able to mention all of these everywhere that they’re relevant throughout the rest of the book. They’re relevant to practically everything in the book. They’re the fundamentals you need to keep in mind while reading, the context in which everything else takes place.

Thinking Like A Fraudster

The most fundamental skill of them all, for a fraud analyst, is thinking like a fraudster. The technical aspects of fraud detection and analysis are vital, but real success requires the right mindset: The mindset of a fraudster.

Fraudsters look for vulnerabilities everywhere, and attack where they see a way in. You can’t rely on them behaving as they have done until now; their whole MO is based on trying to trick your business. Predicting fraud is not like predicting most trends - because fraudsters fight back. To outthink them, both day to day and when you’re working on high level strategy, you must first know how they think.

A Professional Approach to Fraud

Crucially, fraudsters focus on ROI. Their aim is to gain as much as possible (meaning, to steal as much as possible) with as little effort as possible on their side. These are not dramatic state-sponsored bad actors, or black hat hackers from films. These are professional thieves. They’re more like the head of a small business department whose eye is always on their KPIs, than like anything you’ve seen in a film.

This trait lies behind many of the things about fraudsters we take for granted. For example, if one target hardens against their attacks, they move to an easier one - because otherwise they’re putting in greater effort, for potentially reduced returns. Similarly, they often attack particularly popular items - because they’re harder to catch among the flood of legitimate interested buyers, and because the items themselves will be easy to resell for a good price.

It’s important to understand the mindset behind these trends, because in that way you can fully understand the evolution of both techniques and tools used for fraud. Some years ago, fraudsters were most likely to try to get the highest priced items on a site. When rules were put in place to guard against this trend, they moved to ordering multiples, or combinations, or many separate orders of cheaper goods. When fraud protection systems started to flag automated form fills, fraudsters began adding in random breaks in the “typing” to make the automation look more human. For them, it’s all about finding the optimum balance between the easiest, fastest method and the largest payoff.

Similarly, internalizing this mindset will enable you to move between industries in your career and apply all the things you’ve learned until now even though they’re given different expression in different industries. Fraudsters attacking clothing sites will often pretend to need bulk orders for groups, clubs etc. - a perfectly normal use case in this context. The same fraudster on an electronics site will cleverly pair a laptop with keyboard, mouse and headphone - because they know that looks normal in that context. You’ll know it’s all the same move. In the same way, the same fraudster will attempt APP fraud against a bank, and Victim Assisted Fraud against an online gift card store.

Treat Categories with Caution

It’s also important to remember that while we will be breaking types of fraud attack, obfuscation techniques and so forth into categories for convenience and clarity, fraudsters don’t think like this. For instance, fraudsters do often specialize in a particular industry; this is down to ROI again. Specialization maximizes the value of the research they do into which products can be stolen most easily, how to best leverage the results of the thefts etc.

However, fraudsters don’t limit themselves; one report found that 86% of fraudsters commit fraud in more than one industry.1 They need another string to their bow, for when their industry of specialization is going through a slow time, or they’ve made it too hot to hold them for a while.

This comes down to ROI again; a fraudster who has taken time perfecting a particular attack might find that the ecommerce sites they try it against have all learned how to block it. But the same might not be true of banking websites. So they’ll try it there, to get the most out of the technique before they have to invest in revising it.

It’s the same for fraud attack types. We’ll distinguish between account takeover (ATO) and transaction fraud, and your department probably does too - but fraudsters don’t. The same report found that 92% of account takeover fraudsters commit another type of fraud as well.2 For a fraudster, it’s about the result, not how they get there. Taking over an account is one way to trick a site into letting you use someone else’s details to make a purchase. Stealing and using loyalty points, and using stolen credit card info, might seem very different - but to a fraudster, they’re very similar: they’re using something that doesn’t belong to them to steal something they can monetize.

The tricky thing is that for fraud teams, the way you catch ATO, spot stolen credit card fraud, and protect loyalty points, are different. In some organizations they’re owned by different teams. But if you don’t bear in mind that for fraudsters, these are all different moves in the same game, your fraud prevention strategy will fail to combat the mindset of the enemy.

Account Versus Transaction

One division which is often employed in fraud detection circles is that between analyzing and decisioning accounts, and analyzing and decisioning transactions. It’s a practical distinction, because it reflects the differences between two very different types of online interaction, from the consumer side. Transactions can be carried out as guests on a site, without an account, and are primarily about payment. On the other hand, there are many things which can be done with an account (adding details, resetting passwords, using loyalty points, storing up coupons and so forth) which are not directly connected to making a transaction.

The harm a fraudster can cause by attacking an account and attacking the point of transaction is different, too. Transaction is primarily a financial risk; get it wrong, and you’ll get a chargeback. Account liability is related to the trust of the customer, their loyalty to your brand, and the overall trustworthiness of your ecosystem (since the more fake accounts, or ATOed accounts, you have, the weaker and less trustworthy your ecosystem becomes).This reflects a fundamental difference in the question you’re asking in each case. With accounts, you’re asking: Is the person behind the identity the person it belongs to? With transactions, you’re really asking: Is this payment legitimate?

For all of these reasons, in many larger companies, different teams are responsible for account protection and transaction protection. There is, however, an increasing tendency to place these teams under the same leader or executive, working as part of a wider trust and safety team, or risk management team. The increased mutual learning and cooperation this encourages is very valuable, because the work on accounts and on transactions can mutually benefit each other, and we’d recommend ensuring a similar level of collaboration even if your internal structure means the teams don’t work together.

Account protection should be informed by transaction knowledge and decisioning; if an ATO was so ingenious that it slipped through until checkout, then at checkout when more information is provided you need to act at once to protect the account, as well as decline the transaction. On the other hand, your transaction protection will be far more effective if it benefits from the deep understanding of users and their identities that you get from their accounts.

As we said, for a fraudster, these approaches are not that different. The same fraudster will happily carry out ATO and checkout-focused fraud on the same site, sometimes even on the same day. It just depends on the details they have and the different ways they have found to exploit vulnerabilities. So if the ATO team identifies a fraud ring, or a repeat fraudster, the transaction team needs to know what to look for in order to catch them too. Working in silo is dangerous, because fraudsters spend a lot of time analyzing your fraud prevention efforts. Once they’ve identified a silo, they’ll leverage it.

NOTE

In general, the analysis techniques we discuss in this book are relevant to both account and transaction protection, though there may be times we emphasize one over the other, depending on which is the more dominant use case.

Between Blocking Fraud and Avoiding Friction: A Balance

This delicate and continuous balancing act is one which is reflected through every part of a fraud team’s role, and throughout every section of this book. We won’t call it out in every section, since that would get rather repetitive, but we can’t stress enough how important it is to keep it in mind.

Raphael Lawson, Group Head of Fraud at The Hut Group, put this very neatly. “Fraud prevention could be really simple, right? It’s easy to stop all fraud. Just don’t make any sales. This truism in our industry always gets a wry smile, because it’s so true. The challenge of fraud prevention is not blocking fraud; it’s blocking fraud without turning away good business. As we all know, the potential cost to the business of being risk-averse is higher than the amount generally lost to fraud. We’re always stuck between a rock and a hard place.”3

Ten years ago, fraud prevention teams were measured on chargebacks - their job was simply to stop fraud. Businesses were, by and large, not fully aware of the impact the fraud department had on customer experience and related success and growth.

Now, that’s no longer the case. Julie Fergerson, CEO of the Merchant Risk Council, explained this evolution:“When online fraud became organized, it was initially like a flood - merchants were routinely dealing with chargebacks on a scale that would be unimaginable today, crushing chargeback rates that might get as high as 3%. Merchant fraud teams and vendors worked together to bring the situation under control, and once that more stable point was reached, with chargebacks on the sort of scale we know them today, fraud fighters could take a breath and begin to focus on friction and false positives. That’s where a lot of the work and focus comes today.”4 That means that there’s pressure on fraud departments both to prevent fraud and keep chargebacks low, and to minimize the impact of their work on good customers. That generally means two key areas of concentration: avoiding overt friction such as email verification or 2FA, and avoiding false declines, where a legitimate customer is mistakenly rejected or blocked.

Ultimately, the answer to both challenges is the same: Accuracy. The more accurate you can make your system, the more precisely it will avoid irritating good customers. In that sense, the solution to this problem is the same as the one for catching fraudsters; you want to be very good at distinguishing between good and bad users.

Since perfect accuracy is more of a holy grail than a realistic goal, it’s important to be clear about other tactics you can use as well. Friction should be used dynamically, and applied judiciously, in appropriate circumstances. False declines can be avoided using forms of positive validation, to identify good customers. Where that doesn’t succeed, have steps in place to analyze the cases where a customer tries a second or third time after a decline; is it a fraudster trying to trick your customer service department, or a legitimate customer who was misunderstood? Ongoing research into account activity and transactions, separate from the decision analysis you need in the moment, can also help to identify false positives and ways to avoid them.

Tactics aside, the crucial point is that in every decision you make in your fraud team - whether it’s about whether to approve a specific transaction, or which tools to invest in, or which rules to add to deal with new situations, or how to train or retrain a machine learning system - will include balancing between the need to stop fraud, and the need to provide a great experience for good customers.

That dynamic tension is at the heart of what makes fraud prevention so challenging - and so fascinating. It’s also an opportunity. Julie Fergerson’s many years of experience working with merchants across a variety of industries and helping fraud fighters from different organizations to collaborate for their mutual benefit mean that she has gained a clear view of the evolution of how fraud and payments professionals are perceived within their companies. “Payments and fraud prevention professionals have a seat at the executive table, now. There’s a real understanding - which wasn’t there even five years ago - that these elements are crucial to creating positive customer experiences, and to the whole financial framework of the company. E-commerce and online payments and interactions aren’t a checkbox to show relevance anymore. They’re essential to a modern business.”5 Fraud prevention professionals often need to educate other parts of the organization about the intricacies of their work, but the result is better inter-departmental collaboration and better results for both sales and fraud prevention.

NOTE

Within the context of the balance between fraud prevention and friction, it makes sense to mention 3D Secure (3DS) which consumers often know best through its application by the card networks in forms such as Verified by Visa and Mastercard’s Securecode. 3DS inherently involves friction, interrupting the purchase to ask the customer to prove that they are who they say they are by providing something like a password, dynamic code, or biometric authentication. For this reason its first incarnation, 3DS1, which demanded a fairly complex password, remained unpopular except in the UK, with most non-UK merchants eschewing the option because of the high dropout rate it caused in purchases; customers (outside the UK, for some reason) simply weren’t willing to take the extra step. 3DS became far more relevant with the advent of 3DS2, which included biometric authentication options and the sort of two factor authentication consumers have become accustomed to in recent years. The PSD2 regulation in the EU also drew attention to 3DS as an option, since it is the most obvious way to fulfil the Strong Customer Authentication requirement. However, merchant adoption has been bumpy, and once again the question of balancing friction with fraud prevention has been raised and debated. The Merchant Risk Council has been actively involved in negotiations with the relevant parties to ensure that the authorities involved understand the real impact on merchants, and that merchant fraud prevention teams, in turn, are aware of what’s involved from their side. It is likely that the process of finding a process which both provides strong customer authentication and does not put undue pressure on merchants and fraud teams will be one which continues at the pace of government for a long time to come. You can follow updates on this issue on the MRC Advocacy page.

Anomaly Detection

Generally, one can argue that the bread and butter of most analysts and data scientists out there is detecting abnormal data trends. The realm of fraud analytics does not escape this reality, albeit having somewhat unique challenges. We’ll state the obvious and say that every risk/fraud team should vigilantly monitor the traffic trends of their product in order to spot and react to anomalies as soon as possible.

It is only natural (and wonderful) that there are ample resources online for visualizing and monitoring traffic trends in your system. You will probably be working with a BI dashboard or tailoring open source Python scripts to plot your data.

TIP

Pandas and Matplotlib.pyplot counts plot can be helpful, see for example Customize Dates on Time Series Plots in Python Using Matplotlib.

In a perfect world, anomalies in traffic (e.g. a spike of incoming orders to an e-commerce retailer) would quickly be identified and analyzed in order to establish if they are a result of a fraud attack or a blessed increase in sales. Sadly, our imperfect world does not usually grant us the comfort of easy-to-explain anomalies. There are two major factors which make it difficult to tie traffic anomalies to fraud attacks:

1. Most fraud attacks are only discovered, or at least confirmed, due to victim complaints (chargebacks, customer disputes, etc.). Victim complaints take their time to come, usually arriving several weeks or months after the occurrence. American Express card holders can even file fraud chargebacks years after the alleged fraudulent charge. This means that anomaly detection can never truly provide us with the full picture of a fraud attack. At best, it will merely highlight the subset of traffic where we should start our deep dive fraud investigation.

2. Despite the fact that fraud attacks can be very dramatic financially (especially for e-commerce retailers who end up paying for the cost of stolen goods), their share of the total traffic volume rarely reaches double-figure digits. It is common to hear e-commerce retailers in high-risk verticals discussing incoming fraud rates of around 2% (e.g. What Does eCommerce Fraud Really Cost Your Business?). It’s great that fraud rates don’t go wild, otherwise e-commerce wouldn’t be able to survive. However, this reality makes fraud datasets imbalanced - i.e. usually there just wouldn’t be enough fraud cases to “move the needle” and teach you that a fraud ring is emerging, until it’s way too late.

An exception to the two factors mentioned above can be found in the exquisite field of AdTech fraud analytics. AdTech fraud is almost an oasis of big data for anomaly detection based analytics, because attacks in this field tend to grow to huge scale, very quickly. For example, as shown in Figure 2-1, it is not unheard of to see attempted attacks in this field reaching even 40% of traffic volume (see Falsifying Devices: New Fraud Scheme Targets CTV).

Figure 2-1. Traffic anomaly

The traffic anomaly described in Figure 2-1 is a clear example for using counts plot for fraud detection. There could be no legitimate explanation for this explosion of traffic for CTV (connected TV impressions, i.e. how many advertisements are being viewed on smart TVs, streaming devices, etc.). Indeed, once fraud researchers took a deep dive approach into the traffic, they were able to classify 40% of the traffic measured for 10/13-10/14 as fraud.

To recap on the topic of anomaly detection as a fraud analytics method - this allegedly powerful tool should be used only as a “teaser” for triggering deep-dive investigations. If you’re working in e-commerce or banking, it’s worth taking anomalies with a grain of salt, realizing that the data needs to age in order for victim complaints to arrive, plus remembering that fraud rates are often too low to be noticeable. Having that said, anomalies can point you to where a fraud trend is emerging, or at least help you measure the impact of fraud after it’s happened.

Practical Anomaly Detection - Density Case Study

A more granular form of considering anomalies is not to look for high-level trends, but rather to spot abnormal behavior coming from specific users. The following case study assumes that for a certain bank, “normal” users would login up to once a day. The query aims to spot users who demonstrate abnormal login metrics. First, the query generates the following daily-login histogram:

UserID

LoginDate

Count_Logins

1111

January 1st

1

1111

January 2nd

2

2222

January 1st

1

2222

January 2nd

30

2222

January 3rd

1

With daily_logins_per_user_count as:

(select

userID

,LoginDate

,count(*) as count_logins

from user_logins

group by 1,2)

select

userID,

sum(case when count_logins = 1 then 1 else 0 end)/count(*) as perc_of_1_logins_in_perc,

sum(case when count_logins >1 then 1 else 0 end)/count(*) as perc_of_above_1_logins_in_perc,

sum(case when count_logins >1 then imps else 0 end)/sum(count_logins) as perc_of_above_1_logins_from_logins_in_perc,

sum(count_logins)/count(*) as avg_login_density,

sum(count_logins) total_login

from daily_logins_per_user_count

group by 1;

After producing the table of login histogram, we measure the following metrics in order to measure the level of density/velocity in user logins:

1. Rate of “normal” volume days out of total number of active days for this user

2. Rate of abnormal volume days out of total number of active days for this user

3. Rate of abnormal volume days out of sum of logins, allowing a focus on users who consistently show abnormal figures vs. users who spiked only for a short period of time

4. Average login rate - simple division of logins out of all active days

It’s quite easy to add IP/cookie/device aggregations to this standard query in order to measure different aspects of abnormality and gain further analytic insights from this basic practice. Playing with the threshold of what’s considered normal is also simple and effective, if we wish to look both for fraud related abnormalities as well as possible legit scenarios of highly engaged users.

Crises: Planning and Response

We wrote this book in 2021, during the coronavirus pandemic, so the topic of what to do in preparation for, and during, a crisis was an issue on everyone’s minds. Of course part of the nature of crises is that they tend to require you to “expect the unexpected” - if you’d known it was coming, you’d have been well-prepared, and your team likely wouldn’t have been in crisis mode.

However, there are certain lessons you can draw from the pandemic to help prepare your team, and your business, for future crises. This exercise, while outside the normal run of daily duties, is well worth engaging in at least once a year, because whatever else might vary from crisis to crisis, there is one thing you can be sure about: It will put new pressure on your fraud prevention team.

80% of Certified Fraud Examiners say fraud levels rise in times of economic distress, a trend many fraud analysts saw firsthand during the coronavirus pandemic. At the same time, the Ponemon Institute found that organizations’ ability to protect themselves against fraud fell during the pandemic. Prior to Covid-19, 45% of respondents rated their fraud prevention effectiveness as high or very high, whereas just over a year later, only 34% rated themselves that way.6 Crisis planning will help your fraud team to stay on top of things when it happens, so think of it (and explain the very real need for it to management) as similar to penetration testing or business continuity planning. If the worst happens, the impact on the business can be just as great.

Economic Stress Affects Consumers’ Situations - and Decisions

Whatever kind of crisis you’re facing, it’s likely that it places new financial stresses on some or most of your users. This inevitably impacts the landscape in which fraud prevention professionals are working.

When more people are unemployed, that widens the pool of potential mules on which fraudsters can draw. People who are frightened about remaining unemployed are less likely to look into the fine print of their job, and more likely to close their eyes to anything that ought to twig them they’re being used as part of a criminal enterprise. During the Covid 19 crisis, this was notable: criminal organizations started scaling up their muling operations, expanding into new areas and even countries. It wasn’t even restricted to drop shuserping. Mules became a way to outsource elements of social engineering, as fraudsters mined their local knowledge. It meant easy tailoring for geographies, cultures and languages. Even countries which had previously had low levels of online fraud were affected, now that mules could be found almost everywhere.

Of course, some people were responding to the situation by setting up legitimate businesses, which sometimes involved ordering items in bulk which they would repackage with other items and sell as packs - survival packs, hygiene packs, crafts activities, care packages and so on. Fraud teams needed to avoid banning this activity despite its similarity to mule operations.

Additionally, phishing attempts spiked dramatically. Consumers were online more than ever, trying out new sites, and were rushed and stressed, making them easier targets than usual for phishing schemes of all kinds. For fraud prevention, the key consequence is the abundance of stolen information newly available to fraudsters, especially for ATO attempts. Unsurprisingly, ATO jumped in response.

Each challenge requires adjustment by itself. All happening together, fraud prevention teams were faced with risk from all sides. As Alon Shemesh commented, “In a situation like this, it’s like a perfect storm. There’s the social engineering side - it’s much easier to trick people into giving their data. On top of that, there’s the technical - home infrastructure is less secure, but that’s what people were using. And then you need to think about the working experience of fraud prevention teams, who are usually used to working closely together, flagging changes and checking in with one another to verify identities, details or trends. Suddenly that practiced collaboration is much harder.”7

LONG-TERM PLANNING

This is a good place to mention the long-term planning needed in a crisis. In the case of coronavirus, ATO jumped, certainly - but not to the same extent as phishing. That’s a clue that far-sighted fraudsters were storing up details to use more slowly, at times when fraud teams would be less suspicious, and at especially busy times of the year when ATO is more likely to go unnoticed.

In a crisis, much of your resources will go into keeping track of the immediate moment and dealing with immediate challenges. But it’s equally important to track the trends which are likely to mean trouble down the road - and put measures in place to at least address them later, even if you can’t do that right away.

In the same way, if you’re struggling to adjust to a new working situation - such as where analysts are mostly working from home - finding short-term solutions is important, but analyzing what’s really putting pressure on the team and what longer term changes are needed to adapt is vital for longer term success.

Prepare for Shifts in User Behaviors

All fraud prevention teams rely on analyzing the typical behavior of users on their site, and teams which rely heavily on machine learning do so even more. How can you identify a fraudster acting like a thief, if you don’t know what normal ought to look like?

During a crisis, user behavior changes dramatically and quickly. During the coronavirus pandemic, of course, there was a sudden shift to working from home, which meant new users and addresses were in play. Different times of the day became the most popular shopping times, because parents with children at home had to work around their schedules. New delivery times were in demand, now that people were home most of the time. Ordering online and getting from a convenient pickup location was abruptly all the rage. Teenagers were shopping with their parents’ cards.

Those particular changes were specific to the situation, but similar changes may follow in any other crisis. A shift in the hot products of the day is almost certain - and fraudsters will be on top of this as it happens. It’s their business; they’ll be watching as avidly as any warehouse logistics manager.

Depending on the nature of your business and the crisis, you may experience a flood of orders, or a sudden drought. In either case, there’s a consequence for fraud prevention. Uri Arad notes that in this circumstance you need to change your focus on decline rates, chargeback rates, attack rates, and false positive rates. If your business is experiencing less traffic than usual, but fraudsters are hitting around as often as usual, then a spike in decline rate means you’re doing well.

If you have a lot of new accounts being set up, or dormant accounts coming to life, that’s positive - but there’s likely to be a higher level of fraudulent attempts mixed in among the legitimate activity than you would usually expect. Take this into account when you’re analyzing your work, and make sure your KPIs are adjusted accordingly to reflect the new position you and your business are in.

You need to be prepared for analysis that doesn’t rely on typical patterns since, as Arad put it, “in this situation, everything looks like an anomaly.” Think of tools and other trends you can analyze that aren’t affected by this crisis. If your machine learning models are struggling, be ready to retrain on the new data, and have processes in place to get through the time of shifting sands.

Regardless of what’s hitting your particular business, you may well see more new users than usual. That’s something many companies (and their fraud analysts) saw right from the start of the Covid 19 crisis, when so many consumers began shopping online far more than they had previously, including older generations who were fairly new to online shopping as a whole. Fraudsters love to hide in waves like these, knowing you’re busy and may be paying less attention than usual, so have steps in place to analyze new accounts before they become problematic.

Inter-Departmental Communication and Collaboration

Users are going to be under added stress - just like you are. That means it’s not a good time to add extra friction unnecessarily. Users who feel that a brand failed them at a time of crisis may well never return. On the other hand, with fraud on the rise, you need to step cautiously. It’s valuable to place emphasis on inter-departmental communications. It won’t feel like a priority, but when you’re caught between a rock and a hard place when it comes to fraud and friction, having the input and support of your marketing and customer support teams can be essential in finding a solution that represents the best balance for your customers - and is easy to justify to your management.

It’s vital to be transparent about your challenges, solutions and decisions. Keep records of these, as much as possible. Present them regularly (if not frequently) to management so they understand your needs and difficulties. This will mean you are well-positioned to deal with any questions or new developments, and to make a persuasive case for new resources, should you need them.

If there are external organizations you ordinarily work closely with, take into account that your relationships may be changing while you’re both under new stresses. Teams which are in regular contact with law enforcement, for example, may discover that there are fewer resources available to deal with fraud carried out against online banking when different kinds of crime are suddenly growing beyond any expectations. Prioritize issues, be clear about which things can wait, and be sensitive to the wider impact of the crisis on society and your partners.

Friendly Fraud

Some changes in user behavior aren’t so innocent.

As Karisse Hendrick said, drawing on her experience from the economic crisis of 2008/9 and the recent pandemic, “Before the 2008 financial crisis, the term “friendly fraud” was unknown in ecommerce. That changed fast once economic uncertainty started to bite. We saw first party fraud (“friendly fraud”) rise quickly as consumers sought to find ways to still receive the items they wanted, essentially without paying for them. But a lot of companies didn’t learn a lesson from that change of customer behavior for the next economic crisis - in 2020, the new face of friendly fraud was refunding fraud, and most retailers weren’t ready for that, either. It took many merchants a long time to even realize that was what was happening to them. Friendly fraud is a hugely underestimated factor in times of great economic stress. The fact is, when that situation comes, there’s likely to be a massive uptick - and merchants need to be proactive about it rather than wait to get hit. You have to be on top of the data and willing to be flexible. And you need to be very aware of all the context, from people’s living situations to the changing rules of card networks or processors.”8

Unfortunately, consumers won’t always be willing to restrain their buying habits just because their income has decreased. Friendly fraud (or you might know it as “first party fraud” if you’re from the UK) is an easy answer - either directly, by buying goods and then reporting a chargeback, or by using professional refund fraudsters. During the pandemic this trend, previously more the province of physical goods, was exacerbated by the popularity of digital goods (gift cards, bitcoin) and the ease with which these can also be charged back. The need to cancel events also encouraged chargebacks for tickets, another digital good.

For the fraud prevention team, it’s something to track carefully, but it’s also worth warning management of the likelihood of the problem before it starts. Ultimately, there’s a policy decision to be made here, and it’s not generally the province of the fraud team to make it. Flagging the issue early on, based on experience of past crises, shows your expertise.

It also means you’ll have a clear policy to work off once the trend does start to show its head, which in turn means you’ll know from the start what you need to be tracking. (For instance, number of purchases involved, price of purchases, comparison to past legitimate purchases etc.) You’ll also know whether it’s your job to combat this, or whether it will be passed on to Customer Service - in which case you’ll likely have to provide a certain amount of information when required.

PRIORITIZE NEW PROJECTS

This is something your team should start doing as a matter of course, rather than in reaction to a crisis. Something many teams learned during the coronavirus period was that new projects, even ones they were truly excited about, had to be postponed or fall by the wayside due to the onslaught of new challenges they were facing.

There were certain projects where that wasn’t possible, such as the adaptations needed to prepare for PSD2, which was about to become a legal requirement within the EU. Some UK companies had to prioritize Brexit strategies for the same reason: Whatever else was happening in the world, that deadline was still going to be there, and the business was still going to need to be ready. Other companies found themselves prioritizing other projects because they believed they were essential to keeping up with trends or issues such as data privacy or inclusion, even without a deadline to meet.

When you’re planning a new collaborative activity, a new project or a new partnership, work out where it fits on this scale. If a crisis hits and normal functioning is suspended, is this something you would instantly postpone? Prioritize no matter what? Prioritize depending on circumstances?

This will help you understand the position of the project in your company and your team, and it will also be valuable in the event of a crisis. Teams which let the PSD2 preparation slide, hoping for another extension, fell into full-fledged panic mode when it became clear that that extension would not be forthcoming. Don’t put yourself in that position; plan against it.

Summary

In this chapter we’ve looked at some of the fundamentals of fraud analysis - the things a good fraud analyst or manager needs to bear in mind all the time. These are factors which should inform or be reflected in the strategy you build for your fraud fighting team, and the tactics you use to achieve it. They’re also essential in understanding the rest of the book. Bear them in mind as we move to the next chapter, where we’ll look at building, investing in and evaluating your fraud team and the tools and systems that are right for you and your company.

1   Sift Fraud Aftershock Index

2   Sift Fraud Aftershock Index

3   https://lp.identiq.com/providerless-ebook-viewer/

4   Conversation between the authors and Julie Fergerson, April 8, 2021.

5   Conversation between the authors and Julie Fergerson, April 8, 2021.

6   Ponemon Institute, The Real Cost of Online Fraud, 2021

7   Conversation between the authors and Alon Shemesh, November 4, 2020

8   Conversation between the authors and Karisse Hendrick, November 16, 2020

Chapter 3. Fraud Prevention Evaluation and Investment

A NOTE FOR EARLY RELEASE READERS

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at ccollins@oreily.com.

The bread and butter of fraud prevention is in research identification (through fraud analysis, modeling, reverse engineering, etc.) and mitigation - and balancing that process with avoiding friction for good customers as much as possible. But there’s a whole framework of systems, tools and departmental investment that’s needed to support those efforts, and that’s what this chapter will focus on.

The more care you take to ensure that your framework is the right fit for your team’s needs and your company’s structure, the more effectively it will support your fraud prevention work.

Moreover, it’s important to position the fraud prevention department appropriately within the wider organization. Your team will work best if it has close collaborative relationships with various other departments, and you’re more likely to get the resources you need if management understands your work and what you do for the company. Even though it’s only tangentially related to the crux of the job, it’s far more important than many teams realize, and it’s as important to invest in these relationships and educational efforts as it is in analysis and research.

Types of Fraud Prevention Solutions

Fraud prevention solutions are not a one-size-fits all kind of discussion. There’s little point arguing about which the “best” fraud prevention solution or tool is. All the options have different advantages and disadvantages. The question is, what’s best for your situation and goals?

In this section, we’ll look at the main categories of solutions and tools that you can use as the technical basis for your system. Bear in mind, though, that this technical basis must be guided by the experience and expertise of fraud prevention experts, and the research and insights of fraud analysts. Fraud prevention is not a “buy off the shelf, then set and forget” kind of profession.

Rules Engines

The traditional standby of fraud prevention. The principle of how rules engines work is simple; your transactions or online activity flow through a system, which can pick out certain characteristics. You can create a rule to say that any transaction above $200 should always go to manual review, or that logins from a specific geographical area should always be reviewed, or even reject activity from one country in particular (something that would result in an unfortunate number of false positives!).

You can set rules leveraging a huge range of factors, including type of item, price of item, time zone, geographical location, address details, phone information, email details, time of day, device information, browser information and so on. There are also a wide range of consequences - you can automatically approve or reject, send to manual review, automatically require 2FA, etc.

The downside is that rules tend to be rather a blanket approach to fraud - even if you’ve experienced a lot of fraudulent transactions from Nigeria lately, do you really want to block all transactions coming from there? You should, of course, combine different rules for a more nuanced approach, though it will still have a rather broad brush stroke effect.

Rules are also entirely dependent on the efforts of you and your team. They won’t update to reflect changes in customer behavior or fraud tactics unless you’re updating them. Existing rules will remain even if they’re no longer relevant, unless you remove them - which can result in legacy rules causing confusion further down the line.

On the other hand, they’re easy to work with, take effect fast, and they give your team a sense of control. When things are changing fast, it’s valuable to have rules to work with so that your team can react quickly and decisively to swiftly moving circumstances. That’s something more than one team saw during the pandemic.

Rules engines can be built in house, which enables you to make them tightly tailored to your needs and means you are entirely in control of how your system is built and the data in it. They can also be sourced from a vendor, which will mean that they can be spun up fast and should be kept up to date with the latest technological developments for you without your having to continually invest in the system.

Machine Learning

We won’t say too much about machine learning systems here, because that’s something we’ll cover more fully in the next chapter, which deals with fraud modeling.

Machine learning systems have been in vogue in fraud prevention since around 2015, and the concept is simple - that machines can be trained to recognize transactions or activity as fraudulent or legitimate, based on past examples, and that they can then accurately predict whether a new example will turn out to be fraud or legit.

The main advantages are that these systems can adapt quickly to new fraud tricks (unlike manual reviewers, they see all the data, not just a small slice, and they don’t have to wait to confer with colleagues and then laboriously work out the best rule to add), notice patterns humans are likely to overlook, and be very nuanced in how they evaluate each instance, compared to the broad brushstroke approach of a rules engine.

The downsides include that these systems tend to have a black box element; it can be hard to know why they’re making certain decisions, or which factors they’re considering. This can be uncomfortable for teams who like to know what’s going on, and is a risk when it comes to avoiding bias. It can also make it difficult to correct the machine when it makes mistakes, and it can take time for a model to adapt to changes (like those we saw during the coronavirus pandemic, for example). Training a new model likewise takes time.

Moreover, some of the challenges machine learning faces when it comes to fraud prevention (which we look at in detail in Chapter 5) mean that in order to offset these, domain expertise is essential - which can be difficult to employ successfully with a pure machine learning model, particularly if the fraud team wants to be able to do this independently.

Hybrid Systems

Hybrid models combine machine learning with rules in some way. This can be starting out with a rules engine approach, and adding in machine learning for specific purposes such as pattern recognition (a machine can often notice patterns a human might miss) or using a machine learning system as a base, and being able to add rules to cope when things change quickly, or to reflect new research from your team.

A hybrid model has emerged as the most popular with most fraud departments in recent years, because of the potential to combine the advantages of both rules and machine learning in one system. Different companies, and different vendors, mean very different things - and reflect a different balance between rules and machine learning - when they talk about hybrid, so it’s important to clarify this in discussions, whenever that’s relevant for you.

Data Enrichment Tools

When your site sees a transaction or account activity, you have a certain set of data to work with. You’ll receive some information straight from the customer - name, email, perhaps phone or address, credit card number or other means of payment. You’ll also likely have some information you collect, such as IP address, device information, browser information, and so on.

Data enrichment tools let you enter any of those data points into their system, and receive whatever additional information they have from third party sources or simply from having seen this information before. Many of these tools can be integrated directly into your own system, making enriching the data easy and, in some cases, automated.

There are a huge number of data enrichment tools out there. Some focus on specific data points, which are the only ones your team can send them for enrichment - email, device, IP, address, behavior - while others take a more holistic approach. Some provide certain sorts of data - meaning, regardless of whether you send it email, phone or device, you’ll expect them to send you further information on an associated social profile, or credit score, or whatever their speciality is - while others provide a range of types of data in response.

These can be extremely valuable in supplementing your own information, particularly with a new customer, or a customer adding or using new information. However, not every tool will be right for you.

You need to consider the ROI - how much does this tool add to your accuracy? Many will allow you a trial period to determine this, or accept a short-term contract initially so that you can test it out. Different companies have different needs. The fact that behavioral data was absolutely essential to fighting fraud when you were working at a bank doesn’t mean that it’ll be as valuable once you’re working at an apparel retailer. You need to decide if what you’re getting is worth the price you’re paying.

Similarly, some tools are stronger in certain regions than others, and you need to explore that before you sign. One particular tool might be great for North America, but have virtually no coverage in Japan, or Brazil. Depending on your audience, that may or may not matter.

There’s also the question of freshness. Since much of this kind of data comes from third parties, notably data brokers, it can be difficult to ensure that it’s fresh. People move around, get new phone numbers, change companies, and update credit cards. Talking to others in the fraud prevention industry can be essential here - the community has knowledge to share, and is usually very willing to do so. We encourage you to leverage this as a resource.

Consortium Model

As a profession, fraud fighters are unusually willing to collaborate with one another, including across companies and industries. This is in some ways a function of the nature of the job. Other departments, such as Marketing, Sales, Finance, Product, Logistics etc. are competing with other companies and their equivalent departments in those companies. They want to get ahead of their opposite numbers, steal a march on them where possible, and often their success can spell annoyance or frustration for the equivalent departments in other organizations. In Fraud Prevention, this is not the case. Fraud fighters have a shared enemy: Fraudsters. They’re competing, not against each other, but against the common enemy.

As part of that ongoing battle, fraud fighters pool knowledge, sharing information about current trends, new fraudster techniques, and data points known to be associated with fraud. Much of this collaborative effort happens at conferences and industry events, and in regularly scheduled merchant calls. Direct data sharing sometimes happens very informally - via a group email with spreadsheets attached, for example - and sometimes indirectly, as when merchants in an industry prefer to all use the same fraud prevention vendor so that they can benefit indirectly from one another’s experiences.

Using a consortium model is a way to make the data sharing more formalized, and more direct. Various fraud prevention teams upload their fraud records to a centralized location, which all the teams can access and integrate as part of their own systems. You could think of it as a shared decline list, on a large scale.

Since fraudsters like to reuse email accounts, phone numbers, etc. across sites in order to maximize their ROI (as setting them up takes time), this can be an effective way for companies to protect their systems against data points which have already become “burnt” on other sites.

In a way buying into a consortium is like having a special kind of data enrichment, particularly targeted to what you’re looking for and want to guard against.

Using Consortium Data

Consortium data can be powerful, especially when companies within a single industry all use the same consortium, as fraudsters often specialize in particular industries. However, there are some caveats that come with this model.

Firstly, decline list data has a lag built in: You don’t discover that an email address is problematic until a chargeback comes in, which may be days, weeks or months after the address was used. As fraud fighters say, “a decline list is a great way to catch last month’s fraudsters” - potentially valuable, since one site’s fraudsters from last month may be yours today, but on the other hand potentially too late to be useful. You need to be aware of this, and not treat the consortium as a silver bullet.

Secondly, the consortium model can encourage teams to think about email addresses, phone numbers and so on as “good” or “bad” which is inaccurate and misleading. An email address is just an email address. It’s how it’s used that matters. Emails that end up compromised through ATO, for instance, are not problematic in themselves. They just went through a “bad patch” so to speak! Similarly with physical addresses; the fact that a place was used by fraudsters for a little while says nothing about the place itself. Maybe it’s an office or large apartment building with a concierge - most of those in the building are legitimate customers, and you don’t want to tar them with the fraudsters’ brush. Maybe it was a drop house for a while, but now the fraudsters have moved on and legitimate customers live there. Even a credit card which has been compromised may well still be being used by the real customer, whose orders should be accepted. And so on.

Data points are not bad. Use can be bad. Identities can be bad. It’s those which you need to watch out for, and identify. Consortium data can help you do that - as long as you don’t get confused about what it’s giving you.

There is also a third point regarding the consortium model which is more of a limitation than a caveat. Consortiums are useful for sharing decline list data - the fraud records associated with fraudulent activity. In terms of privacy considerations and legal or regulatory restrictions, this falls comfortably into the prevention of illegal activity. These same considerations, however, prevent most companies from sharing information relating to good customers in similar fashion - even were they willing to do so, which for competitive reasons most would not be. The difference between this and the more standard data enrichment model is that there, when companies share their users’ data in order to learn more in connection with it, the data is being shared with a trusted third party - the data broker, or third party provider. In a consortium, it is shared more directly with other online businesses, some of whom may be competitors.

Providerless Consortiums

An interesting alternative has been developed very recently, as part of what Gartner calls the Privacy Enhancing Computation trend.1 In this model, the consortium can pool all kinds of knowledge - regarding both good and bad users - because none of the personal user data is actually shared with other companies or any third party. For this reason, the trend is sometimes referred to as “providerless” since the third party provider is removed from the equation. The sensitive user data does not leave the possession of the company trying to verify it. This form of consortium relies on some form of Privacy Enhancing Technique, such as homomorphic encryption, multi-party computation, zero knowledge proofs and so on. There is an interesting paper from the World Economic Forum going into the details of how each of these works and giving examples of the uses in financial services, but the basic idea is not hard to grasp.

Imagine that you and a friend wanted to see if you had the same CVV number as one another. (There’s something like a 1:1000 chance that you do, so it’s by no means impossible.) You don’t want to tell each other what your number is, since you are fraud analysts and aware of how risky this would be. You could tell a trusted third party - but you really would have to trust them, and, being fraud analysts, you err on the side of caution when it comes to trust and safety. So you roll dice together and come up with a huge random number. You use a calculator to add that huge number to your CVV, resulting in an even larger number. You can now both tell that very large number to a third party, who can tell you if you have a match. The third party gets no information beyond match/no match; they cannot learn your CVV numbers, because they do not know the random number from the dice rolls. You cannot learn each other’s CVVs (unless it is a match, of course) because you don’t tell each other your final number. This is an admittedly much simplified version of the kinds of Privacy Enhancing Techniques involved which can enable companies to see whether user data they’re seeing is trusted by - or, conversely, considered fraudulent by - the other companies in the consortium.

The providerless consortium model is still new and it is not yet clear in which directions it will prove most relevant, but it is an interesting refinement on the data enrichment and consortium tools, particularly in the context of increasing data privacy regulation around the world. It also offers interesting possibilities with regard to pooling knowledge about which customers can be trusted rather than just which can’t.

Building A Res earch Analytics Team

In order to make the most of the solutions and tools you choose, you’ll need a capable research analytics team to make sure that you’re always tailoring to the specific needs of your business, and the behaviors of your customers. Even for a fairly small team, you need to start off with a couple of real domain experts - people who have been fighting fraud for some time and have a good, broad understanding of both the granular level, of what to look for when reviewing individual transactions, and the macro level, of seeing which trends have wide impact and putting that knowledge to use to protect the business. With fraud research and analytics, two is always better than one; fraud analysts benefit enormously from being able to check intuitions against each other, brainstorm, and work through challenges together.

As long as your team is guided by experienced professionals, you can recruit other team members for junior positions. Experience working with data is a plus, but statistical expertise isn’t necessary as long as they show aptitude and are willing to learn. Over time, you can train them to spot anomalies in the data your company sees and develop an intuition for when something isn’t right with a transaction. It’s a good idea to start new employees off with manual reviews, so that they build up an understanding of typical transactions and interactions with the site, as well as get to know the profile of your customers - in addition, of course, to getting a sense for the fraud attacks and fraudsters your team faces. However, it’s equally important to train them in gap analysis, and encourage them to think about what could be changed in your models to improve the system’s ability to both catch fraud and avoid friction. Fraud analysis is not rote work; you want to train analysts to look for patterns outside individual transactions, look for ways to corroborate and leverage that knowledge, and build the insights gained into your system.

In terms of team culture, encouraging creativity is as important as the more obvious virtues of data analysis and careful investigation. You want your team to think about different kinds of data sources they could use to confirm or reject hypotheses, brainstorm new ways to put existing data or tools to use, and be able to balance diverse possibilities in their minds at once. For this reason, it’s important not to insist that fraud analysts be consistently conservative. It’s true that chargebacks must be kept low, but there’s always a small amount of room for maneuver to try out new tools or techniques which could, if successful, improve your results - even if sometimes you’re unlucky and they backfire. Equally, if you consistently make analysts focus on the transactions they miss - the chargebacks they didn’t stop - they’ll become very conservative, and your approval rate will go down. (Fraud managers - you can try this experiment, if you like. Anecdotally, the results seem pretty consistent. An exclusive focus on chargebacks, for your team, is not good for a company’s sales.) It’s vital to help the team focus on preventing false positives as well, to keep the balance.

In the same way, team structure should be kept as flat as possible; stringent hierarchies limit employees’ willingness to experiment and suggest new ways of doing things. It’s also important to remind team members of the positive side of the job (helping smooth customer journeys, protecting the business from loss, solving tough problems) if the negative side of seeing so much criminal activity seems to get them down. This is most relevant in companies which are more likely to hear from victims of fraud, including banks, cryptocurrency companies, and gift card companies, but can be a challenge in other industries as well.

Within this context it’s important to mention the value of bottom-up analysis, as explained in Ohad Samet’s fantastic book Introduction to Online Payments Risk Management.2 The world of online payments has evolved considerably since the book was published, of course, but the key tenets of training and approach which he describes for fraud teams are just as relevant today as they were when the book was written. Samet lays out the importance of inductive research and reasoning, with fraud analysts being taught to sample many case studies (of transactions, logins, account creations etc. - whatever is relevant for your business) and then tease out both the legitimate and the fraudulent story for each one, matching the data that can be seen. Finding more sources to support or refuse each possibility is the natural next step. From there, fraud analysts can draw on their case study experience to suggest large scale heuristics that can be checked against the company’s database.

It’s particularly important to draw attention to this bottom-up kind of analysis because the top-down model, using a regression-based approach, is in many ways more instinctively obvious within the fraud fighting use case. Companies, after all, have so much data - so what does it tell you? What does it lead you to deduce or plan? This model is necessary, of course, and we’ll explore it further in the next chapter on Modeling. But the fact is that often, fraud happens in small volumes, and fraudsters are always trying to mimic the behavior of good customers. You need to balance out both of those challenges, and the best way to do it is by using your human resources most effectively - including their creative abilities. As Uri Arad put it, drawing on nearly a decade of fighting fraud at Paypal, “The data-based approach, with machine learning and statistics, is great at giving you the big picture. And the story-based approach, with people digging into individual cases, is great at giving you the insight into the details that we need to really understand what’s going on. When you put the two together, that’s extremely powerful.”3

Collaborating With Customer Support

Working in sync with and supporting other departments is important generally speaking in your organization, but in many cases the Fraud Prevention team have a special relationship with Customer Support, and where they don’t, it’s possible they should.

Customer Support is on the front lines of consumer interaction with your business. That also means they’re the most likely to be in direct contact with the fraudsters trying to steal from your business. Customer Support training is more likely to focus on company policy and customer enablement, ensuring customers get a good experience, than it is on identifying and blocking fraudsters. Fraud departments should ensure that this important element is covered as well, and updated regularly in line with developing fraud trends.

There are two parts to this collaboration. Firstly, fraud fighters can help Customer Support representatives to understand the tricks fraudsters are likely to play on them, from calling up to change an address after a transaction has been approved, to professional refund fraud, to account takeover attempts. Representatives who aren’t trained not to give away sensitive user information, or even company information such as which systems are used internally, may become a weak link in the security and fraud prevention chain. Hardening against attacks at Customer Support level protects the whole business from fraud and from security attacks more generally.

Secondly, if a tight feedback loop is set up, Customer Support experiences can feed into fraud teams’ knowledge of customers and trends. Companies which are not set up to make the connections in this way may go for months or even years without realizing that they’re suffering a serious refund fraud attack, for example, because the information that shows it (which may include representatives being able to recognize certain fraudsters’ voices on the phone, and the scripts they use) stays within CS, and isn’t integrated into the systems of knowledge belonging to Fraud Prevention.

Measuring Loss and Impact

Once upon a time, fraud prevention teams were measured on how low they could keep the company’s fraud chargebacks. The only relevant KPI was the number of fraud chargebacks received - usually measured by dollar value, though sometimes by percentage of transactions. There’s a compelling logic to it - these chargebacks are the most obvious cost to the business of successful fraud. The rules from the card networks support this approach as well; companies who see their chargebacks rise above 1% are, in ordinary circumstances, likely to see consequences leading to probationary terms, fines or even inability to process certain card brands.

In fact, though, measuring the company’s true loss to fraud, and the impact of the fraud fighting team, is more complex, and this is something many companies have come to realize in more recent years. This has made setting KPIs, measuring loss and measuring the fraud prevention team’s impact all more difficult - not least because part of doing this effectively involves ensuring that upper management decision makers understand fraud, fraud prevention, and the relevant context.

Companies nowadays usually don’t want to keep chargebacks to an absolute minimum. Of course it’s crucial to stay well below the chargeback thresholds set by the card companies, with a comfortable margin of error to leave flexibility for unexpected fraud rings running temporary riot or other disasters, but there’s still a wide gap between this and trying to aim for absolute zero when it comes to chargebacks. An over-focus on minimizing chargebacks implies stringent policies which are likely causing high false positives, which are, after all, another form of loss to the business - one which is widely agreed to be often larger than fraud chargebacks, sometimes by quite some margin. False positives are, unfortunately, notoriously difficult to calculate, and doing so requires continual research into declined transactions and the willingness to let some “grey area” test cases through to see whether they are fraud or not. It’s crucial that upper management understands the trade-off involved between chargebacks and false positives, and that this is part of the process of setting reasonable KPIs and measuring the impact of the team. Some education may be necessary here, and fraud prevention leaders should consider this an intrinsic part of their job. If avoiding false positives is to be a KPI for your team, it must be clear what calculation is involved here.

Regarding choosing which metrics the department should focus on, it’s important to bear in mind that you can’t set KPIs in a vacuum. Does your company value precision or speed more highly? That will impact your policy for manual reviews. What balance should you seek to strike in terms of chargebacks versus false positives? That’s intimately connected to the level of focus on customer experience, as well as the nature of your product. What level of friction is acceptable? That depends on your market and your vertical. Setting realistic targets that will match the priorities of the company as a whole requires educating upper management about how fraud and fraud prevention fits into the wider business, as well as discussions about how they see the role of your team in supporting wider priorities.

Benchmarking against industry averages is also important. The fraud rates, challenges and even chargebacks seen by a gift card marketplace will be very different to those seen by an apparel retailer. A bank would have an entirely different profile again - and neobanks versus traditional banks may well have different norms and expectations. AML is another story altogether (and has a different kind of calculation regarding mistakes, relating to the regulatory requirements involved). You can’t realistically measure your own loss unless you understand it in the context of the wider industry of which you’re a part. If you have 0.6% fraud chargebacks, in an industry which typically sees 0.3%, then you’re in a very different position to a team with 0.6% chargebacks in an industry which typically sees 0.8%-0.9%.

Unfortunately benchmarks are often difficult to assess, since much of this information is the kind companies prefer to keep private, but surveys such as the Merchant Risk Council’s Global Fraud Survey (often carried out in conjunction with CyberSource) or the Lexis Nexis True Cost of Fraud report can give you reasonable insight into metrics across different industries, though there is a limit to how granular these surveys can be. Informal discussions with fraud fighters from other companies will also give you a useful sense of where you stand. This type of information is equally important when talking to and educating upper management.

Measuring impact is tricky as well. The value of a prevented fraudulent transaction is not the only amount involved here. Here are some other factors to consider:

- The actual amount of fraud you’re seeing - your fraud rate - is lower than it would be, were you not protecting the business effectively. If your entire team went on holiday (or were less good at the job) fraudsters would discover this quickly, and the fraud rate would be a lot higher. Think about how quickly fraud rings jump on a vulnerability, once discovered - it would be like that on a larger scale and without correction. So you’re actually saving the business far more than the amount of fraud you’re stopping - there’s a lot of fraud that never comes your way, because you’re making the business a hard target to attack. This is hard to measure, but using examples of fraud rings and extrapolating can provide an illustration of the point. You can also sometimes see chatter in fraudster forums sharing news about companies who are trying to fill a number of fraud prevention positions; fraudsters like it when that happens, because they know the department is over-stretched. They’re likely to attack. These sorts of discussions also illustrate the point.

- There are additional costs associated with most transactions, particularly physical goods, including logistical efforts, cost of replacing the item, and the item’s unavailability for other customers. This sort of thing is included in what Lexis Nexis call their “multiplier” which they use to calculate how much each dollar of fraud (or prevented fraud) actually represents to the business. It’s usually at least three times the amount connected to direct loss through chargebacks.

- If your team protects against account-level fraud, such as account takeover, or fake reviews, or collusion in a marketplace, you’re protecting the business’ reputation in meaningful - and valuable - ways and undoubtedly having an impact, even if it’s one that’s hard to measure with a number. You can, however, provide numbers relating to numbers of fake reviews prevented, number of accounts protected from being hacked, etc. and it is crucial that you do present these figures to upper management. When your impact extends well beyond checkout, it’s important that this be visible. There may be related KPIs you may wish to consider relating to these sorts of metrics.

Justifying the Cost of Fraud Prevention Investment

It can be frustrating, but the reality is that you’re always going to have to justify your team’s headcount, budget and investment in new tools or technologies. Even if you know you’re doing a great job, and compare favorably to the rest of your industry, your upper management likely doesn’t know that. And even if they do, they’ll need to be able to defend that to board members, shareholders and other stakeholders.

First and foremost, you need numbers. Here are just some of the essential figures:

- Your fraud rate - the number of attacks you’re seeing as a percentage of overall transactions or activities. You may well want to break this down into types of attack.

- The number of fraudulent attempts you stop - both as a percentage of the total number of attacks, and in dollar value.

- Your chargeback rate.

- Your successful chargeback dispute rate.

- Your manual review rate - how many transactions or activities you manually review, as a percentage of total transactions or activities.

- What percentage of manually reviewed cases are approved.

- Average speed of manual review.

- If relevant, figures relating to account-level abuses such as coupon abuse, fake reviews etc. which harm the business’ bottom line and/or reputation.

What you want to do is convey how much you’re saving the business every year. (Or quarter, as the case may be.) You need your execs to see your work in the context of the bigger picture. Get them to imagine what life would be like without a fraud team - because it wouldn’t be pretty.

Once you’ve set that scene, you can tie it back to your tools and headcount. If your manual review team is working fast and furious (and accurate) then present numbers for how it would be if that team were smaller. And, if you’re angling for increasing headcount… larger. If your hybrid system enabled you to support the company’s entering a new market with low chargebacks, low friction and low false positives, then make sure credit is given to that system (and your team, for choosing it). If you want a new tool, measure what you estimate the results would be in the relevant area if you did have it.

Some of this is an annual, or biannual, or quarterly exercise. But to lay the groundwork for success, you need to make sure there’s an ongoing educational effort reaching out to the whole company, and especially to upper management. You can’t afford to let that slip.

Inter-Departmental Relations

Fraud prevention departments often operate in something of a silo within their organization. The approach and the kind of work that’s integral to the job can seem very foreign to other parts of the company (except perhaps Cybersecurity, who face similar opponents and tricks). But this distance is a problem, and fraud fighters should not underestimate the importance of interdepartmental relations.

If others in your organization don’t understand your work, what you do, or how you do it, then that’s an opportunity for education. You can run lunch and learn sessions, or Fraud 101 in company onboarding classes for new employees. Fraud is fascinating, and not just to fraud analysts - as long as you relate the topic to your audience and the company, and use some real life stories to illustrate your points effectively.

As we’ve said, much of fraud prevention involves finding the right balance (or the right compromise) between customer experience and aversion to risk. There are a lot of other departments who are affected by that decision. If you ignore them, they’ll continue to grumble and think you’re a bunch of naysayers. If you involve them in the challenges and trade-offs, and ensure over time that they understand the bigger picture from your perspective, they’ll join you in finding workable solutions. They’ll also start remembering to loop the fraud prevention department into discussions like entering a new market, or let you know in advance about a coupon program they’re rolling out, or a flash sale they’re contemplating.

We’ve heard from a lot of fraud analysts that they’re often the last to know about this sort of thing, and that sometimes they learn about it too late - when false positives (or new fraud) have already spiked. To tweak Hanlon’s Razor, never attribute to malice that which is adequately explained by ignorance. They didn’t let you know, because they didn’t realize you needed to know. And it is part of your job to make sure that they do realize that, going forward. Other departments don’t know much about fraud prevention. You need to educate them so they can understand how their work relates to yours, believe that you’re working towards the same ultimate goal as they are, and want to work together to achieve success for the company.

Data Analysis Strategy

Data analysis makes or breaks fraud prevention efforts. It’s easy to focus on the immediate analysis needs - the transactions, logins and so on flowing through the system now. There’s a vital place for that, and in many cases it’s what most members of the team will spend the most time on. But if you never look beyond that, it will take over your horizon, leaving you continually fire-fighting. Strategy is important so that you understand the structure of how your team is approaching the challenges they face - and how to improve it for better results. It’s something you need to take time for, even when things are busy, because it might not be urgent, but it’s very important.

Depending on your priorities as a department and a company, you may take a different approach to data analysis strategy. But there are two points we want to highlight which are broadly relevant. The first is that you need to build into your quarterly plans the right connections between your automation and your human expertise. These should not be run separately; you’ll get best results if each one guides the other. For example, you will get the most out of your machine learning models if you have a domain expert regularly review the features which come from it. Many of those features will be meaningful, but sometimes it will be either random, or related to a trend you understand within a relevant cultural or social context that the machine lacks. In these cases you need to have the machine either ignore the features, or modify them appropriately, if they’re going to stop fraud without adding false positives. Similarly, teams should schedule regular brainstorming sessions to explore, generate and then test complex features which are related to cases too rare or too complicated for the model to notice. Too rare is obvious - and the material for noticing such cases may come from your own randoming sampling of cases, or from collaboration with customer service teams. To give an example of complicated cases which might throw a model left unguided, take third world internet cafes. Flights booked from those are usually more likely to be fraudulent. But what if the person at the keyboard matches the persona you’ve built up of an international traveler? Then it’s actually a good sign. People are complicated. A model which is confused by receiving too many mixed signals (both good and bad) will simply balance this out as non-indicative one way or the other - but a human expert can understand the information in context, and make sure its used appropriately.

The second point we want to highlight is the importance of working with customer support teams as a part of your data analysis strategy specifically. With the right, trusting relationship, and regular contact, these teams can give you the best direction possible when you want to look for developing fraud trends. If you hear that they’ve had a spate of customers who are supposed to be eighty-year-old women whose phones are always answered by a young man, then you can feed that knowledge back into your system and flag those transactions. Work with the customer support team to agree on a list of suspicious occurrences, and the fraud indicators they hear or see. Then add a button into their system so that they can report in real time whenever something like this happens, choosing the appropriate indicator from a simple dropdown menu. The easier you make it for them to help you, the more help you’ll get - and the more data you’ll get. Your team can then look for patterns. It won’t be a huge amount of data, not the kind that you could plug into an automated model, but it will be enough - and chosen carefully enough - to be worth your fraud team’s time. A domain expert will be able to work out whether it’s just a coincidence, or the tip of a fraud iceberg. Make sure you notify the customer support team when their contributions have helped you - it’s great for building up a good relationship, makes it more likely they’ll want to help in future, and you never know, it might inspire one or two of them to become fraud analysts, in time.

Fraud Tech Strategy

Your fraud tech strategy will vary enormously depending on your budget and the resources available to you for onboarding new tools. As with data analysis strategy, there are a few broadly relevant points we want to highlight.

Firstly, your fraud tech strategy should be strategic. Don’t integrate with something just because it sounds fun and shiny and clever. It may well be all of those things, but if it’s going to do something your business doesn’t really need, then you’re just making your system more complicated for no reason. Even if your organization is willing to simply test out new tech to see what’s out there (if we’re talking about you, you’ll recognize yourself here) analyze the real weaknesses of your current situation and try to find solutions for those, rather than looking for whatever sounds most exciting. By contrast, teams who struggle to find budget or resources for new technologies shouldn’t let that limitation stop them from being equally focused on where their weaknesses lie and investing time and research in tools that could have a measurable impact on them. Even if it takes well over a year to get the tool you know you needed months ago, if it’s the right one for you, having to wait doesn’t make it less relevant once you’ve got it. And you do need to keep an eye on what’s out there so that once you have the opportunity, you can get what you need.

Secondly, make sure that when you’re designing your tech strategy you cover all the bases in your system. You need to be able to engage in rule-based tactical work, so that you can stop specific fraudsters or rings right away, or adapt on the fly to fast-changing circumstances. You may well also want to have machine learning-powered tech, in which case make sure you also plan for the maintenance that goes with it, identifying new trends and related attributes. Within this context, remain agile. For instance, if you’ve invested heavily in your machine learning team and technology, and it’s working really well, and your data science partners are helping you to solve problems you’ve been worrying about for years - that’s great. But don’t forget that you also need the ability to use rules to adapt in the short-term (as at the beginning of the coronavirus pandemic, when things changed so quickly in so many ways). It’s better to write a simple rule to tide you over until your model can be trained, than to rely entirely on your normally excellent machine learning system and be blindsided. Make sure you have a variety of tools and approaches available to you, so that you can use whichever tool is most appropriate for the task at hand. You may love your hammer, but that doesn’t mean every problem is a nail.

Thirdly, when you’re considering your tech needs, remember to think about the whole customer journey, from account creation to login to transaction or action and more. If necessary, prioritize which elements are most crucial for extra support or tooling, and address the most urgent first - but don’t forget the rest. This is just as relevant for the parts of a payment that your company doesn’t control; make sure you understand the situation with your authentication flows, and if you’re losing out there, explore solutions which can help. As usual there’s often a tradeoff in terms of user friction, with greater friction being added to reduce false positives, and that should be a part of your wider understanding of company priorities regarding customer experience. Relatedly, if you’re interested in frustrating fraudsters by forcing them to provide (and thus burn) more data through authentication processes, that may be relevant to your tech strategy too.

Data Privacy Considerations

Fraud prevention and other work designed to detect and block criminal activity are exempted from many of the data privacy considerations that constrain other industries and departments. For example Recital 47 of the EU’s GDPR notes that “The processing of personal data strictly necessary for the purposes of preventing fraud also constitutes a legitimate interest of the data controller concerned.” In a similar vein, California’s CPRA maintains that fraud prevention is an exception to the right to delete “to the extent the use of the consumer’s personal information is reasonably necessary and proportionate for those purposes.”

The lawmakers are, of course, following a compelling logic; failing to exempt fraud prevention from restrictions on sharing data would play into fraudsters’ hands by making it far harder to identify thieves at work, since each company would only be able to work with their own data, plus whatever additional information or clarification they could gain by working collaboratively with other groups using privacy enhancing computation of some form. Data enrichment tools and in many cases even third party solution providers would no longer be of use, crippling fraud detection efforts. In the same way the right to delete is reasonably mitigated by fraud prevention needs, since if fraudsters could successfully demand the deletion of their data, they would be much harder to catch on their next attempt. Regulators are not in the business of making life easier for fraudsters (or at least, certainly not intentionally).

For all these reasons, it seems likely that future data privacy legislation, including legislation governing data transfers between jurisdictions, will follow similar patterns in exempting fraud prevention from much of their demands. However, this does not mean that fraud prevention departments are unaffected. As teams preparing for GDPR will recall, the structure and searchable nature of the databases used must be amenable to right of access requests, right to delete requests when appropriate, and so on. Procedures must also be put in place to enable fraud teams to determine when right to delete requests may safely be complied with. Moreover, identity verification processes, necessary to ensure that the person requesting their data really is the person in question, may well fall to the lot of the fraud prevention team.

Beyond this, you’ll note the words “strictly necessary” and “reasonably necessary and proportionate” used in the regulations we quoted above. The interpretation of these words, and the others like them elsewhere in the regulations, is hugely important in marking out what fraud fighters can and can’t do with and about users’ data. That’s a field day for lawyers, but as part of that discussion fraud prevention teams need to be able to explain what data they use, why they need it, and what is done with it. This is also worth bearing in mind when considering new tools for data enrichment.

This is an area where it is important that fraud fighting teams work together with their company’s legal team to ensure that all is being done not only in compliance with the relevant local laws, but also in ways that are likely to be future-proofed against coming changes. A thorough audit of what data is sent to which vendors, and a shrewd analysis of how essential various relationships with data brokers are, may also be valuable. It’s important to know how your team stands with regard to data sharing and legislation, in order to put your case convincingly should it be necessary to defend your data sharing practices.

Identifying and Combating New Threats Without Undue Friction

Much of a fraud team’s day-to-day work is focused on the immediate moment: which activities or transactions are fraudulent, what patterns or fraud trends or techniques are showing up this week, how to find the balance between friction and fraud prevention for the customers you have now.

However, research into new threats remains essential. Firstly, if left undetected for long, a new trick can be enormously costly for the business. Secondly, panicked reactions to surprise threats when they are eventually discovered often result in high friction and lost business. Thirdly, remembering the importance of demonstrating expertise to upper management, it’s important to show that you are on top of developments and not learning about a new threat months after the rest of the community has begun talking about it.

Research is often all very well in theory, but can fall by the wayside under the pressures of more urgent problems. To avoid this, it’s worth choosing specific individuals whose job is to investigate certain areas of your own data a specified number of times per month or quarter, and nominate other team members to keep track of forums, articles and newsletters in the fraud fighting community. Regular weekly meetings for those who engage in manual reviews are also valuable, to enable discussion which can bring new patterns to light. Making these activities set events on the team’s calendar will prevent their accidental disappearance.

When new threats are identified, of course, it is important to remember that detection and prevention should be carried out with as little friction as possible for good users. It is always tempting in these situations to overreact and go too far on the risk-averse side of the spectrum, so establishing a procedure for dealing with new threats, which includes the consideration of friction, may be worthwhile.

Keeping Up With New Fraud Fighting Tools

Just as new fraud techniques evolve over time, vendors are continually developing new fraud fighting tools to combat them. As with new threats, it’s worth having someone, or more than one person, in your team whose job includes regular research into new tools, so that your team doesn’t miss out on the latest option which precisely matches the need you’ve recently discovered. Some particularly large fraud teams have one person dedicated to this role.

It’s important, of course, to assess each tool carefully both alone and in comparison to your existing tools and systems. How much will this new tool improve your accuracy, either in terms of increasing fraud detection or reducing friction? Always test the results against your current setup - however good a tool is, even if it comes highly recommended by trusted comrades in the industry, it might not happen to be a good fit for your needs and risk profile.

Summary

This chapter outlined aspects of the framework that underlies a successful fraud prevention team. The system you use, the structure of your team, your relationships with other departments, data privacy considerations and keeping up with new developments in both fraud and fraud prevention are all important elements of creating and running effective fraud fighting efforts in your company. The next chapter is in a sense a companion chapter to this one, exploring fraud prevention modeling options, challenges and solutions. For many teams, this will be an equally essential fraud fighting necessity - though the kind of model you use will depend on the challenges you face, and the industry in which you’re operating.

1   Top Strategic Technology Trends for 2021: Privacy-Enhancing Computation, Gartner, 12 January 2021, by analysts Ramon Krikken , Bart Willemsen https://www.gartner.com/en/documents/3995508/top-strategic-technology-trends-for-2021-privacy-enhanci

2   Ohad Samet, Introduction to Online Payments Risk Management, O’Reilly, 2013

3   Uri Arad, Fraudology podcast, https://fraudologypodcast.libsyn.com/ep31-the-holy-grail-of-online-fraud-prevention-w-uri-arad-at-identiq

About the Authors

Gilit Saporta is an experienced fraud-fighting leader with well over a decade of experience in the field. Having begun her career at Fraud Sciences, the startup famous for pioneering innovative accuracy in online fraud prevention, during the course of her career she has led and trained fraud analysts at PayPal, Forter, and Simplex. She is currently a lecturer at and a content contributor to the Tel Aviv University MBA Fintech Fraud Course, is on the steering committee of the annual CyberWeek FraudCon Conference, co-hosts Fraud Fighters’ Israeli meetups and leads the Israeli branch of RiskSalon. She is an experienced conference speaker and has contributed to many fraud, risk, and payments publications.

Shoshana Maraney is an experienced writer, with more than a decade of experience in expressing complex concepts, technologies, and techniques in comprehensible and relatable language. She has worked in the fraud prevention industry for more than five years, at first Forter and now Identiq. She is a content committee member of the Tel Aviv University CyberWeek FraudCon Conference, and has created numerous presentations for risk and payments conferences around the world. She has composed many articles for fraud, risk, and payments publications. Her degree is from Cambridge University.

1. 1. Fraudster Archetypes

1. Amateur Fraudsters

2. Mechanical Turk Fraudsters

3. Psychological Fraudster

4. Product-savvy Fraudster

5. Tech-Savvy Fraudster

1. Bot Generator

2. Hacker

6. Organized Fraud Crime

1. Distinction Between Organized Crime and Mechanical Turk Fraudsters

2. Small But Organized Crime

7. Friendly Fraudsters

8. Pop Quiz

9. Summary

2. 2. Fraud Analysis Fundamentals

1. Thinking Like A Fraudster

1. A Professional Approach to Fraud

2. Treat Categories with Caution

2. Account Versus Transaction

3. Between Blocking Fraud and Avoiding Friction: A Balance

4. Anomaly Detection

5. Practical Anomaly Detection - Density Case Study

6. Crises: Planning and Response

1. Economic Stress Affects Consumers’ Situations - and Decisions

2. Prepare for Shifts in User Behaviors

3. Inter-Departmental Communication and Collaboration

4. Friendly Fraud

7. Summary

3. 3. Fraud Prevention Evaluation and Investment

1. Types of Fraud Prevention Solutions

1. Rules Engines

2. Machine Learning

3. Hybrid Systems

4. Data Enrichment Tools

5. Consortium Model

2. Building A Res earch Analytics Team

3. Collaborating With Customer Support

4. Measuring Loss and Impact

5. Justifying the Cost of Fraud Prevention Investment

6. Inter-Departmental Relations

7. Data Analysis Strategy

8. Fraud Tech Strategy

9. Data Privacy Considerations

10. Identifying and Combating New Threats Without Undue Friction

11. Keeping Up With New Fraud Fighting Tools

12. Summary

Table of Contents

1. Fraudster Archetypes

Amateur Fraudsters

Mechanical Turk Fraudsters

Psychological Fraudster

Product-savvy Fraudster

Tech-Savvy Fraudster

Bot Generator

Hacker

Organized Fraud Crime

Distinction Between Organized Crime and Mechanical Turk Fraudsters

Small But Organized Crime

Friendly Fraudsters

Pop Quiz

Summary

2. Fraud Analysis Fundamentals

Thinking Like A Fraudster

A Professional Approach to Fraud

Treat Categories with Caution

Account Versus Transaction

Between Blocking Fraud and Avoiding Friction: A Balance

Anomaly Detection

Practical Anomaly Detection - Density Case Study

Crises: Planning and Response

Economic Stress Affects Consumers’ Situations - and Decisions

Prepare for Shifts in User Behaviors

Inter-Departmental Communication and Collaboration

Friendly Fraud

Summary

3. Fraud Prevention Evaluation and Investment

Types of Fraud Prevention Solutions

Rules Engines

Machine Learning

Hybrid Systems

Data Enrichment Tools

Consortium Model

Building A Res earch Analytics Team

Collaborating With Customer Support

Measuring Loss and Impact

Justifying the Cost of Fraud Prevention Investment

Inter-Departmental Relations

Data Analysis Strategy

Fraud Tech Strategy

Data Privacy Considerations

Identifying and Combating New Threats Without Undue Friction

Keeping Up With New Fraud Fighting Tools

Summary