Scaling Coinbase to Solve Customer Service Problem

Introduction:

Coinbase is currently operating in an exciting position: the company has clearly achieved product market fit in selling cryptocurrency as an asset class to the ordinary consumer who lacks an in-depth knowledge of the underlying technology. As CEO Brian Armstrong mentioned in his open strategy plan[1], the creation of this product market fit, complimented with an influx of institutional investment through GDAX sets the stage for the company to use cryptocurrency for its intended purpose as a secure and more efficient P2P payment network that serves Bitcoin’s original purpose. Coinbase Commerce, an open API replacing Stripe as a bitcoin e-commerce payments processor[2], sets the initial beta stage to execute this third goal. Additionally, the influx of revenues from the recent public interest in cryptocurrency sets the stage for Coinbase to invest in open source decentralization and moonshot projects that move the company from strictly a financial company to a company capable of unlocking the power of different blockchain applications in various sectors to the public.

As Armstrong mentions, the path towards creating increased revenues in existing currencies for institutional investors is a traditional path of ramped up business development partnerships and high-volume sales teams. There remains no question that Coinbase’s new index fund helps to merge the bridge between the average consumer interested in cryptocurrency and the institutional investor, a reconciliation of the primary platform and GDAX brand. However, for Coinbase to actively be able to leverage its power to use its network to move cryptocurrency from an asset class/store of value to a payment network, it must overcome a barrage of issues. While the need to scale technical talent in an industry with no precedent is a daunting task, Coinbase’s track record of hiring extraordinary employees (Linda Xie, Preethi Kasireddy, Olaf Carlson-Wee)[3] has shown that it is more than adequate in achieving the technical execution of its goal.

The Unique Coinbase Problem

However, where Coinbase has a problem it needs to solve to achieve its lofty goals of being the primary driver of a decentralized financial system: public trust. Coinbase has a three pronged problem with public perception, not all of which is the fault of the company itself. For starters, Coinbase has to handle three separate parties of discontent, far more than the average company: the average non-technical user who doesn’t understand the technical limitations and regulations of processing cryptocurrencies, the technical user who is dissatisfied with the company’s performance relative to other alternatives, and the uber-technical user who believes the company’s centralized projects fail to contribute meaningfully to the intended goal of decentralization.

Secondarily, Coinbase has to deal with various parties as a financial company in an unprecedented industry to deliver a consumer product. This includes:

IRS for high value customers to comply with tax law
the FDIC for insuring deposits for cash wallets in custodial accounts
banks for using their payment processer APIs at scale and for the ability to buy and sell securities (with further licenses pending)
the SEC to regulate and operate an asset exchange in accordance with evolving standards
the open-source Bitcoin and Ethereum communities to ensure that the decentralized systems which it operates on top of remain functional with the evolving standards (i.e how to modify the platform when Eth moves from PoW to PoS).

Third, even with high levels of funding, the unexpected influx of interest in cryptocurrency last fall meant recent demand far outstripped the ability to handle customer service requests, leading to a backlog of customer service requests from the different parties identified above. The second problem is highly unsolvable in the short-term due to the current volatile nature of the cryptocurrency market and Coinbase’s fundamental product itself. However, the customer service and addressing different party problems can be solved in a way that minimizes negative public perception.

Coinbase’s customer support problem is unprecedented due to the nature of the product and the sheer volume of different parties it has to deal with, both in the knowledge of consumers and operational parties. For example, if there is a delay in depositing Ethereum into the average consumer’s account, they could falsely think the problem is with deficiencies the Coinbase platform, when the true problem is the network proof of work transaction times being slowed down. If a technical user’s account is frozen, they might believe it’s due to a problem in transaction times and the inefficiency of Coinbase’s infrastructure when the truth might be that the government might have flagged the user as perhaps using cryptocurrencies for illicit transactions.

Creating a New Wave Customer Service Model

In an ideal world, there would be minimal need for customer service. The product of a company would never fail, and in the slight chance it did, the company could preemptively diagnose and rectify it before the consumer found out. This is obviously purely idealistic and no company in the world runs like it. However, the number one thing a customer service team can do is preemptively quell customer concerns by identifying a trend in complaints so that other customers are never aware there is a problem.

However, secondary to predicting a customer’s concern, the best customer service is customer self-service, a massive evolution for the digital age. According to ZenDesk, 67% of all customers prefer self-service over any other form of customer service, relative to only 23% of those who answered similarly in 2010[4][5]. Unfortunately, for a majority of the population which has grown up relying on telephone support, they are programmed to communicate directly with an agent if they can’t immediately get an answer. The same ZenDesk serve indicated that 47% of all customers are inclined to talk to an agent if they can’t immediately find an answer through self-service. Furthermore, an additional 40% of people still call an agent if they can’t find an answer through self-service[6]. So while consumers are open towards adopting new technologies to find an answer to their problems, they still want human interaction if they cannot solve their problem instantaneously. Machine learning powered bots are the primary driver of secondary self-service customer service, but clearly they haven’t yet accumulated the data or intelligence to answer questions instantaneously.

As AirBnb’s former head of customer service Jessica Semaan highlights[7], once past the “self-service” stage, customer service comes down to three primary drivers. As highlighted above, the primary goal of customer service is to solve a problem at a minimal cost in as few interactions as possible. The problem lies in the fact that calls and the multitude of skills needed for call center employees needed for a company at scale, especially in non-peak hours, may resolve complaints much faster but have much higher cost than email response.

The customer service model that we aim to build should:

1. Gather important user metrics to help optimize both growth and customer service

2. Attempt to prevent or at the very least communicate common problems

3. Maximize ML-assisted self-service on complaints

4. Attempt to answer remaining tickets, especially high-value ones, via email support

5. Solve all problems in less <2 communications

6. Create a last layer of analytics based person to person interactions to prevent propagation of bad reviews

Part II: Gathering the Information to Scale Coinbase’s Customer Service

In order to understand how to diagnose the degree to which a company’s customer service is positive or negative and can be fixed, it is imperative to have a measure on some key analytics tied to customer service.

Some of these key analytics are heavily tied with growth metrics, so it’s worth examining the underlying nature of all of these metrics. For example, a good methodology of prioritizing customer complaints that go beyond the initial ML assisted self-service stage is by classifying them in order of approximated LTV (Lifetime Customer Value) as opposed to going with a purely FIFO (First In, First Out) approach. However, LTV is itself tied to another bevy of essential growth metrics, such as a customer’s referral rate. Furthermore, a good example of tracking the efficiency of the customer service being provided is the customer churn rate, which should decrease over time with improving service. But the churn rate itself can be broken down into a bevy of variables independent of customer dissatisfaction such as faith in the volatile crypto market and overall macro investment conditions.

If we look at the important metrics used to measure satisfaction and growth in a startup (referencing Marc Andreessen and Dave McClure’s guides[8]), a lot of the metrics involve the establishment of product market fit and the measurement of customer acquisition. While these metrics are supremely important, for the purpose of customer satisfaction at a company that clearly has achieved product market fit, they are highly irrelevant to customer retention situations (i.e. CAC, Downloads, TAM, Virality). The main metrics that our model has to measure are metrics that can help customer service satisfy the most revenue generating customers, the most total number of customers, limit negative virality, and sustain referral rates. With that in mind, we take a look at that different metrics we need to scale customer service split into metrics that are used by startups that serve a purpose beyond customer service, macro-market specific metrics, and customer service specific metrics.

General Startup KPIs:

· LTV: Lifetime value measures the revenue projection of a single customer to a company over the course of their lifetime. As the famous Bain study shows, the top 20% of customers bring in 80% of the profitability for a company[9]. In the context of customer service, this metric is important because potential high LTV customers should be prioritized in the context of non-urgent customer service (i.e. email tickets). Almost every company has a measure of LTV, but this paper urges that Coinbase use it in a predictive custom servic emodel.

· NPS: Used to measure customer satisfaction. Gives a broad measure of overall support for the product, but given the lowest rated customers are most likely to provide bad reviews and the highest rated will increase referrals and bring a high LTV, this is an incredibly important metric, especially for Coinbase. The Net in NPS emphasizes that a company should look at the overall level of the score. However, Coinbase should also be looking at each NPS survey individually to help isolate dissatisfaction with the product from the dissatisfaction with the market + the product (churn rate), to isolate where customer support is going wrong as opposed to dissatisfaction with market conditions.

· Churn Rate: This is a really important metric of customer service, but for Coinbase in specific, it needs to be used in conjunction with other metrics to be useful. Cryptocurrency is an infant market with decreasing, but still price variance, meaning all levels of investors will make purchases based on the assumption of price changing. In particular, for the non-GDAX Coinbase platform, the Churn rate is going to vary greatly with price volatility as the casual consumer achieves great returns or losses on their investments. Therefore, churn rate for customer service specifically has to isolate: consumers leaving based on market conditions, customers dissatisfied with customer service, and customers dissatisfied with the product. This is a problem for any company whose primary product is based on external markets: the average retailers churn rate in a time of economic downfall rarely increases more than 2% the correlated downfall in GDP whereas a stock exchange’s churn rate can increase by almost 31% based on a 3% decline in market movement[10].

· Referral Rate: This is a metric necessary to calculate LTV. A customer with positive referrals on averages in more than 8x more revenue than a customer dissatisfied with a service. With an existing referral program that Coinbase has, in conjunction with an accurate NPS survey, it’s pretty easy to identify the referral rate directly.

Market/Platform Specific KPIs

These metrics are primarily used to best help isolate the percentage of churn that can be attributed to market conditions as opposed to customer dissatisfaction. For a non-subscription based service, doing so is incredibly hard, especially in the instance of a customer who might start using a cryptocurrency as a medium of exchange but move over to use as a store of value. To establish a cryptocurrency as a store of value, it might take over a year of inaction plus a direct selloff to validate this idea. In contrast, a different customer might be planning on transferring their assets and have seized buying indefinitely, which would be attributable to churn.

· Customers Liquidating Accounts: Pretty obvious, but worth mentioning, the clearest sign of churn in banking is the complete liquidation of an account, either by conversion to fiat or through the transfer to an external wallet address

· Retention Cohort Analysis:

· Currency Pair Ratio:

· Sharpe Ratio/Portfolio: The Sharpe Ratio is a way of evaluating the results of a portfolio given its risk. The basic formula for an asset is essentially given is the (Expected Value (Asset) – Risk-free Invest)/Stdrd. Deviation of the Asset. Using the annualized return of all 4 cryptocurrencies and finding the portfolio’s standard deviation, we can find the Sharpe value.

This is valuable because an especially low or high ad-hoc Sharpe Ratio portfolio for any given individual could be helpful in isolating the reason for churn, especially with a big enough sample size. For example, if a customer files a complaint handled in three interactions with Coinbase and his Sharpe Ratio is 3.5, the reason for departure is pretty clearly mediocre customer service or the finding of an alternative. Similarly, a person who files 3 complaints and has a .4 Sharpe Ratio might have a departure that is most likely related to market circumstances.

· Bollinger Bands: A very Wall Street trading metric, Bollinger Bands measures the probability for short-term volatility. This is important because in the short-term, a continuing fluctuation in volatility could mean an unstable market and more buying/selling crypto, which by association will lead to an uptake in customer service requests. Given raised volatility in one market, Coinbase can prepare for the influx of requests in the short-term by adjusting customer service staffing in a separate market.

Customer Service Specific KPIs

· Customer Effort Score: A seminal HBS studied showed that the best indicator of a loyalty after customer service is not the overall quality of the service; contrarily, it’s how hard the customer had to work to get their issue resolved. This survey offers a simple 1-10 scale asking how hard the customer had to work with customer service in order to get their issues resolved and might give the best indicator of how well customer service is doing.

· Backlog Inflow/Outflow: Backlog inflow and outflow or net backlog just measures the number of customer tickets still in the backlog and the rate at which they are coming in and out, both relative to past performance and proportionately to both the number of customer reps hired, as well as the growth of the company.

· Average Resolution Time: This metric is self-explanatory, but should also note that there shouldn’t be a single benchmark for each individual type of group. For example, resolution of identification verification with Coinbase’s ML system will take less time to resolve than a case with an account shutdown.

· Time to First Contact: As apparent, this metric just measures, on average how long it takes for a CSR to get in touch with a customer. If possible, creating a system that relates TtFC to the predicted LTV of a customer would be ideal, but is non-essential.

· Average Number of Contacts: This is an essential metric that should be a benchmark for how good the customer service is performing if preventive or self-service methods are ineffective and a ticket is filed. As previously mentioned, the hallmark of great customer service is if this number of interactions is <2:

· CSR Specific Metrics: These are the metrics used to evaluate the overall quality of Customer Service reps. An overview and more detailed approach is highlighted in Section IV (Link here).

Why To Put a Special Emphasis on Predictive LTV and Churn?

Churn is a metric that is generally important to the bottom line of any business. Studies show that across industries, a 10% increase in acquisition costs adds less than 2% to overall customer value (present value of profit streams over customer lifetime with the company), whereas a 10% increase in customer retention adds up to 30% to customer value[11].

The added bonus of finding predictive LTV for Coinbase is that it can indirectly help glean further macro insights into the cryptocurrency market. By examining a customer’s purchase frequency, order values and lifecycle, Coinbase can help answer a broad question that can help determine future market strategy: how do consumers view cryptocurrencies?

Coinbase GM Adam White’s paper on cryptocurrencies last year argued that Bitcoin specifically should be considered an asset based on the established baselines of 1) Investiability, 2) Politic-Economic, 3) Price Independence and 4) Risk Reward Profile. The latter two factors are highly specific to institutional environments and the macro bitcoin investors and have changed dramatically in light of the influx of interest in cryptocurrencies in late 2017. One of the interesting data points White uses is the broad segmentation of Coinbase (including GDAX) Bitcoin customers into three separate entities: long-term investors, transactional users and traders. White essentially defines those who don’t trade the balance purchased for a year as “investors” using Bitcoin as a long-term store of value and those who transact with an external wallet key as those using Bitcoin as a medium of exchange. As internal Coinbase data below shows, the segmentation skews slightly towards investors as opposed to a medium of exchange.

GDAX and the retail Coinbase platform will presumably be the company’s highest source of income for the short-term future. However, predictive LTV calculation and estimation per user could be useful for targeting new products in each subsection and defining more narrow meanings for these sub segments.

For example, while the Coinbase Index Fund is geared towards institutional investors, having an internal store of potential high LTV customers for the retail customers segmented as Bitcoin “investors” could be used

How To Collect the Data to Calculate These Metrics?

Coinbase is presumably limited in the amount of user-specific data it can collect for multiple reasons. First, as a company hoping to build a platform in a decentralized financial system, holding on to swaths of customer specific information beyond the purpose of serving the customer optimally is both a PR nightmare and antithetical to the company’s mission. Second, as the number of assets on the platform are limited, using some

NPS/Referral Rates

While collecting user data might seem odd for a financial services company (especially one that deals with decentralized currencies), a couple of simple metrics could help to create a much more efficient customer service operation and help to optimize growth. EY’s global banking survey indicated that “more than 70% of respondents around the world are willing to provide more data on themselves if they see tangible improvements in the suitability of products and services they are offered”[12]. As mentioned, for a company in hyper growth tied to an entirely new industry, quantifying essentially metrics (i.e. Churn, first-time user, crypto experience) should be measured to track Coinbase’s performance independent of the volatility of crypto as a whole.

One possible way to collect data is the very old-fashioned way of surveys, which still remain one of the most efficient ways of collecting user data. According to SurveyMonkey, “of the 89% who receive surveys, 67% say they complete at least half of them or more”[13]. The same study also showed that 34% of people who participate in surveys only do so if there is a direct incentive involved. Currently, the primary survey Coinbase uses emails random NPS (Net Promoter Score) surveys to customers upon Coinbase purchases.

Net Promoter Score are simple surveys where customers answer “How Likely Are You to Recommend This Product to A Friend” and helps for companies to group their customers into Detractors (0-6), Passive (7-8) and promoters (9-10). NPS is one of the primary ways of calculating the aforementioned LTV, as the estimated LTV of customers labeled “promoters” is a full 2.5x higher then detractors and detractors are almost 2.3x to leave the platform[14]. Furthermore, on tech companies at scale are likely to get almost an extra 4% referrals from those labeled promoters instead of those labeled detractors[15]. NPS, in conjunction with Customer Referral Rates and Customer Retention Rates, should help to create a pretty broad overview on respondents that can help to establish general LTV and Customer Churn levels.

Retention Cohort Analysis

Customer Experience Score

Using Machine Learning to Calculate and Store the LTV per User

An analysis of the British e-commerce site ASOS helps give a good framework on using machine learning to calculate the customer’s LTV using predictive machine learning. Of course, an online retailer and Coinbase are vastly different because of the breadth of product lines: Coinbase offers between 2-5 products to the average consumer whereas an online retailer might offer hundreds. However, the machine learning concepts they used were designed to a) minimize churn rate, b) increase customer frequency and c) increase the order size, the same goals that our data serves the purpose of. Understanding the model and adjusting it to account for the specific objectives of Coinbase could lead to an efficient way of analyzing LTV, which could be used to maximize growth, revenue, and of course customer support satisfaction.

Without getting excessively technical, the ASOS system relies on what is referred to as a Random Forest (RF) regression model with ~100 ASOS specific data categories. A Random Forest regression is a supervised learning model works by merging various decision trees. For those unfamiliar with rudimentary machine learning, a decision tree is essentially a series of true/false questions that help to narrow down broad observations to specific characteristics. Random forest regressions essentially take the estimation of a broad set of factors and finds the mean from the summation of the decision trees. Each decision tree has access to a random subset of factors and data and the factors of data categories (i.e. Days since last season, country) are referred to as “features” whereas the output identified at the end of a singular decision tree is called a “label”. In this case, the label is relatively simple, which is the net spend for a consumer (total spent – returned merchandise).

ASOS split its customer data into four specific classes: customer demographics, purchase history, returns history, and web/app session logs, finding session logs to be the most important metric by an order of magnitude. Of course, Coinbase would have way less than 100 different features due to the limited breadth of the product lines and the broader features would be different. Web/app session logs would be less valuable as investors might be compelled to check the price of their assets continuously and in accordance with the market. Purchase history might be one broad category, as would broad demographics,

ASOS also eventually replacing the RF with what is referred to as a Deep Neural Network to increase the efficiency of predicting future spend and account for the high number of “0” LTV customers, but the implementation is way beyond my (and most people’s) technical understanding.

The below process broadly shows the data pipeline of the process. All of the customer data is accessed from the data warehouse and subsequently pre-processed by category in Apache clusters. From there, they broadly are fed through the ML pipeline described above before making predictions and for ASOS, used to make specific predictions for the customer. Of course, this data could subsequently stored in some type of database or warehouse which we will refer to in usage in the context of customer service.

Quantifying the Current Nature of the Problem

Formal Complaints:

The first step of finding the number of people who have had problems with Coinbase is using the CFFB statistics to outline the number of formal complaints filed, which is public data. These complaints represent a very small portion of all customer problems as going to the CFFB is often a measure of last resort. The news site Quartz did a visualization of this recently shown below:

While CFFB complaints should not be discounted in significance, relative to Coinbase’s burgeoning user base of 10-15 million people this is a pretty minute number. The Quartz article with an inflammatory headline is obviously not great publicity for any company, but the actual complaints in themselves represent a minute problem.

Unofficial Complaints, Querying a Subset of Social Media Data

To help better quantify the cost of complaints, I queried all of the comments from the r/Coinbase Reddit and all the comments that mentioned the term Coinbase from r/Bitcoin using Google’s BigQuery data set of Reddit comments from September 2017 to December 2017[16]. The reason I used Reddit in particular was because it seems to have one of the highest densities of crypto users and the topic threads are sorted to a degree (i.e. r/Coinbase), instead of having to crawl Twitter for keywords. These comments also Furthermore, having the static comments in bigQuery instead of having to make an API call made life a lot easier as well.

The primary challenge was how to structure an SQL query that would help to isolate actual complaints from dissatisfaction of Coinbase strategy. For example, a complaint that talks about BCH insider trading is notable, but is not something that is useful in improving customer service. In contrast, a complaint talking about how $100,000 has still not reached an account with a specific case # is something that could have widespread repercussions.

.

PART III: Preventative Measures

Preemptively Identifying Customer Complaints

Part One: Internally

At a lot of startups, engineers and customer service work parallel to each other because of the extremely different nature of their jobs. While face to face interaction between the two parties is valuable, creating an intermediary visual solution to connect the two branches seems like a more time efficient solution. AirBnb’s internal ticketing dashboard has been able to “has been able to reduce their overall ticket volume by 3%” by internally monitoring customer ticket trends [17]. While total system outages and site failures will always be obvious for engineers to monitor, an internal system such as this one could potentially help to prioritize urgent transaction problems before they become vocalized. For example, if Coinbase received 15 complaints from users in the first hour that they were being overcharged by Visa, someone on the strategy or engineering teams could have alerted consumers through a push notification on the mobile app that there was a problem with the payments API instead of correcting it ad-hoc[18] a couple of days later.

AirBnb’s internal architecture helps to provide some clues of what an internal ticketing system might look like. The process for creating Airbnb’s internal ticketing system was relatively simple and could be replicated quite simply at scale with modifications depending on the preferred stack of engineers. Describing the approach from a bottom-up approach instead of in order of implementation:

1. Front-end complaint submission pushes ticket to Node.JS web app

2. Node web app processes ticket by data type, presumably to make for efficient indexing and clustering in ElasticSearch. The data type is essentially a type of issue, such as “

3. All tickets are indexed in ElasticSearch (to help compute trends in real time) and batch queried used Hive at regular intervals

4. The node web app contains two separate instances working simultaneously but independently. The aforementioned “web worker” helps to process the tickets and store them and the “job worker” which queries ElasticSearch in real-time for the necessary data to detect time trends

5. The queried data from Elasticsearch is in time series order (each data point is graphed based on the time it was received, one such example is a sine wave)

6. The time period domain data is then transformed into a frequency domain using a fast Fournier transformation (a frequency domain is almost like a bar graph; each frequency is on the x and the amplitude on the y).

7. The fast Fournier transformation is essentially just a way of deconstructing a sinusoidal wave into elements of frequency. Although not specified, the engineers at AirBnb probably used a built in third-party library for executing the transformation.

8. From the frequency domain for a pre-defined time period, the highest frequency value and a narrow range of close values are isolated and put into a real-time aggregate data set

9. A reverse inverse fast Fourier transformation (IFFT) is used to graph this aggregate set in a frequency over time graph to highlight peaks

10. Subsequently, the frequency value for a certain time is subtracted from the original graph to show how outside the ordinary this is

11. Scored data is served in a Redis cache

12. React attains data from the cache and publishes

The Basic Process Architecture Diagramed

The End Dashboard

Part Two: Externally

Coinbase already has a strong customer service response on all social networks, presumably powered by Desk’s external Twitter response tool for Twitter and active customer service members on Reddit. As previously emphasized, CRMs such as Desk and ZenDesk are incredibly efficient for ad-hoc social media problem correction but not early detection. However, by using extremely cheap third-party social media keyword monitoring and basic machine learning models using Twitter and Reddit’s APIs, Coinbase can predict customer feedback coming from these social networks and quell any chance of negative posts going viral.

Predictive analytics monitoring will be much more effective based on predicting customer behavior based solely on tweets directed at the @CoinbaseSupport account. Monitoring tweets directed at the regular @Coinbase account and containing hashtags or keywords with Coinbase is incredibly cumbersome as cryptocurrency Twitter users have the tendency often include the Coinbase keyword or hashtags in an attempt to attract other followers or to lobby for smaller currencies to be included on the platform. A simple server-side call from the Twitter API using Node loaded over 3,000,000 keyword results in just the last 30 days, and performing sentiment analysis on all of these results to farm out a minute number of complaints seems cumbersome and costly.

An excellent predictive social media customer service model that would be extremely useful for Coinbase is based on the real-life data and subsequent Monte Carle simulation of Singapore academics Agus Sulisyta and Abishek Sharma[19]. These researchers were given access to the data, social media accounts, and social media monitoring tools of Indonesian Telecom giant Telkom, which is essentially the Comcast of Indonesia and has thousands of complaints directed at their Twitter support handle annually.

The primary purpose of the experiment was to use Twitter’s API and the first tweet from a specific customer about their problem to determine whether the customer’s problem had a chance to become widespread and cause harmful media attention for the company using a simple Random Forrest ML model. Although this experiment specifically focused on Twitter, the model could easily be applied to different platforms (i.e. Yelp or Reddit, where Coinbase has a way bigger problem).

In order to build any supervised machine learning model, it’s necessary to leverage historical data. In this case, the researchers used a cheap tool called Brand24 ($19.99 a month) to collect all the Tweets related to the company while filtering out the irrelevant posts. Altogether, the researchers found 11,800 different usable posts, which they put into a social media data dashboard called BrandFibres. This is where the limited amount of manual work came it; the analysts had to manually classify the sentiment of the tweet on a scale from -5 to 5 and put each of the tweets into one of the following clusters[20]:

Next, using a Tweepy, a very basic Twitter Python library, the individual accounts for each tweet were isolated, yielding 6,031 different accounts for those 11,800 tweets. Each user was then what is referred to in ML as clustering using Expectation Maximization algorithm (which the authors do not expand on) based on the total number of feedback tweets, the number of positive tweets from the user, and the number of negative tweets from the user.

Based on this data, the researchers further classified the data in accordance with the categories highlighted in the table. The researchers created three separate categories: kind tweets (categories 0, 1 -> 23 %), single bad tweets (categories 4 -> 47%) and tweets with damaging potential/customer loss (categories 2,3 -> 30%).

After these classifications, the researchers constructed their model using the following process, diagramed below:

Optimizing Self-Service

The current infrastructure for Coinbase is similar to many other Silicon Valley companies using third party CRMs: a help center with articles explaining frequently asked questions and displaying a telephone number. If the customer, is unable to sufficiently answer the question by themselves, they can interact with the predictive search window or the ADA customer service bot, which helps to redirect the user to the appropriate article or in certain instances, give a user the answer to their question without having to leave the chat window.

Like other customer service ML bots, the effectiveness of the experience is based on the strength of the bot’s ability to comprehend the language of the user (NLP) and of course, the syntax used by the customer themselves. Bots, and machine learning models in general, are most effective when decisions are pre-classified for them before they have to process information. In fact,

Coinbase has an advantage in regards to customer service that many other large tech companies lack: they employ a single vendor model. With the multi-vendor model, there is enormous amounts of variability in the possible situations that might arise. A seller could have shipped a box wrong, attached the wrong license to their software, had a sick cousin, etc. which prevented them from delivering a product on time. With a single vendor model, it is quite easy to determine what might have gone wrong given details about the transaction or item being sold.

Instead of asking customers what their specific problem is, if the user is already logged in we should retrieve the list of transactions they’ve recently had. By having the user pre-select the transaction, for which we saved associated keywords at the beginning of the process, Coinbase drastically reduce the response time and effectiveness of the ADA Bot. In addition, the classifiers associated with the transaction can be used . The single vendor transaction support model is not unprecedented. Apple, another single seller vendor, uses your iCloud device history to narrow down the list of possible topics that could require support in their seller app.

Coinbase already has a user’s transaction history for each individual currency on both its desktop and mobile interfaces so listing it is simply a matter of calling each API endpoint and aggregating it instead of sorting it by currency. All of this transaction history and support should theoretically be native to the app,

Of course, the biggest technical problem involved with this is how to prepopulate the text of a third-party bot from the server-side if the issue is not instantaneously solved.

PART IV: Responding to Customer Complaints (Non-Self Service)

If a user is unable to solve their problems using self-service, the next step involves customer representatives having to answer tickets or phone calls from customers. While machine learning has largely improved the customer service experience, many cases inevitably end up at this point. There are many questions for a company to think about at this point and the best way to efficiently handle customer tickets in as few interactions as possible. Of course, this starts with how to get and hire the customer service reps (CSRs) needed to create a feasible customer service response at scale.

What Should the Fundamental Response Strategy Be?

There are essentially three options when addressing how to structure a customer service operation. The first is to separately segment digital channels (i.e. chat, email, tickets, response) from a telephone only base (what we traditionally refer to as call centers). The second is to allocate a full-fledged “contact center”, where CSRs are trained as “multichannel” employees, meaning they will be responsible for all channels of contact. The third is a hybrid model, with a full-fledge contact center during peak hours and then relying on an external “telephone” only call center during non-peak hours or for harder markets to rely upon. As with any decision, there are pros and cons for each of these options. Before exploring some quantitative academic studies and unit economics of maintaining each of the facilities, we lay out the pros and cons of each of these options:

Separate Channel:

Pros:

· Phone is much more expensive than email customer support. With a separate call center, you can outsource all calls to a country such as India and save a ton of money immediately on per call basis

· Call center outsourced will likely already be set up saving a ton in upfront fixed costs (training, equipment, etc.)

· Allows local CSRs to focus exclusively on digital channels without interference

Cons:

· Communication gap between dissatisfied customers who try to access both channels (i.e. sends 3 unresolved tickets than calls into a rep)

· CSRs are less likely to be exposed to the full knowledge of Coinbase problems

· Hard to centralize unstructured data that can be converted and analyzed to improve future interactions, especially from voice interactions

· Could be problematic in times of asymmetric demand

· High employee turnover

Full Contact Center:

Pros:

· CSRs fluent talking to customers through all channels about all issues

· Different tasks lead to higher stimulation, which is proven to lead to higher productivity

· Centralized data which can be analyzed to create much lower future variable costs

· Overall, likely to have highest levels of customer satisfaction

Cons:

· Employees require well-rounded skillsets, will cost more, especially if they produce competently and expect higher compensation

· Have to also pay high rates for CSRs in non-peak hours

· Might have tradeoffs between handling communication from different channels

· Requires high upfront costs to build a center and higher high-competence employees

Hybrid Model:

Pros:

· Most balanced financial model: Minimizing costs on non-peak hours, maximizing productivity during peak hours

Cons:

· Could be hugely problematic in times of asymmetric demand

· Crypto market doesn’t really ever stop a la Wall Street so if expansion into Asian markets happens, could be problematic

· Same problem for fully centralizing data; lesser so than the separate

· Some upfront costs in building non-peak contact center

Within a full-contact center, how should I structure how my employees work?

In order to answer, this question, it’s important to examine queuing theory and its implications on customer service, a model that has been studied extensively in academic papers. A lot of the literature was based on single-channel call centers and analyzing how best to route their calls. However, a lot of these theories are still prevalent today and can be applied to full-fledged “contact centers”. For CSRs to be “multichannel” employees, they must prioritize calls as these are the matters of highest urgency.

In call centers, queuing theory has two distinct extreme situations of distributing calls: full flexibility and full dedicated. Full flexibility is when all members of a call center are full generalists: they are “cross trained for any problems” and can solve all problems. The basic pro of full flexible agents is that there are much fewer of them required to staff a center and their proficiency increases with economics of scale. The downside is that there are very few of these agents that have the skillset to manage a diverse range of complex issues, so they are extremely costly.

On the complete opposite of the spectrum is the fully dedicated model. In this model, each representative can only have a single skill. Of course, this is an extremely inefficient model, as growing companies often have various complex problems and staffing each of them individually for a single problem is unrealistic.

The prevalent model for the last twenty years has been “chaining” proposed by X and Y in the mid 90s. In this model, each call type can be assigned to one of two adjacent agent teams, and each agent can handle calls from two adjacent types as demonstrated below where lambda is the inflow of calls and s represents a single agent.[21]. Most modern online routing tools and call centers employ some variation of this model. However, chaining is severely limited in times of asymmetric demand, which is pretty much exactly what a cryptocurrency is probably going to be dealing with. Why is chaining bad in asymmetric?

For a company with severely asymmetric demand such as Coinbase, dealing with customer tickets in a way that is with a traditional customer service model such as chaining or having full generalists is essentially a terrible idea. This isn’t a bad indictment on Coinbase; it’s also something that companies with high asymmetry markets still struggle with. The biggest comparison to Coinbase would be a stock exchange, and an average stock exchange has a NPS of 13 on a 1-100 scale[22], which means they are leaving a ton of business on the table.

There are a lot of problems laid out for a company with high asymmetric demand, but what are the possible solutions? A comprehensive thesis by French student Benjamin Legros proposes one of the first studies of how to deal with asymmetric demand companies using a method called single pooling. The Monte-Carlo simulation based on varying traffic times at a scalable company, the details of which will be presented more extensively below, performed significantly better than the current customer service method[23].

Single pooling describes a model where each agent describes where each agent has a particular are or issue they are competent in. In addition, each agent is comfortable answering what we will refer to as “easy issues” or what Legros refers to as 0 issues. In the model, when a particular issue that fits the CSRs skillset is on the line, the issue is routed to the agent of expertise or added to the CSR’s queue. The 0 issues are kept in queue and routed to any agent who doesn’t have any current issues to deal with. In Legros’s model

Taking a look at how a single pooling model might work for Coinbase, we have to classify the issues into separate big picture issues that we could group individual reps on. Taking a better look of the results of our Reddit data, the five groups we could classify the model on are:

· Inaccurate Fraud Detection

· Frozen Accounts

· Identification Problems

· Money not received by banks

How Should Coinbase Hire Effectively and Integrate CSRs at Scale?

Harvard Business School performed an extensive customer service study which classified a sample size of over 10,000 reps into seven particular categories based on personality traits:

1. Controller: “Outspoken and opinionate, likes demonstrating expertise and directing the customer interaction”

2. Rock: “Unflappable and optimistic; doesn’t take difficult conversations personally”

3. Accomodater: “Meets people halfway, involves others in decision making; eagerly offers discounts and refunds”

4. Emapthizer: “Enjoys solving others’ problems; seeks to understand behaviors and motives; listens sympathetically”

5. Hard-worker: “Follows rules and procedures, likes working with numbers; is persistant and deadline-oriented”

6. Innovator: “Identifies ways to improve procedures and processes; generates new ideas and opinions”

7. Competitor: “Focuses on winning, outperforming colleagues and changing others’ views”

While managers might be naturally inclined to pick those who are “empathizers”, controllers offer by far the most value to any company. The key distinction between controllers and the other class of CSRs are that once controllers find the primary driver of the customer’s problem, they make sure to take control of the situation. While many might presume that there is a deep psychological reason that controllers perform significantly better than other CSRs, the study highlights the reason is quite simple: 84% of customers want a simple solution instead of excessive choices and controllers tend to eliminate variability by making a decision and telling the customer what to do instead of guiding them through a treacherous process, even if it involves going off-script from upper-management.

So how can Coinbase recruit “controllers”? The same HBS study looked at a candidate pool of 1000 job candidates looking for customer service studies and came to the conclusion that the syntax of the job posting made to fit the psyche of controllers. Using syntax such as “owning customer issues from start to finish” and “self-starters comfortable taking initiative” led to an influx of interest from Controllers as opposed to phrases with generic value propositions such as “serving the customer” or “exciting career opportunities”. Furthermore, candidates with a lack of customer service experience shouldn’t be viewed as an impediment. Candidates who have prior experience are less likely to be controllers and CSRs answering to postings that reflect a diverse background of high organization controllers i.e. “you love organizing road trips” will likely yield a higher pool of quality candidates.

Even while maximizing the number of controllers, there might not be sufficient amounts in a given geographic area to fit Coinbase’s needs. Training employees to fit the mold of a “controller” can pay dividends as well by applying leadership and listening techniques mimicking controllers. The study showed that work-day integrated training in short bursts as opposed to a single day or weekend off-premise lectures yielded a 17% increase in performance, almost 12% over average for Australian bank Macquarie. For reference, integrated training involves managers pointing out examples or dropping mini-seminars during the workflow of an ordinary day.

Finally, CSRs should have an open channel to access to hire level management, offering suggestions for product and customer experience changes based on their interactions with customers. Fidelity, which serves a good foil to Coinbase based on both product-type and scale, implemented an open forum allow CSRs to communicate with high-level management. In the process, “3,000 comments, including 350 ideas that management considered worthy of further evaluation. For example, reps identified a website timeout issue that was frustrating customers and leading to increased calls—a problem that was rapidly fixed once it came to light. More than 100 improvement ideas have since been approved by senior management, helping the organization to save more than $4 million” (HBS).

One possible radically alternative way of sorting for qualified candidates that can deal with the variety of issues Coinbase is dealing with is to partner with Coursera or some other MOOC to pretrain candidates using different level. There are really no statistics which would back the efficiency of such a program, but it is not without precedent. Google partnered with Coursera to try to pre-train their increasing demand for Cloud IT technicians at AT&T did the same for their Sales program applicants. However, the results of these programs aren’t public, which makes it hard to measure their effectiveness. There are studies that measure the employability of MOOC learners, but the key basis of all these studies is the psychology of a candidate that goes out of their way to learn a skill with no immediate incentive such as employment.

Without offering too much proprietary information, Coinbase could offer practice exercises which mimic the real-life transactions that a customer service rep might encounter in separate areas such as fraud, account management, etc. Having reps pre-trained, along with open access to the course, would solve multiple problems. Open access would allow for more potential candidates to apply and in the process, attract controllers who take the initiative to complete the course and eliminate candidates who lack the desire to clear a high-latency barrier to entry. Secondly, this program would make waves, especially in the tech community, which would help to reduce the flawed perception that Coinbase doesn’t care about customer service and bring about positive publicity.

How Should Coinbase Evaluate and Train or Replace Reps?

As customer self-service increases, so does the cost of live calls and support tickets. This might seem counter-intuitive at first, but as the simple tasks are being eliminated by self-service, live interactions are of an increasingly high complexity. As a result, their has to be unanimously

All of these metrics should be compared to other reps working on similar cases to standardize the possible variances from the types of cases and customers that the reps are working on. -++

· Open cases

· Time to resolution:

· Resolutions: Total number of resolutions that the CSR has achieved over the course of some standardized level of time (i.e. per hour, per quarter).

· First-Response Time:

Studies show that there is an obviously a very tangible benefit to retaining CSRs as opposed to having to find low-cost replacements, because of the costs associated with retraining levels, hiring costs, etc. Of course, this begs the question of what the CSRs typical career trajectory looks like and what the breakeven point between salary a CSRs replacement costs are.

How Should Coinbase Optimize Customer Service for Short and Long-term Market Fluctuations?

Having a platform tied to a specific asset has its pros and cons: increased fluctuations in prices bring positive attention to industry and thus the platform through trickle-down effects. However, the downside of such a situation is that peaks and spikes in transaction volume and customer growth make resource allocation highly unpredictable, which customer service is not immune to.

Creating a Customer Service Model that Optimizes Revenue:

As previously mentioned, a radical concept in customer service is to create a program that can calculate the potential LTV of a customer based on segmentation and subsequently ranks the customers in order of importance using some internal CRM. It was previously discussed how this might be useful from a growth aspect, but how does this factor into customer service? Obviously, for phone calls, creating an efficient system that cross-references a phone number with a customer’s LTV in real time is a nightmare and FIFO is a much more practical solution. However, for answering customer service tickets, in the case of a large queue how is it possible to make this system a reality to integrate with the existing Desk.com CRM? As previously mentioned in the general vs specific section, the ordinary ticket workflow process looks something like this:

Now for something where LTV is a factor in deciding which tickets are resolved the process probably looks something like this:

A lot of the steps are the same as the original process or are relatively elementary. The user login step is easy enough; a popup window asking the user to login or exit out pops up when the user tries to submit the contact form file, as seen below. From mobile, where the user is required to be logged into use the app, this aspect is even easier.

The much harder process is creating an external system that can help to process the customer information, compare it and then send a notification. Creating such a system from scratch seems like a huge engineering challenge, so a better idea might be to take advantage of Desk.com’s existing APIs and integrations to optimize the situation.

Should technical engineers help on support teams?

There’s two ways to look at this. Quantitatively, a comparison of the cost-value of an engineer’s time contributing three hours every two weeks to customer service and the upgrade the provide by instantaneously fixing customer’s problems vs. the opportunity cost of this time that could be spent on an external project would be ideal. However, running the unit economics of such as a scenario is pretty pointless as it relies on a series of key assumptions. As Paul Graham highlights in Hackers and Painters, there biggest struggle of relatively big companies is that they have no definitive way to quantify the production of an engineer past a certain scale because it’s impossible to isolate a single engineer work from the production of a team product[24]. Anecdotally, some teams have started to implement engineers on support teams so that systems can be resolved really fast, but of course the bigger platform related problems are immediately identified. As previously mentioned, having a channel of communication between customer support and engineering is probably more essentially and having some level of predictive technology definitely accelerates this process. Coinbase has recently had some engineers commenting on Reddit about resolving the platforms problems, and this seems like a sustainable solution.

PART V: Optimizing Support Response Using Data Collected During Interactions.

So, after a customer interaction is over, how is it possible to analyze the massive amounts of data to better optimize customer service for the next call or ticket? There are a couple of different ways of measuring the effectiveness of customer service, both in-call or via email support. However, the analysis of these channels highlights the inherit problem of why many companies fall short at customer service. The primary problem is that most data coming through contact centers is semi-structured or unstructured data (almost 80%, including emails, phone calls, etc.), which a lot of tools can’t understand.[25] Furthermore, there is a conversion consistency problem, as voice data must first be transcribed and converted to unstructured data whereas semi-structured data like answers to a prompt is a faster, yet different conversion.

Fundamentally, this is what a chatbot such as ADA is doing, converting swaths of unstructured data to structured data and adjusting its decision tree. However, providing a solution to this in a cross-channel platform is a completely different challenge. With a chatbot, the bot is dealing with a single input and does not have to associate the prompt answer to a specific customer. However, dealing with a customer who has sent two emails and then calls and is then dissatisfied, how can a company centralized all of those forms of unstructured data and assign it to a certain customer?

The reality is that Coinbase doesn’t have and shouldn’t devote the resources to solving such a large-scale macro problem. The long-term solution should be to rely on 3rd party services to analyze the content of interactions and find some way to centralize the insights into a relational database using the APIs of these third parties.

Another reason that Coinbase should rely on third party providers is that the aggregation of unstructured data being processed in real-time is irrelevant to Coinbase’s core competency. As currently structured, Coinbase’s platform is binary, either the consumer continues to buy cryptocurrency or they don’t in whatever form, through index funds, GDAX, etc. Whereas companies such as Amazon or Facebook need the dissatisfactory data from customers to adjust other parts of their systems (Alexa, NewsFeed, seller reputation), Coinbase does not.

Voice

As Saberi, Hussain, and Chang lay out in their review of contact centers, the premier companies use four methods of telecommunications monitoring, emotion detection, automated CSAT, problematic call detection, and call segmentation. Developments in telecommunications since 2012 have essentially rendered problematic call detection obsolete, so we should focus on the other three.

1. Emotion Detection – This concept is as simple as sounds. It involves using monitoring tools in real-time to detect dissatisfaction so that a QA analyst can step in and change the direction or methodology of the call in real-time. The best platform for transcribing these calls, gleaning these insights, and aggregating them is Cogito. Cogito is backed by twenty years of research, has a strong team and funding, and can deliver these insights effortlessly.

2. CSAT Analysis – An automated analysis which tries to isolate the reason for customer dissatisfaction. This part of voice analysis is much harder than emotion detection and there are a bunch of machine learning based startups that are trying to solve this problem in lieu of customer surveys at the end of calls.

3. Call Segmentation –Segmentation involves broadly organizing customer calls into different patterns. There are two primary benefits of doing so: CSRs can be trained on the response to certain scenarios that have minimal variation and common patterns can be used to automate certain portions of the customer service experience. Customer service automation is a great idea for companies that have sterling reputations of resolving issues (i.e. Apple) but for a scaling startup that already has some PR problems, caller menu options is not a great idea even if it makes the routing process more efficient.

Text:

1. The ideal solution for analyzing the text data from customer support tickets would be an integration that ties in with the existing data pipeline of tickets (Desk.com). Luckily, an analysis tool called Scope AI does just that, using data in real time to gather insights and sentiment analysis directly through Desk or any other customer ticket CRM. Being a relatively new startup, there is some risk that the company might not be a long-term solution so it’s worth working with Scope to backup the ticket analysis in an external pipeline as well. Some features from Scope AI are real time analysis, automatic tagging of topics, trend display and most importantly, customer insights from a subset of tickets or the user’s transaction history.

Data Aggregation

[1] https://medium.com/@barmstrong/what-is-coinbases-strategy-1c5413f6e09d

[2]

[3] David Sacks,

[4] https://www.zendesk.com/resources/searching-for-self-service/

[5] https://fonolo.com/blog/2012/05/customer-service-statistics/

[6] https://www.zendesk.com/resources/searching-for-self-service/

[14] https://www.getcloudcherry.com/blog/net-promoter-score-banking-and-finance/

[15]

[16]

[17]

[18] https://techcrunch.com/2018/02/16/visa-coinbase-not-at-fault/

[23][23]