Do Self-Service Systems Really Lead to Better Results? Our Member Survey Offers Surprising Answers to Industry Questions

The CDP Institute just published its annual Member Survey, which is always a treasure chest of interesting data. I’ve already published my primary analysis on the Institute site (you can download it here) but wanted to call out a number of findings that either contradict or confirm martech industry conventional wisdom. After all, nothing’s more fun than tweaking the nose of authority.

Data unification is growing: false. Most of us have a deep-rooted confidence in progress, at least when it comes to technology (human nature is another matter). The need for unified customer data has now been so widely understood for so long that we just naturally assume that more companies will have developed it. That has been true since the survey began in 2017 through last year’s survey: both CDP deployment and presence of a unified customer database have increased steadily. But both measures fell in the current survey. It’s hard to imagine companies have actually abandoned their unified databases, but even if growth has only slowed, that would be a big surprise.

Budget pressures are slowing industry growth: true. Some industry vendors report business is booming, but most will admit buyers are taking longer to make decisions. Sure enough, the fraction of vendors who reported growth in CDP investment is down from last year’s survey, both for the past year and the current year. 

Budgets may not be the only reason for slower growth, but the budget pressures rose more quickly than any other obstacle (from 20% to 29%). Cooperation, which can also be a symptom of budget pressures, grew the second-fastest (36% to 43%). Budget-pressed buyers are making smarter decisions: false. I can’t point to anyone who has made this exact claim, but think it’s implicit in reports that companies have more martech than they need and hope to simplify their stacks in the future. 

The survey does show that budget pressures are changing martech selection methods: more are selecting on cost (up from 43% to 54% for operating cost and 42% to 51% for initial cost), while selection on feature sophistication and breadth have fallen the most (26% to 15% and 41% to 24%).

Unfortunately, our surveys have consistently found that selection on cost correlates with low satisfaction with martech investments, while selection on features correlates with high satisfaction. So it seems that a short-term focus on cost is likely to cause long-term problems with martech results. Martech departments are growing: true. The fraction of survey respondents who said they work in a martech department has more than doubled since the last survey, which was the first that listed martech as an option. It’s unlikely that the number of people working in martech has actually doubled in the past year, but it does seem reasonable to believe that some meaningful fraction of employees have been moved from marketing or IT into a dedicated martech department. IT staff is playing a larger role in martech decisions: true. Despite the growth in respondents who work in martech departments, the current survey showed a small increase in the fraction of respondents who reported that that corporate IT manages their marketing technology (28% to 30%) and sharp declines in the fraction reporting that a martech team was in charge (43% to 32%) or each department runs its own martech (27% to 18%). This may be another cost-saving measure or it may reflect the growing importance attached to customer data (and martech in general) throughout the enterprise.

The bad news is that IT responsibility also correlates with lower martech satisfaction. Bear in mind that survey respondents are themselves mostly martech and marketing people, who are generally happiest when their own team is in charge. IT people probably give a different answer but are a small fraction of the survey respondents.

CDP projects are easy: false. Past surveys have found that about 60% of deployed CDPs are reported as delivering value, while the remaining 40% are struggling. I have always suspected most of the 40% are new projects that will deliver value eventually. The success rate is much higher in the current survey (80%), possibly because the slowdown in deployment has meant there are fewer new projects. But I’d want to see similar results in one or two other surveys before accepting that as a trend.

A separate question, asking vendors about success rates, finds the fraction reporting that say almost all or the majority of projects are successful has grown from 48% to 54%. While this might suggest some improvement in success rates, the more important message is that a bare majority of vendors say most CDP projects succeed.  Even when answers from CDP vendors are tabulated separately, just 68% say nearly all or a majority of projects are successful. Figures are lower for service providers (45%) and other respondents (15%). 

It’s important to realize this is no worse than success rates for other large system deployments.  In fact, it’s apparently better than average, since most studies put failure rates at 60% to 70%.  (See this page for a compendium.)  But these findings should dispel any notion that CDP deployments are easy. 

CDP projects fail because of technical complexity: false. It’s also important to recognize that CDP failure rates are not due to any inherent problem with CDP technology.   As in previous surveys, by far the top reason for CDP project failure is organization. This has grown even more prominent in the current survey, while problems with poor requirements and CDP performance have fallen.Privacy is growing more important: true. Last year’s survey showed a distressing decline in the priority given to data privacy regulations, with the share of firms making little effort to comply growing from 12% to 20%. That trend has now reversed, with the share of companies using privacy as a selling point increasing from 21% to 27%. Privacy was also listed in the CDP benefits question for the first time.  It was cited by 22% of respondents, ranking seventh of eleven items, and shows a below-average satisfaction score. This may indicate that most CDP users are looking elsewhere for their primary privacy management solution. Self-service leads to success: false. The martech management section of this year’s survey added a new question about whether companies seek systems that empower business users to execute tasks without technical assistance. The concept of “no-code” is directly relevant here, although we didn’t use the term.  This capability was by far the most common management technique, which wasn’t surprising: “empowering end-users” is a popular goal that saves money and makes users more effective.   But it also correlated with a low satisfaction score, which was surprising indeed. Is it possible that self-service doesn’t save money or make users more effective after all? Answers to another survey question shed a bit more light. Among the options listed for CDP capabilities were self-service data extracts and self-service predictive models. Self service extracts were a common requirement (41%) correlated with a roughly average satisfaction rating, while self-service models were an uncommon requirement (8%) correlated with a very low satisfaction rating. My interpretation is that self-service extracts are a simple task that’s well understood by business users, so companies with well-run martech operations provide that capability.  By contrast, self-service predictive models are a complicated task that few users are equipped to handle on their own, so they are prioritized largely by companies with a poor understanding of what makes for martech success.  The larger message is that self-service should be deployed only when users are ready for it – and pushing beyond those limits can cause problems despite the apparent savings in cost and time.Real-time processing is a high priority for most users: mixed. Real-time processing is commonly cited as an important CDP capability. In fact, the CDP Institute’s RealCDP requirements include real-time access to profiles and real-time event triggers. Again looking at the capabilities question, we see that real-time profiles are indeed a common requirement (42%), while real-time recommendations are much less common (15%) but correlate with very high satisfaction. The lesson here is that there are different types of real-time processing, and users consider some more important than others. Discussions about real-time should include similar nuance.

CDP must load data from all sources and retain full details: mixed. Both of these items are on the RealCDP requirements list. But loading data from all sources is the highest ranked capability (78%) and correlates with roughly average satisfaction, while loading full detail is cited by just 14% of respondents and correlates with much lower satisfaction. As with real-time and self-service, I take these answers to show that users have a mature understanding of what they do and don’t need from the CDP.  Loading all data sources is a core CDP promise and goes back to the problem CDP was originally designed to solve: getting access to all customer data. Storing full detail is not needed in most situations.  In practice, users choose which data elements are worth the cost. That said, I still believe that users want the option to store any particular details they need.

CDP users want to access their data warehouse directly: false. This year’s survey added a capabilities question about reading data from an external source without loading it into the CDP. This is a hot topic in the industry, both as a way of supplementing a traditional CDP (which maintains its own data store) and as a way of building a CDP-equivalent system that relies only on data in the enterprise data warehouse. Only a small fraction of respondents (14%) were interested in this and that group reported exceptionally low satisfaction levels. I interpret this to mean that, despite all the marketplace noise, there is actually very little interest among knowledgeable users in building a CDP that relies primarily on external data.

Summary

This report contains more bad news about industry growth and budget pressures than I like to see, but the tech sector’s troubles are well known and it’s not surprising that CDP projects should suffer with everyone else. I’m especially reluctant to highlight the data about success rates, because I know it will be taken out of context by vendors eager to promote alternatives to CDPs. I should stress again that the main obstacles to CDP success are organizational, not technical, and that success rates reported here are consistent with tech project success rates in general.  Bottom line: CDPs are major projects that don’t always go smoothly but are not more failure-prone than any other major IT effort.

What worries me more than bad news is the hints that companies are making poor decisions. Prioritizing cost over requirements will surely lead to more companies purchasing unsuitable products. Putting IT in charge of martech will almost surely lead to unhappy martech users. While budget issues are unavoidable, poor management decisions are unforced errors. Your company should work hard to avoid them.

All that said, the most interesting results are the ones that challenge conventional wisdom. Do self-service systems really lead to bad results? Is the importance of real-time overstated? Is there really so little interest in reading data directly from a warehouse? These are headline topics in martech today, and are often accepted as truth with little discussion. The questions are worth asking because the reality is almost always more complicated than the simple answers. When is self-service useful and when is it over-used? What kinds of real-time processes are really important? How is external data best integrated with a conventional CDP? Answering those questions will help companies make better decisions and ultimately lead to greater martech success.