Dissecting What Makes a Data Strategy Fail: Fluff, Challenges, and Objectives

This is the Three Data Point Thursday, making your business smarter with data & AI.

Let’s dive in!

---

Actionable insights

• Understand the Essence of a Good Data Strategy. Recognize that a strong data strategy is more than just a well-documented plan. It should address specific challenges, leverage unique strengths, and clearly articulate how data will drive competitive advantage or operational excellence.

• Bad Data Strategies Have Incoherent Actions and No Leverage. Bad strategy is a concept that describes “strategies” that really aren’t strategies. Strategies fail to be strategic if they don’t have leverage OR a set of actions that is coherent with the leverage point.

• Spotting Bad Data Strategies. There are three primary ways to spot bad data strategies. Use them to figure out whether you might not yet have a strategy. The three primary ways are fluff, failure to face the challenge (the elephant in the room), and bad strategic objectives.

• Spotting them in the wild. BI toolmaker Mode loves fluff. The data observability framework has great expectations, and the company behind it fails to address the elephant in the room; competitors Kensu and Monte Carlo data do much better. DbtLabs fails to set good strategic objectives. 

Whether you’re a data startup founder, a PM, or a data leader, if you cannot spot a bad data strategy, your stuff will stay average, without impact, and be unsuccessful.

Unfortunately, bad data strategies are everywhere. Sometimes, I find it hard to find examples of good ones! Most data startups start out with a bad strategy. Almost every single internal data initiative I read is bad. 

I don’t mean it’s not well thought out; most are. Most startups talk to tons of customers to come up with their product strategy. They are usually well written and documented and contain a plan to achieve it.

 What I mean is that they have what Richard Rumelt calls a “bad strategy” or, more specifically, a bad data strategy. The word doesn’t just mean it’s a strategy that is bad. Rumelt invented that concept to describe a very specific constellation of attributes, and it is sadly even more pervasive in data than in business strategy. Bad strategy is an identifiable way of thinking and communication about strategy that at its core isn’t strategy at all. 

The good news: You can learn how to spot bad data strategies. So what I’ll do in this article is to help you first of all understand how Rumelt’s concept of bad strategy extends to into the data world. Then, we’ll start to spot a few in the wild to get you some practice!

FWIW: We’ll talk both about external data strategies like the business strategies of data startups, or the product strategies of data-heavy products, and internal data strategies, the ones of your data department, of your data mesh initiative, or your business intelligence efforts. 

---

Sometimes even Napoleon sucks at strategy

Meme link.

On the 24th of June 1812, the french army crossed the Niemen river, effectively entering Russia starting the Russian campaign. 

Napoleon is admired as one of the greatest strategists of all time, yet his invasion of Russia looked very different from what people had seen from this amazing military leader before.

Napoleon's invasion of Russia in 1812 is also a classic example of a bad strategy leading to a catastrophic campaign. Napoleon's overconfidence and lack of preparation for the harsh Russian winter, coupled with the Russian strategy of scorched-earth tactics, led to the severe weakening of his Grand Armée. This campaign significantly damaged Napoleon's military might and reputation. (This probably can show goal for strategy, from an amazing strategist, failure to face the challenge, and a few more…)

When data-heavy products fail, data startups won’t grow but rather drag along, when data-initiatives fail to deliver results, it is usually not just the plan, or simply the product management that’s at fault. It’s the whole data strategy. The strategy isn’t just the product, it is the entire set of actions, of policies a company has in place. It’s the community they try to create around the product, their thoughtleadership, their open source project. For internal data initiatives, it is …

Bad data strategy isn’t just one that doesn’t deliver, or one that fails for some reason. It is a very common way of getting data wrong. In fact, today, the default way to go about data is the bad data strategy. 

Bad data strategies underlie both internal data initiatives, but also data-heavy products. Data companies that switch from an open-source solution to a commercial place will often fall into a bad data strategy. Data companies targeting a new segment will often fall into the same trap. 

While it isn’t easy to fix bad data strategies, it is easy to spot a bad data strategy.

The three markers of bad data strategies

There are three concrete markers that indicate a strategy that lacks leverage or a set of coherent actions to use the leverage point:

• Fluff = restating the obvious. 

  • hot data trend fluff - “the coffee cup for data mesh companies”

  • generic data fluff - “the spreadsheet to improve your decision making”

• Failure to face the challenge

  • Your one challenge inside the data flow.

• Bad strategic objectives 

In the world of data, these three markers take very specific incarnations. 

Fluff is restating the obvious. Stating 2+2=4, or “all natural numbers are natural.” There’s nothing wrong with the idea of fluff, it just has nothing to do with strategy. So if it is inside your strategy, inside your marketing, inside your messaging, it’s a sign you don’t have any idea of what your strategy really is.

In the data world, two kinds of fluff are perversive. One kind is hot data trend fluff. Basically trying to fit “data mesh” and “generative AI” into every product, every marketing message, everything you have. 

The second kind is generic data fluff. If you’ve been with this newsletter for some time, you know I propagate that the only point of having data is to take better actions by making better/faster decisions. A lot of people intuitively know that and thus turn it into their strategy. But if it’s the only point, it’s obvious and logical, not any kind of strategy at all.

Failure to face the challenge in data means just one thing: You cannot state your one challenge, your one bottleneck. Your one data flow that will push on a lever that in turn will lift a huge weight. You’ve failed to identify your one challenge in data flow.

Finally, the third marker is a strategy that contains guiding policies that are incoherent with the kernel of your data strategy. A 10 step plan to creating a data mesh inside a company is good and well, but if there’s no need for a data mesh, it’s just a means to an end, and a bad data strategy. We’re typically meeting two kinds of incoherent policies. 

For internal data strategies, quite often we end up with a means to an end, a plan that doesn’t belong to ANY strategic kernel, without a clear goal that brings the company in a strategically better position. Of course, this might also happen for external data strategies focused on products.

For products, more often than not, there is an idea, a lever, and a key data flow to improve, but the steps to get there are simply incoherent. Either simply impossible or just plain not in line.

Enough of the fluff, let’s dive into some examples!

Good strategies

How can we inspect strategy? That’s actually a tough task. Most companies don’t share their internal strategy documents. But even if they do, they might act differently than described.

But that’s good for us, because strategy is first and foremost a set of actions! By observing what companies say and do we can peek behind the curtain of their strategy. That’s what we’re going to do. 

Of course, well designed data strategies ARE written out and continuously reinforced by leadership, but that’s the part we sadly won’t be able to observe.

So, we’re going to look into marketing messaging, blog posts, and product features to get an idea of strategy.

But because I really don’t want to rip on other companies all day long, I’ll try to share one personal story at the beginning of each section, one where I know 100% of the strategy behind it because I created and enforced it.

While everything I’ll share applies to both internal and external projects, I have even less insights into other peoples internal projects, so keep that in mind with relation to the stories. 

A fluffy mode

Napoleon was a master at rallying his troups, its one of the reasons he could come back from exile as easily as he did. However, when entering the Battle of Borodino, his address to the troops was this:

"Soldiers: This is the battle you have so much desired. The victory depends upon you! It is now necessary to us. It will give us abundance of good winter quarters, and a prompt return to our country. Behave as at Austerlitz, at Friedland, at Witepsk, at Smolensk, and let the latest posterity recount with pride your conduct on this day; let them say of you, 'He was at the battle under the walls of Moscow.'" - Address to the troups

I translate this to: You wanted this (unlikely), you need to win (yes that’s how battles work), we need this (of course, loosing isn’t good), win as before, and at the end you can be proud that you won. In short, it translates to “let’s win” but with a lot of fluff. He didn’t provide motivation or incentives like in all the other speeches he held, he simply fluffed it up.

I’m a master fluffer. When I start to write a strategy document, it’s 100% fluff. At least it starts out that way, it’s always “let’s improve decision making” (who would want worse decisions???) or “let’s be successful” (who doesn’t want to?). 

Meme link. Just like a fluffy strategy.

When I started out as PM of an internal data team, one of the first things I wrote down & discussed with the team was to create a strategy. Well, we set out to create a vision, we decided what our job inside the company was was to “improve decision making inside the company.”

In the coming weeks, I used every chance to mention our vision. And got little reaction. I tried to use our vision to guide our roadmap, but I pretty quickly realized, everything basically was under the “improve decision making” roof. 

In short, I had created fluff, and failed to create strategy. So instead of getting caught up with more visioneering, I decided to dig deep into strategy, actions, steps, leverage. 

I used a technique called impact mapping to draw up a big tree of the company, the departments, and where we could create the most value. I quickly identified one department, the sales department, as being critical. We already had a way in, salespeople were willing to use our data, and their decisions mattered a lot for the current company strategy. 

So, instead of keeping our broad vision, I switched. I narrowed it down to a strategy. The kernel was clear: Help salespeople get more value out of their existing customers by providing them with more and better data. Our guiding policies were simple:

• Focus on the sales department

• Focus on the salespeople most involved with recent changes in our company strategy

• Make sure they get good data and are able to make quicker decisions.

• Focus on speed! 

• Provide them with completely new data that might help to get more out of existing customers

Based on this we were able to build a good focused roadmap, and deliver measurable value within months of starting down this road.

But it all started with a lot of fluff, a lot of fluff that led to incoherent actions and lots of wasted efforts.

If we look into the company mode, it looks like we’re onto fluffy terrain as well. Mode provides a business intelligence tool, like so many other players in the market. 

Marketing messaging retrieved from mode.com in Jan 2024

Mode markets its solution as the “central hub for analysis,” it’s a prime example of what I call generic data fluff. Of course, a business intelligence tool is about analysis inside an organization. 

Mode then continues to market their tool as a means to “drive business outcomes.” a good example of generic strategy fluff - things that are virtually true of any tool one would buy. If a tool doesn’t help my business, why would I buy it? 

What mode tries to say but doesn’t seem to focus on is their idea of “uniting data teams and business teams.” Apparently, the people at mode had the vague idea of building a tool that breaks up an existing cliff between data and business teams. 

The question is, did they then focus all of their energy on lifting this cliff? Or did they stick with the fluff, and just made this one of many potential goals, thereby loosing any momentum and any leverage they might have? 

One piece of evidence we can pull from is more marketing messaging. If you look at the diagram provided by mode below, you’ll pick up on how they are designing the data strategy for their product: They don’t believe in uniting the teams, but rather the technology used by the teams, still keeping the teams separate. 

Marketing messaging retrieved from mode.com in Jan 2024

Next, we can look into the recent releases to see what mode is working on. If you go through the list, you’ll see two things: Features aimed at data teams and features aimed at business teams. Yet what is missing completely from the list are either features to “unite data and business teams” or features to “unite the platform for data & business teams.” 

Finally, we can look into one strategic document, the “The Next Chapter” blog post outlining what Mode imagined their strategy to be like in 2022. FWIW, Mode was acquired by ThoughtSpot in June 2023, which potentially can have a big impact on the strategy. In this article Emily Ritter, VP of Marketing of Mode, makes the case against “fragmented analytics” and for getting everyone into one room to drive outcomes. 

Emily’s argument relies on the idea that fragmented analytics doesn’t make data teams happy. But ignores the question of whether it does make business teams happy or WHY they would be unhappy. Essentially, we find more fluff, covering a not well-thought-out argument.

Mode had an insight, they saw “fragmented analytics” they saw unhappy data teams, and probably something that didn’t look like “delivered outcomes.” However, instead of uncovering the mechanisms behind it, and truly finding a good lever, they sadly fell for fluff, fluff that covered up a potentially good strategy. This in turn resulted in incoherent actions by the product managers, with many developed features that go in different directions. It resulted in an acquisition by ThoughtSpot, likely because of the bad data strategy. ThoughtSpot likely needed to buy any BI tool with a broader set of capabilities and data teams as users. Mode made for a cheap acquisition target because they didn’t go 100% all in …

While fluff is a clear indicator of bad strategy, in mode’s case fluff likely covered up a meaningful kernel that they could’ve executed on, if companies aren’t able to find a lever however, no key advantage, there’s nothing that fluff could cover up, they simply don’t face the challenge.

Great expectations that don’t face the challenge

The invasion of Russia wasn’t short of a big strategic objective. Napoleon clearly wanted to enforce French dominance in all of Europe. And yet, sometimes it looks like he failed the most fundamental task: To look at a map. To see the vastness of Russia, understand the climate conditions, and face the fundamental challenge: How can a French army conquer such a vast country in potentially terrible weather conditions? I’m, not saying Napoleon didn’t know all this, he most certainly did, it’s why he tried to move as fast as he did. But what he lacked was a strategy to deal with it, his strategy was hoping for the optimal path, and that’s not a strategy at all.

A strategy for a data-heavy product tells us how the product uses data to give us leverage. Netflixes recommendation engine gives me as a customer leverage, it helps me to quickly find good movies. Yet if we fail to define what this leverage is over, what the problem is, and what the challenge is, the product will fail. 

Way back in 2011, I was the co-founder of a data startup in Munich. We wanted to revolutionize the vacation rental space by providing business intelligence to vacation homeowners. We’d scrape websites to get all the data we could, pricing, bookings, distances to the beach and the likes. We knew from a few calls we made, that people really had no idea what to charge, whether to build a pool, or a terrass or nothing at all. We figured if we could give the underlying data to them, they’d be willing to pay quite the price.

We immediately got to work, coded, talked to potential partners, talked to people about additional use cases for the technology we were building. We did a lot. And yet we failed because we ignored the elephant in the room: Why would a vacation rental owner, who doesn’t know how to make these business decisions, spend money on a software application with lots of data inside it? The few calls we had were with people not comfortable with SaaS applications, one of the reasons they didn’t know what to charge - they simply didn’t care that much.

Today, a decade later, I can only guess at the true problems. I can tell you what we should’ve done: Proper product management! We should’ve had hundreds of discovery calls to get to the ground of the problems our market was having. It’s what every startup does to find the elephant. Yet we didn’t and we failed (to be fair, I failed even earlier as me and my co-founder decided to part ways over the general strategy - and I don’t think I was right.)

What we had wasn’t a strategy, it was a wish list of nice to haves, and a piece of technology.

The data startup Great Expectations (GX) seems to be falling into a similar trap, at least from what I can observe from the outside. GX is broadly speaking in the data quality space. They secured $40m in Series B funding in 2022 for a valuation of close to $300m and have a successful open-source solution called great_expectations with 9.2k GitHub stars.

Open source based companies often have marketing problems, because they fail in their heads to clearly separate the open source project from their paid offering. Lucky for us, Great Expectations is no different. They sell their paid service as “the OS thing but easier”, that’s it.

Great Expectations Cloud website retrieved in Feb 2024.

While this isn’t a great way of growing GX, it makes it easy for us to understand their strategy, because with this, everything they do to develop the OS is also part of the strategy of their paid product.

So, first, let’s look at the messaging GX uses on their website. 

Great Expectations website retrieved in Feb 2024.

Based on this message, we might induce that the challenge in the data quality space is. Let’s deconstruct it piece by piece: 

• Have confidence in your data = A lack of confidence in data?

• No matter what = that’s fluff - or do you know what this should mean?

• Built on the strength of a robust worldwide data quality community = Challenge is that we don’t have a community for data quality? 

• … is revolutionizing data quality = we’re lacking a revolution? Fluff.

• … and collaboration = We’re lacking collaboration?

Meme link.

As you might notice, GX points to several potential challenges that are troubling people. From a perspective inside the data space, only one would make sense to me, and that’s “a lack of confidence in data.” I assume the messaging is filled with the rest because GX isn’t able to focus and, as a result, produces fluff.

We can digg deeper into GX by taking a peak into the “GX story”, written by the company itself. GX was started because data practitioners needed a better way to test and document their data. In another blog post, GX writes about how they noticed over time, that whatever they see as a problem, collaboration is also one, collaboration across technical and non-technical teams. 

“Data quality’s real challenge is that it’s a collaboration problem. Data professionals face challenges in communicating with other data teams and nontechnical teams alike, though not the same challenges.” - Erin Kapp from GX

Looking through these strategic bits and pieces by GX, I keep thinking back to my experience 12 years ago. I would like to have confidence in my data, just as much as my homeowners wanted to know how much to charge for a night. But is it the elephant in the room? Or maybe the reason there’s so much fluff and multiple challenges in the messaging is… that GX doesn’t know what the elephant in the room is.

To see what the elephant might look like, we can take a look at other companies inside the data quality space, kensu.io and montecarlodata.com. Interestingly, both of these companies have a very simplistic and clear message, as you can see below:

Monte Carlo website, retrieved Feb 2024.

Kensu website, retrieved Feb 2024. 

If you take those two together, it seems like the elephant in the room is simple: bad data/breaking data pipelines hurt your business. 

The strategy both companies have is simple: Help data teams resolve this problem as fast as possible. No collaboration, no revolution, no talking around the elephant. Both companies attack the elephant head-on.

If you take these things together, it looks like GX fails to address the challenge in analysis and in doing. Not many of the features GX has worked directly towards taking care of the elephant in the room. The money GX spends on building out a worldwide community around data quality certainly doesn’t do much. While I’m a fan of open source, GX should’ve long realized the investment into OS is merely a marketing vehicle and not a tool to solve their key problem.

But then again, if you don’t see the challenge, you can’t solve it. 

Let’s turn to another dead sure sign of bad strategy. When you have identified a challenge, a lot of people love to have the strategy to “solve the challenge.”

Bad strategic objectives 

Napoleon had a good strategic kernel, he assembled the largest army at the time, and was the best tactician. He had leverage, and he wanted to secure dominance in Europe. But going deep into Russia simply wasn’t a good objective. His guiding policy of what he planned to do and his strategic kernel were incoherent.

I remember back to a time when my team was tasked with building out the data collection infrastructure inside a company. It was an objective straight from top management. Their strategy was simple:

• We have a good data team and lots of data from the participants in the network

• If we make sure our data is collected with high quality, we can then use AI to turn the data into value.

We went to work and built out a decent data infrastructure. We hit our objective, and the data was used for lots of different use cases, but all of them focused on internal reporting. So what had happened? The strategy was incoherent. The goal of bringing value to the network participants through this data was good, but it was misaligned with the objective of building out a great data collection engine.

Disclaimer: For some reason, I find myself writing critically about DbtLabs again and again. Not because I don’t like the company; I love the product and the movement they started. I just wish they would take a more direct course to use the assets they have.

In a similar way, I think DbtLabs is fishing for a good data strategy while trying to run their business. To me it looks a lot like they compromised their data strategy while rolling into new markets, thereby reducing the power of their product.

Let’s dissect what DbtLabs has to say about dbt (cloud) to understand the product strategy:

dbtLabs website, retrieved in Feb 2024

On their website, the key message is followed by these supporting factors:

• “Experience the new standard for data transformation”

• “Accelerate the data development lifecycle”

• “Maintain governance and control of your analytics code”

• “Build for scale and complexity”

While “data products” sounds like a fluff word, riding on the data mesh wave, the implied key challenge seems to be “data (products) want to be shipped faster by everyone.” From my own experience, that sounds true enough. It does sound like the word “trusted” is stuffed in there; who would want to ship crappy data products? So, with a tiny bit of fluff, a true challenge is laid out. 

However, if you look at the other parts of messaging, things get messy. 

First, the subheading, the second most important part of the messaging, tells us “you have to use dbt, because everyone does.” This idea is repeated by the line below about dbt being “a new standard.” 

Second, looking into the four supporting ideas on the main website, only one, “Accelerate the data development lifecycle,” picks up the idea of faster data shipping. 

So it looks like the objectives dbtLabs sets for the Dbt Cloud product might not be aligned with the key strategic kernel they laid out. Let’s look into more bits and pieces to dig deeper.

• In Feb 2022, CEO and founder Tristan Handy wrote an article explaining a big shift in his thinking about dbt cloud. In his article “The next layer of the modern data stack” he essentially argues that dbt has grown beyond a simple transformation tool and should place itself as kind of a glue, a layer to make much more possible. The gist of the message: Dbt Cloud will extend the user's abilities and will allow them to do more.

• In Feb 2023, dbt Labs acquired Transform, in a continued effort to create a standard around the so-called semantic layer, extending the strategy laid out by Tristan in the post above. 

• In Oct 2023, the biggest product update of the year was announced. The message was a clear focus on complexity and scale: “These new features are geared to help our customers solve problems of complexity. As organizations—and organizational data—grow in complexity, it becomes increasingly difficult for data teams to remain agile, maintain quality and consistency across a wide assortment of data products, and keep spend under control. “

The objectives of dbt Cloud (and dbt core) seem to be very clear:

• Help bigger companies work with data with consistency and high-quality

• Help data people do more and extend their abilities.

There’s one thing missing from this list: speed. You may think these objectives serve speed, but if you think them through carefully, they are almost orthogonal. Extending abilities means extending the complexity of dbt, and thus making it slower to use, requiring experts, again slowing the development speed of data products throughout an organization.

Helping bigger companies work with data is important, consistency is important, and quality is important. It’s just that all of those things come at the cost of speed. Especially speed for small to medium-sized businesses. 

In essence, dbt Labs fails to deliver a good strategic kernel by misaligning what they do. They are consistent in what they do, that’s to help big companies do more with data. But it is not aligned with what they set out dbt cloud to do. 

Inherently, that’s not a bad thing. The question remains though: Did dbt Labs pivot to a different strategy, one with a different kernel and challenge? One where these objectives make sense? Or are they stuck in a place between two chairs? 

My perspective: While small and medium-sized companies need a standard to work with data and a community around it and all that stuff, large companies do not. 

Notes on Resources

The three markers of a bad data strategy are adapted from Rumelt’s amazing book “Good Strategy Bad Strategy” one of the best business books every written (see Chapter 3 on “Bad Strategy” for more details.)

I’ve ignored one of the markers he lists called “mistaking goals for a strategy” because I couldn’t write well about it. It’s hard to get the level of insights into company internals to provide good examples of it. Furthermore, while it is an important sign of bad data strategies, there’s no adaptation I could make to make it more relevant to data. So, keep that in mind when looking for your own strategy writing. 

---

Here are some special goodies for my readers

• 👉 The Data-Heavy Product Idea Checklist - Got a new product idea for a data-heavy product? Then, use this 26-point checklist to see whether it’s good!

• The Practical Alternative Data Brief - The exec. Summary on (our perspective on) alternative data - filled with exercises to play around with.

• The Alt Data Inspiration List - Dozens of ideas on alternative data sources to inspire you!