What’s Wrong with BI

“BI people produce reliable dashboards.” - the sad status quo of BI according to random voices from the internet

“How do you think the future of Business Intelligence (BI) will look?” An innocent question at first sight. But since I was asked that exact question a couple of weeks ago, I can’t stop thinking about its implications.

It’s not that I don’t have a good answer; I think a lot of us agree on the general direction; it’s that I see the need for big changes, both on the sides of companies and on the side of builders of BI tools. It’s those implications that brought me here to write his piece.

Let’s wind back a bit: I believe BI, in its current state, is a far cry from what we need it to be, even today. The industry has developed a narrow focus on ultimately delivering tabular data to graphical tools. All of the backbone of ETL or ELT is based on that premise.

Nothing wrong with that, right? Rolf Potts, in his travel book Vagabonding, describes how we once walked the Philipiine port of Cebu trying to locate an ATM machine. He made use of the natives half-baked English skills by asking yes-and-no questions.

For two hours. Until he realized his method didn’t match his task.

I think that’s a lot like the state of BI right now. The discipline of BI isn’t about tabular data; it’s about making better business decisions through data. There’s a clear distinction between the discipline of BI and the industry of BI tools. And just like yes and no questions won’t bring you to an ATM machine in Cebu, tabular data and reports won’t bring you the right better decisions in business (sorry to disappoint you.)

The biggest challenge for the BI industry is to align back with the original goals of the business intelligence discipline. Similarly, every company can proactively realign business intelligence to drive better decisions instead of just using fancy-looking tools.

So let’s discuss this in detail.

You’re working with a data team and want to improve how you work with BI? Level up your game? Well, you’re out of luck, sort of; you’ll be stuck with tools that aren’t developed to make that possible. That sucks; it doesn’t mean this article isn’t for you; it means you’ll have to do some serious work to get through it, using the slivers of hope I’ll highlight along the way. If you’re a builder, a founder, or a PM of such BI tools, I encourage you to think through those wrongs and the misalignment and figure out how to fix it; otherwise, someone else will!

How the BI Industry Tricked Us All - What True BI Is

Let’s ask some of our friends what business intelligence is.

Microsoft - What is Business Intelligence: “Business intelligence (BI) uncovers insights for making strategic decisions. Business intelligence tools analyze historical and current data and present findings in intuitive visual formats.”

Take a close look at the two sentences, the first is a clear definition of business intelligence as a discipline. The second tells me what BI tools do. And nothing links the two together. That’s the great divide of BI.

IBM - What is Business Intelligence: “Business intelligence (BI) is software that ingests business data and presents it in user-friendly views such as reports, dashboards, charts and graphs. Analyzing this data helps businessses gain actionable insights and inform decision-making.”

Another close look at IBM’s version is even more telling: IBM clearly defines the discipline of BI as purely the domain of software (it’s not; carpeting isn’t equal to a hammer).

The people at IBM score a point with their second sentence, trying to claim that these tools help us make better decisions. In reality, it’s clear they know the divide exists and should be closed, and they have no idea how.

So what is BI truly? According to Wikipedia, a bunch of people played a vital role in defining the term, including the IBM (!) researcher Hans Peter Luhn, who described it as

"the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal.”

From the early days of BI, we all knew that BI is about guiding actions.

*https://imgflip.com/i/8ntvg5*

I don’t know how the divide between the BI discipline and the BI tool industry emerged; I just acknowledge its existence. Let’s now explore three concrete ways this divide shows up: three wrongs to be righted, so to speak.

What’s Wrong 1: BI Tools Are Built For People Who Can Use Them, Not For Those Who Should Use Them.

For some reason, inside the BI, we see reinforcing feedback loops that stop the industry from progressing and from reaching the users they genuinely need to get: decision-makers. Things like collaborative BI seem to make steps in the right direction, but these are few and far between. You should start your journey with education, first and foremost, education for decision-makers and education for analysts on how to help decision-makers make decisions. Keep the decisions are primitive, never the data.

The negative feedback loop of analysts. BI tools are made for analysts, the people inside the company who work with data, and the bridge between decision-makers and the technology. These analysts in turn are trained on those BI tools, and thus, a feedback loop goes around, reinforcing the idea that all that analysts do is to work with BI tools, and BI tools being built mostly for analysts.

How to work for decision-makers. A BI tool should be one of the many tools that help to achieve the goals of the discipline of BI. A BI tool, thus, should help to drive better decisions through the use of data. Decisions inside companies are made by humans, so-called decision-makers. And we can be certain that all decision-makers today know how to use a computer. All that’s left, is to build a toolkit that works with them, not with their analysts.

Slivers of hope. Luckily there are a dozen trends that somewhat try to right this wrong. Unfortunately, they are scattered, trying to pull on lots of strings in a huge ball of wool.

Trend 1: Self-serve analytics. Self-serve analytics is a simple idea: Enable people do analytics themselves, 100% of it. Unfortunately, while the promise of self-serve analytics sounds exactly like what we need, what is delivered isn’t. Every self-serve analytics solution I’ve seen so far could be rebranded as “self-serve analytics for analysts.” Unless self-serve starts to focus on decision-makers, it’s not self-serve.

Trend 2: Automatic storytelling. Automatic storytelling is a different idea. Instead of giving people the ability to analyze and thus deduce an analytical story themselves, automatic storytelling tries to also analyze the data for you. This effort is basically trying to automate the analyst as his role is currently understood away. In it, it truly tries to bridge the gap between data and decision-makers. Unfortunately, I’ve not yet seen a great implementation of it.

Trend 3: Collaborative BI. Collaborative BI acknowledges that decision-making goes beyond the analyst and might also involve the data engineer or whoever collects the data. The idea is good, and most implementations are working in the right direction, yet they still fail the next two wrongs.

What can you do for your BI journey? The single most important tip I can give you when it comes to BI is to start with education, which is never a tool. As you can see, the tools aren’t really ready and will do much to redirect you from your actual efforts. But starting with education won’t. If you educate your decision-makers first, they will be able to use the tools they already have (hello, Excel and Salesforce!) to do BI. Then, when you start to introduce actual BI tools, they will be ready to guide the implementation in the right direction.

In terms of tools, make sure to get your data as close to decision-makers as possible; think about embedded BI, reverse ETL, and data activation.

What’s Wrong 2: BI & ETL Tools Work With The Data Companies Have, Not Should Have

BI tools emerged to work with structured data; by definition, that’s only the data we choose to look at. Unfortunately, all the data that matters today and tomorrow isn’t structured. There’s sadly not much hope; by and large, it’s still a huge opportunity to explore. What you can do is to get your hands dirty and focus on working with the data that answers your key questions, not the one you have at hand.

https://imgflip.com/i/8ntw22

Stuck in old data. The discipline of BI is decades old, but even the newest trends like dbt and SQL as a lingua france of BI have cemented one view: the goal of BI is to provide lots of structured data to decision makers (or their analysts) in elaborate hierarchies of models.

How it should be. Almost the entire BI industry focuses on providing and working with structured data. Yet it is also clear that 90% of the data that is generated this year, or the next, or the one after that, that could be available to your company is unstructured, event-based (time series), and real-time. Almost the polar opposite of structured data.

It simply isn’t possible to force this data into structured formats all day long; it’s too slow and too costly. Instead, we need a new way of working with this kind of data, one that works with it as it is, in its natural form. We have a few ways of working with this kind of data, but they are isolated and disconnected from most BI efforts.

Slivers of hope. Not much, to be honest; what we’ve seen so far is the emergence of special-purpose tools to work with each of those special data pieces individually. There are time-series databases, the rise of data lakes to work with unstructured data, and quite a few real-time engines. The rise of Databricks is a good sign for the world of unstructured data. But it is safe to say that we’re still ignoring most of what’s valuable when it comes to data for BI.

What you can do: Get your hands dirty. Don’t get lured into buying fancy BI tools; Start your BI journey (after you’ve seen success with your education program) with asking finding the right questions to ask. Don’t say, “We won’t get the data to answer this,” but rather focus on finding the 3-4 true key questions you need to answer to make your business better. Then, build your data around that, structured or unstructured, real-time or not.

What’s Wrong 3: BI Tools = Reports & Dashboards

BI tools have become equivalent to reports and dashboards. They became the interface and the medium. Thus, there’s no easy way to acknowledge that the interface and the medium both might suck at being useful to the people who truly need the data, decision-makers. But there are a bunch of efforts to take care of this, reverse ETL being one of the most important ones. You need to start with reverse ETL or data activation; you need to get the data into the tools that your decision-makers use before you start a central data team.

Suffocating in reports and dashboards. BI tools have come to mean two things: reports - a fancy word for tables with a few filters really, and dashboards, a fancy word for pie charts and graphs. I’ve heard millions of people say that Apple designs amazing products, and yet I’ve never ever heard anyone commenting on the great design of BI tools.

The BI industry has converged on two things; first, a special kind of interface: The graphical user interface. And second, the medium, reports and dashboards. Both seem to be the status quo and hard to move.

How it should be. Interfaces and mediums should be optimized to make using a tool simple. Data in itself is multivariate, yet most BI tool visualization forms tend to display only 2-dimensional data. Interfaces and mediums should first and foremost address the question of who the user is, and who the user should be.

If the data in the BI tool is made for salespeople, then the interface should be simple to use for a salesperson. That can mean designing a good interface, especially for them, or it could mean getting the data into an interface they are already using (like a CRM system). It’s the same story for all other decision-makers: either design good interfaces that work for them or get the data into the ones they already use and like.

Slivers of hope: Luckily, there are a bunch of diverging movements that have helped solve this problem.

Trend 1: New interfaces and mediums. Jacek from GoodData posted an interesting experiment a year ago where he essentially loaded a 3d graph into his VR headset to walk through! That’s definitely an effort to change the interface and medium at the same time. Sadly, this technology is nowhere near production-ready (but could be).

Trend 2: New mediums. Tools like count.co and BigQuery data canvases change the medium away from reports and dashboards towards what they call stories. Their idea is to incorporate more of the assumptions and data models right into the story to make the resulting analysis deeper. While I like the exploration of these ideas, I don’t think they solve the problem at hand, so far, they just layer more reports and tables on top of each other.

Trend 3: Data Activation & Reverse ETL. Both of these words basically describe bringing data into other tools, into the tools already in use by decision-makers. In general, I’m a big fan of this idea, because it pushes the can down the road a bit. It assumes the tools we push data into are well-designed for the decision-makers at hand.

Maybe they are in some cases, but in my experience, we’re more in a 50/50 situation where this only works for half of the cases at hand.

What can you do? What you can do inside your company is to embrace the idea of reverse ETL from the beginning. Don't try to build out central infrastructure at all; try to have the data in the hands of decision-makers from day one. If, at some point, that shouldn’t be enough anymore, find the minimal tool that works well for your decision-makers.

Questions and Actions!

Is your business intelligence still on track with its original goal? Let’s find out.

Here’s a quick list of questions you can ask inside your company.

1. Who’s using your BI tool? Get a list of the weekly active users (yes, weekly active is a good unit to count in), then get their roles & departments. Then count!

2. What percentage of people inside your company are using the BI tool landscape? Again, I’d measure both WAUs and MAUs (monthly active users) and then compare them to industry benchmarks.

3. What data pieces are most used? Go through the usage statistics of your reports and dashboards. Map them back to your database tables.

4. What are people using your data for? Once you have a list of the most used reports and dashboards, go to people and ask them: what do you use this for?

5. What do you think are the key questions for the business? Are people using your data to answer them? How do you make decisions on products? On marketing initiatives? Sometimes, answers to these questions are hard to come by, but they are the essential ones!

6. Should the data be somewhere else? If the people told you in (4) they export it to CSV, there’s a good chance the data really wants to live somewhere else.

I know those questions just scratch the surface, but you gotta start somewhere, right? They should give you a general feeling of where you stand. They’ll give you a feeling on whether you’re using the right data, the right interfaces, and whether you’re reaching the right people.

What now? Message me your answers if you want; I’m always up to help people make more out of data.

Related Writing

• I gave a presentation at GoodData in Prague a couple of weeks ago that prompted this blog post. Shout out to you guys!

• The questions above come from hard-learned experience; I asked myself those constantly while I was Product Manager at Unite for the data platform for 700 people: Hello Product Data Team, Goodbye Ad-Hoc Work

• For a detailed look at the journey with our central data platform towards a more decentralized approach, read How Mercateo is Rolling Out a Modern Data Platform

• If you want to improve talking to internal stakeholders about data, check out my free course: Product Discovery for Analytics Teams - Talking To Internal Customers

• Why You’re Missing The Point Of Analytics

• Finally, the book Data Strategy by Bernard Marr has a decent section on business intelligence. He’s a good reference for starting with the right questions. However, he tends to favor centralized and dashboard based approaches to BI.