Why dbt Labs acquired Transform
Why dbt Labs acquired Transform
Dbt Labs expands; you can too; Analytics sucks; data is better; chaos.
I’m Sven and I’m writing this to help you build things with data. Whether you’re a data PM, inside a data startup, internal data lead, or investing in data companies, this is for you.
Dbt Labs expands beyond analytics engineers.
Dbt Labs follows the standard tech expansion business process.
Analytics sucks; go with data instead.
Chaos data engineering is picking up.
Let’s dive in!
1st vs. 2nd order data teams
What: Taylor Murphy started a substack and recently wrote about dbt Labs acquisition of Transform, a metrics layer company. In the article, he discusses how analytics is a second-order need of any business.
My take: The article sparked two generic thoughts in me, one about the dbt Labs acquisition and the second about "analytics being a second-order need."
(1) So let us start with the dbt Labs acquisition
Dbt Labs acquired a metrics layer company for one sole reason:
Dbt Labs has been targeting a new market segment for quite some time. A customer segment that isn't the analytics engineers anymore.
Dbt Labs changed its product messaging about a year ago, away from analytics teams and engineers towards data teams and workflows. In addition, they specifically added ML models and operational analytics into their messaging.
Dbt Labs also introduced multiple features to target a broader market segment:
Python-model support
Their own metrics layer
I'm not explaining this because it's a bad idea. This process is precisely how any tech company usually works. It's a great business example of this in action.
Rolling into a new tech market
Stage 1: Tech companies start by targeting a tiny specific customer segment. They try to find a pond they can be the biggest fish in. For the company dbt Labs that was the "analytics engineering pond" they created from scratch. Nice move, dbt!
Stage 2: They build up enough momentum to target a second larger segment (that includes the first one!) and leverage something from their first segment to get into it. For Dbt labs, it is the fact that many companies have mixed data teams consisting of both analytics engineers and data scientists or machine learners.
Stage 3: They adjust their product messaging to be broader and add features to support it.
Stage 4: If they pick wisely, they can become the biggest fish in this new pond and continue to ship away at the incumbents in the much bigger market. In the dbt case, the big incumbents are the big data transformation tools, pure Python, Jupyter, Apache Spark, and databricks.
If you want to read more about this strategy and see an example of a company that picked their segment suboptimally (elementl, the company behind dagster), look at the Thoughtful Friday number 19.
FWIW: I'm not sure Dbt Labs picked wisely either.
(2) 2nd order analytics
Let's go ahead and get the fact straight.
The word analytics means analyzing data on an organizational scale inside the company.
And yes, I agree with Taylor on this; analytics is a 2nd order need of running a business.
However, data isn't analytics. Not at all. Once you get tricked into thinking this, your business is lost. Data is inside the whole company; data is the life and blood of many companies, necessary to running every aspect of the Amazon and Netflixes of this world.
Data isn't analytics. Not at all. Once you get tricked into thinking this, your business is lost.
Taylor writes from a perspective from within a data team performing analytics work. That makes it easy to think all data work is that. But weighted by importance, that's just a tiny fraction.
Data teams inside the analytics department do not need to show off their value because there is little (as Taylor explains, with the opposite conclusion). That is, until the company decides to go all-in on data and initiate a top-down initiative to make data vital.
Companies must initiate a top-down approach to making data necessary inside the company.
Inside all places:
Inside the company strategy,
the business & the products
Inside the critical decision processes in sales, marketing, and overall business
and finally, therefore, also inside the small part that the analytics function plays
I haven't seen this transformation happening from within; I've only seen plenty of examples of top-down approaches. My favorite example of this is Capital One, which I explain in detail in the article "Good data bad data".
(3) Chaos Data Engineering
What: Shane Murray, the CTO of Monte Carlo Data, a data observability company, just published an article about chaos data engineering.
My take: I love how Shane describes the results of a single chaos day his team at the NYT perform:
"The next day the team made adjustments. We decoupled systems, implemented failsafes, and returned to the court for game 2. As a result, the 2020 election was the first I can remember where the on-call engineers weren't on the edge of their seats, white-knuckling their keyboards…At least not for system reliability reasons."
Once your data becomes vital inside the company (once you got a top-down approach to making that happen), you'll need tools like this.
How was it?
I genuinely believe that you can take a lot of shortcuts by reading pieces from people with real experience that can condense their wisdom into words.
And that’s what I’m collecting here, little pieces of wisdom from other smart people.
You’re welcome to email me with questions or raise issues I should discuss. If you know a great topic, let me know about it.
If you feel like this might be worthwhile to someone else, go ahead and pass it along; finding good reads is always a challenge, and they will appreciate it.
Until next week,
Sven
I got a little lazy in my prose towards the end of the post. I said "When data is already a 2nd order need for a company" when I should've said "analytics".
I agree with your framing of analytics as a small part of all of "Data" within an organization.