Yeah well, and I did truly feel it. The tension between what I saw with Pat, and what I was now delivering myself (and of course others in the new company).
The only push back I have is on: "dbt, you’re likely the infrastructure that will slow down AI development, not enable it." My thinking here is that a lot of these AI agents, at least right now, need a text/code-based representation of the world to work with. Sure an agent can query the schema and write some SQL, but there's a ton of context it would do well to have stored in tools like dbt.
That said, I don't know how well Codex / Claude Code would work with dbt Cloud or any other SaaS offering, so maybe you're right here.
Taylor - appreciate the pushback! And fair warning: I know I've been the personal dbt -in-residence hater since basically the creation of dbt Labs, so take this with a grain of salt—I may be lacking objectivity here 😅
You're absolutely right that AI agents benefit from structured context. No argument there. The schema alone isn't enough—having semantic layers, business logic, and well-documented transformations is genuinely valuable.
Where I think the friction happens: when you're an ML/AI engineer trying to ship fast, you're often building that context layer yourself because you can't afford to wait. Not because dbt's approach is conceptually wrong, but because:
The governance/approval cycles through data teams are too slow for rapid AI iteration
AI teams need the flexibility to restructure context on the fly as models evolve
Production AI systems often end up maintaining their own semantic understanding anyway.
Thing is, this sounds like a technicality but it is NOT.
When ML engineers push past dbt to build their own infrastructure, it signals where the value actually accumulates—and it's not in the data infrastructure layer.
Think about e-commerce and parcel shipping. The entire e-commerce revolution was powered by the parcel shipping industry. But where did the profits go? As far as I can see, they all went to the Amazons of the world. The shipping infrastructure became so commoditized that most larger e-commerce companies eventually started their own logistics operations, eliminating any profit that could be made there.
That's what I see happening with dbt positioning itself as "AI infrastructure." Even if they're technically correct that AI needs structured context, being the infrastructure layer that AI builds on top of is likely a shitty place to be—with no profits to be found. The value accrues to whoever owns the AI application layer, not the data transformation layer underneath.
That's why I'm arguing strongly for looking in a different place entirely—one that's closer to the original discipline of BI: actually helping people make better decisions, not just providing infrastructure for those who will.
I think I see what you're saying. My mental model here is that a tool like dbt (let's just say a SQL/context orchestrator) will be responsible for managing the shared "truth" data model that can then be used by application layer teams. Those AI/ML teams will reference the ground truth data and pull (or be pushed) updates as needed, but they're free to go off and do their own thing. (There's an analogy to git branching here in my mind).
I just have a stronger bias towards have a shared, org-wide source of truth that can be used as needed by teams (hopefully w/o slowing anyone down). But! This also depends on your scale as a company. I do think a lot of these conversations are muddy because we're imagining different size orgs in our head.
One thing you're hinting at is that between the user-facing applications and true infrastructure (data center, GPUs, database vendors etc) dbt is always going to be middleware and will see compression from both sides.
Excellent analysis! The line about a $100k position becoming a $20 subscription perfecly captures the disruptive economic reality AI brings.
Yeah well, and I did truly feel it. The tension between what I saw with Pat, and what I was now delivering myself (and of course others in the new company).
I agree with the general thrust of the article.
The only push back I have is on: "dbt, you’re likely the infrastructure that will slow down AI development, not enable it." My thinking here is that a lot of these AI agents, at least right now, need a text/code-based representation of the world to work with. Sure an agent can query the schema and write some SQL, but there's a ton of context it would do well to have stored in tools like dbt.
That said, I don't know how well Codex / Claude Code would work with dbt Cloud or any other SaaS offering, so maybe you're right here.
Taylor - appreciate the pushback! And fair warning: I know I've been the personal dbt -in-residence hater since basically the creation of dbt Labs, so take this with a grain of salt—I may be lacking objectivity here 😅
You're absolutely right that AI agents benefit from structured context. No argument there. The schema alone isn't enough—having semantic layers, business logic, and well-documented transformations is genuinely valuable.
Where I think the friction happens: when you're an ML/AI engineer trying to ship fast, you're often building that context layer yourself because you can't afford to wait. Not because dbt's approach is conceptually wrong, but because:
The governance/approval cycles through data teams are too slow for rapid AI iteration
AI teams need the flexibility to restructure context on the fly as models evolve
Production AI systems often end up maintaining their own semantic understanding anyway.
Thing is, this sounds like a technicality but it is NOT.
When ML engineers push past dbt to build their own infrastructure, it signals where the value actually accumulates—and it's not in the data infrastructure layer.
Think about e-commerce and parcel shipping. The entire e-commerce revolution was powered by the parcel shipping industry. But where did the profits go? As far as I can see, they all went to the Amazons of the world. The shipping infrastructure became so commoditized that most larger e-commerce companies eventually started their own logistics operations, eliminating any profit that could be made there.
That's what I see happening with dbt positioning itself as "AI infrastructure." Even if they're technically correct that AI needs structured context, being the infrastructure layer that AI builds on top of is likely a shitty place to be—with no profits to be found. The value accrues to whoever owns the AI application layer, not the data transformation layer underneath.
That's why I'm arguing strongly for looking in a different place entirely—one that's closer to the original discipline of BI: actually helping people make better decisions, not just providing infrastructure for those who will.
I think I see what you're saying. My mental model here is that a tool like dbt (let's just say a SQL/context orchestrator) will be responsible for managing the shared "truth" data model that can then be used by application layer teams. Those AI/ML teams will reference the ground truth data and pull (or be pushed) updates as needed, but they're free to go off and do their own thing. (There's an analogy to git branching here in my mind).
I just have a stronger bias towards have a shared, org-wide source of truth that can be used as needed by teams (hopefully w/o slowing anyone down). But! This also depends on your scale as a company. I do think a lot of these conversations are muddy because we're imagining different size orgs in our head.
One thing you're hinting at is that between the user-facing applications and true infrastructure (data center, GPUs, database vendors etc) dbt is always going to be middleware and will see compression from both sides.