Thoughtful Friday #6: Data as Code - Cutting Things Smaller
Thoughtful Friday #6: Data as Code - Cutting Things Smaller
I was about to write why ELT & ETL (and whatever more letters you can attach to that acronym) are concepts that aren’t made forever and will probably wind die.
But, then I had a short Twitter exchange with Benn Stancil which provoked me to write about something related but different.
About how to apply a few software engineering best practices into an existing data team, without trying to push a hole organizational change like the “data mesh change”.
Svens Thoughts
If you only have 30 seconds to spare, here is what I would consider actionable insights for investors, data leaders, and data company founders.
Data Mesh is optional in a way. There are also a lot of improvements to be made to existing data teams as they stand in their centralized form.
Data Teams tend to create monoliths & dependencies. The whole concept of a “DAG” is essentially a huge system of dependencies, something that would be broken down into a lot of independent pieces by any software architect in the software world.
We can apply microservices & domain-driven design to the data world. But if we want to take the central data team for now as given, we need to make a few modifications.
Since the images are pretty essential, I opted to post this piece on my blog! So please just visit the link below!
Data as Code – Cutting Things Smaller – Datacisions & Three Data Point Thursday
Domain-Driven Design and microservices have changed the way software engineers work. And I think, they can be used to multiply the productivity of data teams as well. But I also think, they have to be used slightly differently because the common centralized data team is in a very different situation than the common (decentralized) software…
I hope you enjoyed this piece!
Leave feedback if you have an opinion about the newsletter! So?
It is terrible | It’s pretty bad | average newsletter… | good content… | I love it, will forward!
Data; Business Intelligence; Machine Learning, Artificial Intelligence; Everything about what powers our future.
In order to unsubscribe, click here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue