🐰#21 SQLFluff, Data Orchestrators, Decoupling BIs; ThDPTh #21 🐰
🐰#21 SQLFluff, Data Orchestrators, Decoupling BIs; ThDPTh #21 🐰
Finally, a good SQL linter, how to choose a data orchestrator, and why decoupling makes the BI world better.
Data will power every piece of our existence in the near future. I collect “Data Points” to help understand & shape this future.
If you want to support this, please share it on Twitter, LinkedIn, or Facebook.
(1)🔥 SQLFluff, Finally a Good SQLLinter
Finally, there’s a SQL linter that actually might make the cut. Almost all SQL linters fail on two problems, the variety of SQL dialects, say hello to RedShift & Snowflake, and the ability to lint templated, usually Jinja, SQL.
SQLFluff at least got these two problems nailed and seems to be in a good shape. So if you haven’t found your SQL linter yet, I suggest you check out SQLFluff. They even got an online linter you can try out.
GitHub - sqlfluff/sqlfluff: A SQL linter and auto-formatter for Humans
A SQL linter and auto-formatter for Humans
(2)🎁 Comparing Data Orchestrators
I find it pretty hard to navigate between all these data orchestrators/processing frameworks like airflow, prefect, dagster, and all the others. So I really enjoyed the article by Kapernikov which takes a step back, evaluates these tools, and above all provides a good framing around them.
Even though all of these frameworks already evolved quite a bit since the article was written last year, it still helped me take the bird’s eye view on all these tools.
Oh and to add just one thing: There’s really only one thing all these tools have in common, they build & manage DAGs. And then some.
A comparison of data processing frameworks – Kapernikov
Data, industry 4.0 & prototyping
(3)😍 Decoupling In the BI World
I’ve already touched on this topic in my newsletter on building evolving data architectures, so I really enjoyed another company taking up the topic of decoupling in the BI world.
I really enjoyed the ideas the company GoodData shares with respect to bringing more software engineering to this world, so go take a look at the article.
Here is What Happens If You Decouple Your BI Stack | Medium
The field of Business Intelligence has significantly evolved over the last decades.
🎄 In Other News & Thanks
Thanks for reading this far! I’d also love it if you shared this newsletter with people whom you think might be interested in it.
P.S.: I share things that matter, not the most recent ones. I share books, research papers, and tools. I try to provide a simple way of understanding all these things. I tend to be opinionated. You can always hit the unsubscribe button!
Data; Business Intelligence; Machine Learning, Artificial Intelligence; Everything about what powers our future.
In order to unsubscribe, click here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue