Last Mile of Analytics, Encrypted Spark, Barr Moses; ThDPTh #71

I’m Sven, and this is the Three Data Point Thursday. The email that helps you understand and shape the one thing that will power the future: data. I’m also writing a book about the data mesh part of that. 

Time to Read: 4 minutes

Another week of data thoughts:

• There is a last mile of analytics, and it’s not what you think.

• Encrypting spark computations is possible and doable

🐰🐰🐰🐰🐰🐰🐰🐰🐰🐰🐰🐰🐰🐰🐰

The Last Mile of Analytics

What: Taylor Brownlow, Product Lead at Count, writes on the importance of what she calls the “last mile of analytics”. I think the piece can be nicely summed up by the title “the last mile of analytics can make or break your startup”,  and the following quote: 

“The Last Mile of Analytics refers to the space between the data team and the business teams; ”

My perspective: I love that they're focusing on this, and it is great that the whole data cycle gets attention. She isn’t focusing on technical stuff but also turns to the business side. 

But really, I am not sure the focus on this particular spot is right for most companies. 

Just because the “last mile” didn’t get much attention before, doesn’t mean solving the last mile is the most important step. 

The data => action pipeline is really that, a pipeline. So whatever step is the bottleneck, well, is the bottleneck. Only if the last mile truly is the bottleneck, it should be the priority for a company. But if you’re getting on well, then a company has other problems to solve inside this pipeline.

The second thing is, that the “last mile of analytics” actually isn’t the space between the data team and the business teams. That assumes that there is a central data team and that the business teams automatically implement the “great analytical output”. 

Instead, the last mile of analytics is the step of going from information => to insights & decisions & actions. And that’s a pretty long mile! What Taylor is talking about, this “space” is much wider than what she describes. 

Take for instance the central data team Taylor has in mind, that team creates some kind of information, dashboards, a churn prediction system, a recommendation API, anything like that.

The last mile for the recommendation API is then for the product org to pick the topic up, use the data across channels and develop user-facing components which use it with the “business teams”. 

The last mile for the dashboard is for the “business teams” to discover them, use them in their decision-making process, come up with some smart insights, make decisions and then put them into action, possibly even include tracking there to validate a possible hypothesis.

However, while I don’t think there is such a “last mile of analytics”, I do like the advice Taylor puts out which is very much about focusing on what I’d call good product management for data teams. 

Resource: https://blog.count.co/the-last-mile-of-analytics-can-make-or-break-your-startup/

Encrypted Spark

What: This is a great talk by Gidon Gershinksy, from the Weizmann Institute about an open-source spark framework meant to run computations via spark on encrypted datasets.  

My perspective: As I’ve talked about in my last Thoughtful Friday, I do like the idea of simply encrypting everything in analytical stores. It just makes things easier and more secure. I find his talk pretty fascinating as well as his analysis on the speed bottlenecks, which from a technical perspective sounds like speed isn’t an issue at all, give or take a bit of crypto-algorithm magic. His tests show an increase in computational time of about 10%, which sounds like almost nothing to me.

Resource: https://databricks.com/de/session/efficient-spark-analytics-on-encrypted-data

Barr Moses & Matt Turck

What: Barr goes from explaining data downtime to data observability as it is inspired by software engineering. She explains how the idea of observability is to determine the health of a system by its outputs.

My perspective: I love how she goes through the experience of “data downtime” because I’ve lived through that again and again. The sweating when you get the call “hey, this is completely wrong, the CEO is really pissed”, I’m already reliving very unfun moments! If you’re interested in these concepts, I recommend this short discussion.

Resource: https://mattturck.com/montecarlo/

🎄 Thanks => Feedback!

Thanks for reading! I’d also love it if you shared this newsletter with people whom you think might be interested in it.

And of course, please provide me with feedback:

It is terrible | It’s pretty bad | average newsletter… | good content… | I love it!