This is a fantastic piece. Truly. I’ve been struggling to understand the point of the “semantic layer” and the real world example you gave of counting up the devices is super helpful. As an aside, and this obv isn’t your problem or a criticism of your piece at all, but I still feel lost as to the utility of doing all that work because it still doesn’t solve the problem of Mr. Engineer wanting to use a totally different source to answer the question. Like, if the data team has their own warehouse and the fulfillment team has their own warehouse and the org can’t get on the same page as to which one makes sense as the right source for a count of devices, why would a semantic layer sitting on top of the data team’s warehouse solve that problem? Is the idea that the business user would stop asking Mr. Engineer questions like that altogether because pivoting out the answer in their BI tool themselves is truly “self service”?
The way you describe why a semantic layer exists by breaking it down into parts really resonates with me. It reminds me of ThoughtSpot's built-in semantic layer, aka a Worksheet. It's a light weight semantic layer, since context is only given to tables in the worksheet with a particular join, but it's what ultimately powers all of the liveboards. And that's what helps ThoughtSpot call themselves self-service; end-users are actually querying from that curated semantic layer. :)
This is a fantastic piece. Truly. I’ve been struggling to understand the point of the “semantic layer” and the real world example you gave of counting up the devices is super helpful. As an aside, and this obv isn’t your problem or a criticism of your piece at all, but I still feel lost as to the utility of doing all that work because it still doesn’t solve the problem of Mr. Engineer wanting to use a totally different source to answer the question. Like, if the data team has their own warehouse and the fulfillment team has their own warehouse and the org can’t get on the same page as to which one makes sense as the right source for a count of devices, why would a semantic layer sitting on top of the data team’s warehouse solve that problem? Is the idea that the business user would stop asking Mr. Engineer questions like that altogether because pivoting out the answer in their BI tool themselves is truly “self service”?
I’m writing something about data standardization as well — but a little bit more unhinged — as always, great post
The way you describe why a semantic layer exists by breaking it down into parts really resonates with me. It reminds me of ThoughtSpot's built-in semantic layer, aka a Worksheet. It's a light weight semantic layer, since context is only given to tables in the worksheet with a particular join, but it's what ultimately powers all of the liveboards. And that's what helps ThoughtSpot call themselves self-service; end-users are actually querying from that curated semantic layer. :)