Semantic layer

dbt MetricFlow - semantic layer 2.0

dbt is doubling down on the semantic layer and this time it’s serious.

Johan Baltzar
Johan BaltzarOctober 2023


dbt’s first go at a semantic layer fell short of data community expectations, they wisely decided to reboot with MetricFlow and this time it’s looking very good folks. If you skipped the previous version (dbt 1.5 and earlier), now is the time to give it another look.

At Steep, we are convinced that the semantic layer is the future of BI and business analytics. So when dbt - the de facto standard for data transformation - was also pushing the semantic layer idea, we were thrilled. Being able to define metrics in code right next to all your other data models is an appealing concept indeed, and we could see that serious data teams would go all-in on this setup.

Unfortunately the first version that dbt launched in late 2022 failed to live up to the hype. A lot of teams looked at it but not enough found it worth their time. Two big issues was the lack of joins and a not-so-strong reusability of definitions.

To dbt lab’s credit they realised this and quickly changed course. Instead of their in-house developed v1 spec, they moved to acquire Transform, the team behind open source project MetricFlow, and decided to go all-in on MetricFlow as the new dbt standard (1.6 and on).

We’ve been working with MetricFlow since summer and we like it a lot folks. Kudos to the Transform team for the great work. If you are curious about semantic layer and like your definitions in code - this new spec is likely worth your time.

Here's what a basic example looks like:

  - name: orders
      agg_time_dimension: ordered_at
    description: |
      Orders fact table. One row per order.
    model: ref('orders')

      - name: order_total
        description: The total revenue for each order.
        agg: sum
      - name: cancellations
        description: Total canceled amount
        agg: sum

      - name: country
        type: categorical
      - name: is_food_order
        type: categorical
      - name: ordered_at
        type: time
          time_granularity: day 
  - name: cancellation_rate
    type: ratio
      numerator: cancellations
      denominator: order_total

Things we like about MetricFlow

  • Support for both semantic models (measures, dimensions) and metrics
  • Easy metric definitions with less boilerplate
  • Dynamic joins (entity relationships)
  • Generally a well-structured spec

Things we still want

  • Segments/slices - saved filters for (non-trivial) segments is helpful for end users. A nice feature that is both in Cube and Steep.
  • Time grain restrictions - for some bespoke metrics, say MRR, it can be useful to control exactly which time grains are allowed. Right now MetricFlow only supports specifying the minimum allowed time grain.

MetricFlow and Steep

We believe that having all your models and metric definitions in dbt and publishing your metrics with Steep is a great combo. That’s why you'll be able to connect Steep directly to your dbt project and use MetricFlow for your semantic layer. With this setup, the role of the data team can truly be to build the data model and the semantic layer, and ship the metrics catalog to all users. No more repetitive dashboard work!

The road ahead

The semantic layer field is moving fast and becoming much easier to work with end-to-end. In my view, it would be good for all of us to work towards a standardization here. Just like you can learn SQL once and use it all your career, it would be great if we all can learn one general way to do semantic layer.

The fact that MetricFlow is actually pretty close to Cube spec is great and an important step toward that standardization. And whatever dbt bets on will have a lot of impact. Next week Coalesce 2023 is happening and I look forward to continued discussion with the main players and the community at large.


More posts

Data culture

Analytics for startups

If you're running a startup and thinking about your data setup - good news! Getting started is not that hard anymore, and you don't need a data specialist. This is how we did it at Steep.Read more...

Johan Baltzar
Johan BaltzarNovember 2023
Product update

New in Steep: Tables

This month, we’re introducing powerful and intuitive tables that work like you expect in 2023.Read more...

Nino Höglund
Nino HöglundNovember 2023
Data culture

When building a data culture - start with getting teams focused on impact

One of the big frustrations I’ve had as a data leader is working with teams that are too busy shipping to care about impact and analysis. Here’s my approach.Read more...

Johan Baltzar
Johan BaltzarNovember 2023
Data culture

Why data leaders have the hardest job in tech

I’ve been an analytics leader three times over, and my friends - we sure didn't choose this job because it’s easy!Read more...

Johan Baltzar
Johan BaltzarOctober 2023
Product update

New in Steep: Search, caching and more

Wondering what we’ve been working on lately? Check out the latest updates below. Read more...

Nino Höglund
Nino HöglundOctober 2023
Inside Steep

How we ended up building a Notion for data

The other day Janne, one of the developers on the team, turned around and said - oh I just got totally mixed up, I thought I was in Notion but I was actually in Steep!Read more...

Johan Baltzar
Johan BaltzarSeptember 2023
Product update

New in Steep: Next-level analysis

This month we’re introducing more ways to analyze, combine and visualize your metrics, all in one place.Read more...

Nino Höglund
Nino HöglundSeptember 2023
Company update

The new Steep

The product has come a long way in two years, the mission remains the same.Read more...

Johan Baltzar
Johan BaltzarSeptember 2023
Product update

New in Steep: Reports reimagined

This month we are bringing you a whole new way to create, collaborate and communicate with data - introducing the new Reports.Read more...

Nino Höglund
Nino HöglundJune 2023
Product update

New in Steep: Metrics catalog and joins

This month we are introducing new ways of bringing order to your metrics and more flexibility defining metrics with joins. Read on for more. Read more...

Nino Höglund
Nino HöglundMay 2023
Product update

New in Steep: dbt integration and more

Steep has partnered up with dbt to provide a best-in-class metrics integration. But that’s not all. Read on for more exciting updates.Read more...

Nino Höglund
Nino HöglundApril 2023
Product update

New in Steep: Team spaces, Help center

This month we are making it easier to grow with Steep using Team spaces, launching a completely new Help center, as well as adding a few nice visualization improvements.Read more...

Nino Höglund
Nino HöglundMarch 2023
Product update

New in Steep: Demo mode, Mac app and much more

Trying out Steep is now easier than ever with the introduction of demo data. There are plenty of new features this month, so make sure you read to the end for a special treat 🍰Read more...

Nino Höglund
Nino HöglundFebruary 2023
Product update

New in Steep: Rank, compare, custom templates

The year is coming to an end and we are wrapping up with a whole bunch of new features and improvements. Our focus has been on enabling even more teams to be able to use Steep as well as new ways of gaining insights from your data. A big thanks to all of you for this year! Let's get to the news.Read more...

Nino Höglund
Nino HöglundDecember 2022
Product update

New in Steep: New edit mode, permissions

Early last month marked the public release of Steep. Thanks to everyone who signed up and got started so far. Please keep the feedback coming! We have not been idle in the meanwhile and there is a a whole bunch of news to share - so let's get right to it.Read more...

Nino Höglund
Nino HöglundNovember 2022

Announcing Steep

Im very glad to share the next step in our journey with Steep - announcing that we raised €1M pre-seed round co-led by Alliance and Inventure, and that today we are launching Steep to general access!Read more...

Johan Baltzar
Johan BaltzarOctober 2022
© Steep Analytics
Stockholm, Sweden
We don't use cookies