What's new in dbt Cloud - June 2024

It's that time again...settle in for the latest regular installment of our product announcements blog post 🌞. We've had a very busy couple of months, including hosting our inaugural dbt Cloud Launch Showcase where we announced dozens of new features: an AI-assist co-pilot, a visual editor, unit testing, automatic exposures, and much more! Some features are still in private beta, and trust that you'll hear from us when they're ready for prime time. For now, we've consolidated all of the latest features you can get your hands around today—including features that we didn't announce at the Showcase—in one easy-to-read blog post. Let's dive in 👇!

🔎 dbt Explorer

Our vision is for dbt Explorer to be the best place for data teams to discover, understand, and improve their dbt Cloud projects so downstream teams can leverage trusted data assets with confidence. Here’s what’s new:

✅ dbt Explorer is GA (generally available): The foundational dbt Explorer experience—including column-level lineage, model performance analysis, and project recommendations—is now generally available! Get started by clicking the “Explore” tab in dbt Cloud.

📊 Column-level lineage now includes new features like a lineage lens to view when columns are transformed, icons to identify primary keys, and propagating descriptions for reused columns. 

🎦 Support for staging environments: In addition to production environments, dbt Explorer now supports staging environments! Staging environment support is also available for cross-project refs through dbt Mesh. This makes it easier to understand pre-prod/QA state and catch issues before they hit production, while allowing users to bolster their data isolation and governance posture.

💻 Azure support: dbt Explorer is now generally available for customers running dbt Cloud on Azure single tenant.

📝 Open in IDE: Enjoy more cohesive and streamlined developer workflows by jumping directly from dbt Explorer into the dbt Cloud IDE to edit a resource.

📈 Performance and search improvements: We continue to invest in the underlying experience to deliver improved performance for large lineage graphs, including faster load times and a new default loading state. We also now support more dbt selector methods and you can find auto-suggested selectors in the lineage search bar.

📈 dbt Semantic Layer

Check out what’s new in the dbt Semantic Layer, and peruse customer FAQs here:

🗄️ Declarative caching: Save relevant queries to “pre-warm” the cache and significantly improve the performance of key dashboards or common ad-hoc query requests while also reducing compute costs for frequently-queried metrics. Declarative caching is now GA for all Semantic Layer customers.

🌎 Integrations: As part of the GA of our Tableau integration, you can now find and install the dbt Semantic Layer directly from the Tableau Exchange. Also, our product experts recently wrote a blog post all about how you can improve data change management in Tableau using the semantic layer.

Additionally, our Google Sheets integration is now GA. You can find all of the dbt Semantic layer integrations here.

🔢 Metrics as dimensions in MetricFlow: MetricFlow is the powerful SQL query generation tool behind our semantic layer, and we continue to make improvements to it to make your workflows more streamlined and flexible. Case in point: with metrics as dimensions, you can use the value of another metric in your metric definition. For example, say you want to count “activated accounts,” which is defined as (1) an account (2) with more than five log-ins to your platform. To express this metric in SQL, you’d first write a query to calculate the number of log-ins per account, then count the number of accounts who have with more than 5 log-ins. Now, you can do this natively in MetricFlow by adding metrics as filters to other metrics! Read the docs to learn more.

🌐 dbt Mesh

dbt Mesh, a pattern for collaboration at scale in dbt Cloud, is now GA. It enables teams to make use of multiple, inter-connected dbt projects, each aligned to a domain — boosting collaboration without compromising governance.

New capabilities include:

👉 Trigger on job completion, across projects: Get even more more flexibility in how you deploy your dbt models into production. Read the docs to learn more.

🎭 Support for canonical staging environments: Improve data isolation and build in dbt Cloud without access to production data. Read the docs to learn more.

☁️ Azure support: dbt Mesh is now generally available for customers running dbt Cloud on Azure single tenant.

✏️ Develop

We shipped lots of exciting improvements to both the dbt Cloud CLI and IDE. Check 'em out!

✅ The dbt Cloud CLI is now GA: Develop anywhere using your code editor of choice, bolstered by dbt Cloud, including capabilities like dbt Mesh support, defer to production, and improved performance. Other features at GA include:

• 💪 Support for dbt Power User: If you use VS Code, you can now also use the Power User for dbt Core and dbt Cloud extension with the dbt Cloud CLI to bolster your productivity.
• ☁️ Azure support: Additionally, the dbt Cloud CLI is now available to organizations running dbt Cloud on Azure single tenant.

🔢 Unit testing is now GA: Use unit tests to validate the behavior of model logic before the model is materialized with real data. If a test fails, the model won’t build—saving you from unnecessary data platform spend, while improving data product reliability.

🌱 Prune branches in the IDE: Using this Git button, you can delete local branches that have been deleted from the remote repository, keeping your branch management tidy. Available in all regions now and will be released to single tenant accounts during the next release cycle.

🎋 Git branch as an environment variable: You can now reference your current Git branch as an environment variable, allowing you to do things like dynamically use the Git branch name as a prefix for a development schema.

🧹 Support for SQLFluff v3 in the IDE. In addition to other benefits, you’ll now get better feedback on .sqlfluff configuration errors directly in the dbt Cloud UI as logs and toasts.

⚠️ Better notifications around invocation failures in the IDE. Now, when an invocation fails, the IDE will surface a prominent notification banner above the system log, making it easier to immediately see when a job has failed

👍 Other Cloud IDE improvements: You can now make changes to multiple projects at the same time, which is really helpful for users operating in a mesh, and we’ve also made improvements to our backend Cloud IDE file system to improve overall performance.

🔄 Deploy

Ship pipelines faster, and more reliably, with these workflow improvements.

🔀 Merge jobs. Immediately trigger a job to run when a pull request is merged, and enjoy native functionality for continuous deployment (CD) in dbt Cloud. Coupled with deferral, you can be sure the latest data is always reflected in production…without driving up data platform spend.

🛑 Job deactivation: Runs with repeated failures are automatically deactivated so they don’t continue to run and fail indefinitely. They can be easily reactivated by editing a deactivated job.

💻 Platform improvements

The team is always hard at work to make dbt Cloud more performant, reliable, and interoperable.

💫 Keep on latest version: dbt Cloud should feel and function like the other SaaS apps your team uses: you shouldn’t have to manually upgrade versions under the hood. Now generally available, just select “Keep on latest version” in your environments and jobs to get immediate access to the latest and greatest functionality in dbt. Going forward, this is how we will be delivering dbt to our customers: reliably and continuously. Read our recent blog post for more!

⏰ Parse time improvements. We’ve also made optimizations under the hood to significantly improve parse performance in dbt Cloud. These are available today to everyone running on “Keep on latest version.”

🔒Databricks OAuth: Now generally available, dbt Cloud supports developer OAuth with Databricks, providing an additional layer of security for dbt Enterprise users.

👭 Partnerships

It's been an exciting few weeks on the partnership front!

❄️ Snowflake native app: dbt is now available on the Snowflake Marketplace as a native app! The dbt for Snowflake Native App brings dbt Cloud’s discovery and semantic capabilities to Snowflake's robust, governed architecture. Now, the dbt Cloud experience extends directly to the Snowflake UI, allowing users to jump in and gain insights from their dbt projects with one Snowflake login. Moreover, you can use your Snowflake committed spend to pay for dbt Cloud and sign on Snowflake paper (no additional vendor approvals required! 🙌). Read the blog post to learn more.

🗣️ Ask dbt: Now in open beta for dbt users on the Snowflake native app, we also launched an AI chatbot designed to help users get trusted answers to their questions, faster—without writing a single line of SQL. Ask dbt combines the power of a Snowflake Cortex LLM with the dbt Semantic Layer to translate natural language questions into a semantic query. Check out the blog post to learn more.

👋 Microsoft adapters: In addition to the GA of our Microsoft Fabric adapter, dbt Cloud now supports Microsoft Azure Synapse Analytics (in Preview). To get started, create a new dbt project in dbt Cloud and choose Fabric or Synapse as your data platform.

Wrapping up

We're so excited to get these features in your hands and as always, look forward to hearing your feedback. Until next time!