A new era of data engineering: dbt Copilot is GA

Generative AI is redefining how engineers work. In fact, Gartner® predicts that, "by 2028, 90% of enterprise software engineers will use AI code assistants, up from less than 14% in early 2024."¹ Today, dbt is stepping boldly into that future with dbt Copilot, an AI-powered data assistant now generally available in dbt Cloud.

This is a defining moment for AI in data engineering. One that will not only change how data teams work but redefine the work itself.

AI demands high quality inputs

At its core, data engineering has focused on cleaning, structuring, and optimizing raw data to deliver reliable business outcomes. Since the shift to the cloud, dbt has long streamlined these processes, and now with GenAI, we’re entering the “AI era” for data engineering teams.

In this new era, these once routine data engineering tasks are now automated, freeing teams to focus on strategic innovation and building the advanced AI systems of tomorrow. As the pressure for faster AI innovation intensifies, organizations must integrate GenAI into their workflows to stay ahead. Supporting this shift, "Gartner estimates that AI software spend will grow to $297 billion by 2027–representing an annual growth rate of 19%."²

The problem is generic LLMs focus solely on code, missing the deep context required to build truly robust data pipelines. For LLM outputs to be useful, it has to be accurate—and the stakes for data quality have never been higher.

With dbt Copilot, you’re not just injecting GenAI into your analytics workflows; you’re leveraging the full context of your data—its relationships, metadata, and lineage—to automate routine tasks and consistently uphold key ADLC best practices like documentation, data testing, semantic modeling, and SQL formatting. The result is a refined, governed dataset that serves as a solid foundation for building high-quality analytics and advanced AI systems.

dbt is the only solution uniquely positioned to supply your LLM with this level of deep metadata context.

For example, if your data warehouse uses a column named customer_id, a generic LLM might generate code referencing just id. dbt Copilot avoids these mistakes by tailoring outputs to your actual schema.

“dbt Copilot is the future of data engineering. If you aim for truly high-quality, well-governed data—or are building AI systems that depend on trustworthy inputs—you need an AI solution that operates at the critical juncture where data is refined and contextualized. With dbt Copilot, you're laying the foundation for next-generation AI and driving true data-powered success.”
— Mark Porter, CTO at dbt Labs

Embed context-aware AI into every workflow

With dbt Copilot now integrated into your analytics workflows, we're modernizing data engineering and empowering teams to deliver high-quality analytics and AI innovations faster than ever before.

Your day-to-day tasks as a data engineer can be executed more efficiently. It’s like having a dedicated data intern that standardizes legacy documentation, improves query optimization, checks for SQL syntax errors, enhances metadata compliance, and speeds up migrations, all within dbt Cloud. Our beta users are already cutting hours of tedious work, standardizing processes faster, and unlocking greater value from every data asset.

As Cody McLean, Sr. Data Engineer at Hard Rock Digital put it,

"dbt Copilot has completely changed how we approach documentation and query optimization. Instead of spending hours manually updating models, I can use natural language to generate tests, infer metadata, and enrich our data models with valuable context. The more metadata we add, the better our entire team benefits, from analysts to executives.”

Today with dbt Copilot, data teams can: auto-generate documentation, data tests, semantic models, metric definitions, and inline SQL using natural language prompts, all within the dbt Cloud IDE.

By making standardization and YAML development effortless, dbt Copilot ensures your models are secure, rigorously tested, and built to the highest standards—empowering teams to work faster and focus on what truly matters.

And now with support for Open AI Bring Your Own Key (BYOK) service, Azure OpenAI service, and a custom style guide (in beta), dbt Copilot is even more flexible, secure, and enterprise-ready.

How our customers are using dbt Copilot today

Since the beta launch at Coalesce 2024, hundreds of customers have experienced firsthand how dbt Copilot transforms their data workflows. Users are streamlining outdated documentation, enriching metadata, and tapping into dozens of use cases that improve data quality and team efficiency. Here are a few common use cases:

Standardize legacy documentation and enhance metadata enrichment

Writing model descriptions and column definitions can take hours, especially for legacy data where documentation may be missing or outdated. With a click of a button, dbt Copilot contextually generates YAML-based documentation leveraging SQL logic, past queries, and metadata, so teams can instantly improve clarity and maintainability. dbt Copilot helps decipher cryptic column names, making it easier to work with legacy data and maintain consistency across models.

Accelerate data testing

Rather than manually crafting each test case, dbt Copilot uses the context of your dbt models—understanding dependencies, transformations, and schema relationships—to suggest context-aware validation tests. With the click of a button, it adds the corresponding test code directly to your project—ready to run during your builds. This not only speeds up the process but also helps teams detect schema drift by flagging unexpected changes in column structures before they impact production.

Improve formatting and query optimization

Messy SQL slows everyone down. With natural language prompts, dbt Copilot generates SQL inline and then leverages a built-in style guide to ensure it’s formatted correctly—eliminating inconsistent casing, indentation, and redundant syntax. This saves time in code reviews and improves maintainability. dbt Copilot can also refactor legacy queries and suggest cleaner, more efficient SQL, making it easier to optimize older code.

Automate semantic layer and metric definitions

Defining business metrics across teams is often inconsistent. Based on existing data models, dbt Copilot recommends useful key metrics aligned with business objectives to be used for your semantic models ensuring analysts and stakeholders are aligned on consistent definitions.

"My CFO doesn’t want to see a dashboard—she wants direct answers, like our ARR for the quarter. With dbt Copilot, she can instantly query well-defined metrics using conversational language so she can get the answers she needs instantly and my teams can get out of the business of building dashboards. The future of analytics is AI-powered insights and governed metrics via Semantic Layer, not dashboards. As these tools evolve, organizations that embrace them will lead the way."
— Josh Carlson, Director of Analytics at Code42

What’s new in dbt Copilot

With the general availability of dbt Copilot, we’re introducing new features designed to give users greater control while making collaboration even easier:

OpenAI Bring Your Own Key (BYOK) and Azure OpenAI service

By default, dbt Copilot comes with an integrated OpenAI service, making it incredibly easy to get started. For enterprise users seeking additional control, you can opt to bring your own OpenAI key (BYOK).

Additionally, dbt Copilot now integrates with the Azure OpenAI Service, allowing organizations in the Azure cloud to seamlessly incorporate AI into their workflows. All of these options are built to enterprise-grade standards, ensuring a simple experience while ensuring reliable performance.

Custom style guide for standardized SQL formatting (in beta)

SQL consistency is critical for maintainability, and dbt Copilot now includes a custom style guide to enforce best practices across dbt models. Teams can configure style preferences, reducing manual code reviews and ensuring consistency across projects.

The future of AI in dbt

Integrating AI into data engineering isn’t optional anymore—it’s essential. With dbt Copilot, you’re not just adapting to change; you’re setting a new standard for data management.

The general availability of dbt Copilot is just the beginning. As AI continues to evolve, dbt Copilot will become more intuitive, proactive, and aligned with how modern data teams operate across the entire analytics development lifecycle.

dbt Copilot is available to dbt Cloud Enterprise customers, please contact your sales representative to request access. Get started with dbt Copilot today and experience the future of AI-assisted analytics engineering.

References:

1. Gartner, Magic Quadrant for AI Code Assistants, Arun Batchu, Philip Walsh, Matt Brasier, and Haritha Kjandabattu, August 19, 2024. (Accessible to Gartner subscribers only)
2. Gartner, Forecast Analysis: AI Software Market by Vertical Industry, 2023-2027, Anna Griffen and Ina Agamirzian, March 27, 2024. (Accessible to Gartner subscribers only)

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.