May dbt Community update

Welcome to the dbt Community update, a monthly blog about everything happening in the dbt Community! In May we hosted an AMA, presented the Spring 2024 dbt Community Spotlight, hosted eight in-person meetups, and had a ton of great discussions (and fire memes) on our Slack channel. Are you ready for the recap? Let’s get started.

dbt Community Slack AMA

Each month we host a live Ask Me Anything event. This month Phoenix Jay, Dave Connors, and Jeremy Cohen hosted the AMA and discussed the importance of collaboration, data quality, and some of the newest features in dbt. Here’s a recap if you missed it. You can also check out the full video recording.

Uncovering SQL errors with unit testing

One of the standout features discussed in the AMA is unit testing. Jeremy expresses his excitement about how it's been implemented in dbt Cloud and dbt Core v1.8. But why is unit testing such a game-changer? Dave shares a compelling example from the Jaffle Shop project, where unit testing helped uncover a SQL error. By integrating unit testing into your data analytics process, you can catch errors early on, ensuring the accuracy and reliability of your results.

Exploring the dbt Mesh pattern

dbt Cloud customers have been buzzing about the dbt Mesh pattern mentioned by Jeremy. It tackles the challenge of cycles across models and projects. While it sounds complex, Jeremy and Dave find it to be an interesting problem to solve. The mesh pattern provides a solution for teams working on interconnected projects, enabling seamless collaboration and efficient data workflows.

Collaboration and data quality

Collaboration and data quality are the foundation of successful analytics projects. The speakers emphasize the need for well-maintained and documented datasets. Without the right documentation, valuable insights can slip through the cracks. Maintaining data products and ensuring their accuracy is crucial for making informed decisions. Don't overlook the power of good documentation and how it impacts the quality of your data-driven responses.

Data contracts for seamless integration

Data quality and team collaboration are seamlessly integrated into existing workflows through data contracts in dbt. Jeremy highlights their importance as a crucial aspect of implementing data mesh or decentralized structures. By establishing data contracts, you can provide consistent standards across teams, while also emphasizing the significance of data quality and team collaboration. It's a win-win situation for everyone involved.

Tackling data quality challenges

Data quality challenges plague data teams of all sizes. Poor data quality can lead to faulty insights and, ultimately, poor decision-making. The discussion highlights the growing interest in adopting data mesh or decentralized structures to address these challenges. The integration of data contracts allows teams to prioritize data quality and collaboration, making sure that the right data is available when and where it's needed.

Simplifying data transformation for complex problems

Data transformation can be complex, but it also presents an opportunity for tackling more intricate problems. Dave and Jeremy stress the significance of thinking about interfaces and governance early on in a dbt project. By considering these factors from the start, you can streamline your data analytics workflows and make sure that your team is set up for success. Don't let data transformation become a stumbling block. Embrace its potential for solving even the most challenging problems.

Enabling teams and addressing people problems

While tools are essential for data analytics, it's equally important to address people problems and enable other teams within your organization. Dave and Jeremy highlight the value of a strong data team that goes beyond using tools. By fostering collaboration, emphasizing transparent documentation, and providing the necessary support, you can empower other teams to leverage data effectively. Remember, it's not just about the tools, but the people who use them.

dbt preferences

When it comes to dbt preferences, Jeremy emphasizes the importance of explicit configuration and using SQL plus Jinja. Explicit configuration provides clarity and reduces ambiguity, enabling smoother collaboration. SQL plus Jinja offers a powerful combination for manipulating data, making it easier to perform complex transformations. Additionally, unit testing is highlighted as a valuable practice for teams of any size. While the dbt mesh pattern is more relevant for larger teams, it's crucial for all teams to think about interfaces and governance early on in their dbt projects.

Data contracts as future focus

The conversation concludes with a mention of contracts as a potential future topic for an AMA. Contracts hold great promise for streamlining data analytics workflows even further, emphasizing data quality, and facilitating collaboration across teams. Stay tuned for more exciting updates on this front.

Get ready for another AMA in June

Join us for next month’s AMA on June 27th at 4 pm EST. We have Alex Welch, director of data at dbt Labs, to answer questions and discuss more about BI tools, analytics trends, and the Semantic Layer. Register now to get the link to watch and join the #dbt-community-merge channel in Slack to participate live!

Spring 2024 dbt Community Spotlight

Every quarter, we highlight community members in the dbt Community Spotlight. These are individuals who have gone above and beyond to contribute to the community in a variety of ways. We're excited to present the Spring 2024 dbt Community Spotlight!

This round, we are featuring Johann De Wet, Tyler Rouze, Juan Manuel Perafan, Mariah Rogers, Yasuhisa Yoshida, and Safiyy Momen. Visit the Community Spotlight page to learn about their backgrounds, their plans to grow as leaders, and their experiences—both learning from others and from sharing their own knowledge.

If you’re interested in being selected for future rounds of the Community Spotlight, learn more about becoming a contributor.

Johann de Wet

I'm forever indebted to my manager, John Pienaar, who introduced me to both dbt and its community when I joined his team as an Analytics Engineer at the start of 2022. I often joke about my career before dbt and after dbt. Our stack includes Fivetran, Segment, Airflow, and BigQuery to name a few.

Prior to that, I was a business intelligence consultant for 16 years working at big financial corporates. During this time I've had the opportunity to work in many different roles from front end development to data engineering and data warehouse platform development.

The only two constants in my career have been SQL en Ralph Kimball's Dimension Modeling methodology...which probably makes me a bit partial to those.

Tyler Rouze

My journey in data started all the way back in college where I studied Industrial Engineering. One of the core topics you learn in this program is mathematical optimization, where we often use data files as inputs to model constraints on these kinds of problems! Since then, I've been a data analyst on both small and large teams, and more recently a consultant shepherding our firm's dbt-based projects towards success.

Since joining the dbt Community, I've spoken at the Chicago dbt Meetup, Coalesce (a milestone for my career!), dbt's Data Leaders Series, and even made open source contributions to `dbt-core`! It has been the joy of my career to be a part of this vibrant community.

Juan Manuel Perafan

Born and raised in Colombia! Living in the Netherlands since 2011. I've been working in the realm of analytics since 2017, focusing on Analytics Engineering, dbt, SQL, data governance, and business intelligence (BI).

Besides consultancy work, I am very active in the data community. I co-authored the book *Fundamentals of Analytics Engineering* and have spoken at various conferences and meetups worldwide, including Coalesce, Linux Foundation OS Summit, Big Data Summit Warsaw, Dutch Big Data Expo, and Developer Week Latin America.

I also love meetups! I am the founder of the Analytics Engineering Meetup and co-founder of the Netherlands dbt Meetup.

Mariah Rogers

I got my start in the data world helping create a new major and minor in Data Science at my alma mater. I then became a data engineer, learned a ton, and propelled myself into the clean energy sector. Now I do data things at a clean energy company and geek out on solar energy at work and at home!

I attended my first Coalesce virtually in 2021 when my former colleague Emily Ekdahl gave a talk about some cool things we'd been working on. She inspired me to propose a talk the following year, so I submitted two topics and, surprisingly, both were accepted! I ultimately chose to speak about Testing in dbt in New Orleans in 2022, and the community's reception of that talk continues to be a highlight of my career.

Yasuhisa Yoshida

I currently work as a data engineer at a startup called 10X. Specifically, I work with BigQuery to provide data marts for business users. Before using dbt, the queries for creating data marts were overly complex and lengthy, resulting in low data quality. With dbt, we have improved our process by breaking down queries into manageable parts, visualizing data lineage, and enabling easy creation of tests.

I am actively involved in the dbt community and share our insights on using dbt at #local-tokyo. Specifically, I shared our experiences with efficient metadata management using dbt-osmosis, and visualizing data quality using elementary.

Safiyy Momen

I've been in the dbt community for ~4 years now. My experience is primarily in leading data teams, previously at a healthcare startup where I migrated the stack. The dbt Community was invaluable during that time. More recently, I've built a product, Aero, that helps Snowflake users optimize costs with a Native extension. I'm exploring ways to automate analytics engineering workflows.

I've spoken at various meetups, including the New York dbt Meetup, on data warehouse cost optimization.

May dbt Meetups

In May, we hosted one meetup in North America in New York City. We hosted five meetups in EMEA in Copenhagen, Dubai, Berlin, Stockholm, and the Netherlands. And in APAC, we had two meetups, one in Melbourne and one in Sydney.

At our Stockholm dbt Meetup, organized with our partner Solita with 60 folks in attendance, we ran a Peer Exchange led by dbt Labs’s Kshitij Aranke. In the peer exchange, a newer format that you'll start to see in more Meetups, we broke out into three groups and discussed the topics of Embracing AI, Data Analytics at Scale, and Analytics Engineering Best Practices. Attendees shared their professional experiences, asked each other good questions, took actionable notes, and forged new relationships.

You can see photos from Stockholm, Melbourne, and Copenhagen meetups below!

dbt Community announcements

We’ll wrap up this month’s update with some of the exciting announcements that are regularly posted in our #announcements channel on Slack.

dbt Cloud Launch Showcase

On May 14th we had our dbt Cloud Launch Showcase virtual event. It was a jam-packed 90 minutes with executive keynotes, new product announcements, and demos that all centered around the theme of how dbt Cloud is helping teams deliver Data That Works. Check out the recap blog to learn more.

Snowflake Data Cloud Summit

We’ll be at Snowflake’s annual user conference June 3–6! You can connect with the dbt Labs team at booths #1327 and #2503. Data Cloud Summit has an agenda packed full of interesting talks and workshops, including seven dbt Labs sessions:

• Best practices for optimizing dbt models: selecting the correct warehouse size
• Welcoming stakeholders to the dbt party
• Building customer-facing data products on Snowflake and dbt Cloud
• Build, deliver and govern data products at scale using data mesh principles
• How Techstyle manages analytics complexity with data contracts and data mesh
• How Medtronic optimized and future-proofed pipelines architecture to save $1.4M
• Unlocking self-service on unstructured data with Omni

Databricks Data+AI Summit

We'll also be attending the Databricks Data+AI Summit 2024 from June 10-13. Join our team for insights on accelerating data workflows and improving reliability. Learn from practical demos and network with data leaders and professionals.

Don't miss the chance to advance your data strategy as there are three great ways to connect with us during the conference:

• Book a meeting with our team: Schedule a one-on-one session with dbt experts at Data+AI Summit. Gain tailored advice and insights to enhance your data strategy.
• Visit our booth at Data+AI Summit (booth #91): Stop by booth #91 and explore live demos, get answers to your data questions, and see our solutions in action.
• Looking for a break after the sessions? Enjoy an evening of fun, networking, and entertainment and our Data on the Rocks event.

Upcoming events

• June 27th - Register for our next Community AMA featuring Alex Welch, director of data at dbt Labs
• Ongoing Cloud Demo with Experts in North America, EMEA, and APAC-friendly times
• October 7-10th, 2024, Coalesce, by dbt Labs in Las Vegas and Online
• Add dbt Events to your calendar!

June dbt Meetups

We’ve got a busy month coming up with 13 in-person dbt meetups scheduled. If you’re looking for opportunities to learn with fellow members of the dbt Community, and have fun while doing so, join us at one of the sessions listed below:

🇳🇴 Oslo | Thursday, June 6th, organized by Glitni

🇧🇪 Belgium | Thursday, June 6th, organized by dataroots

🇨🇴 Medellín | Tuesday, June 11th, organized by Factored

🇺🇸 Chicago | Thursday, June 13th, organized by Analytics8

🇺🇸 Atlanta | Tuesday, June 18th, organized by Aimpoint Digital

🇹🇼 Taipei | Wednesday, June 19th, organized by community members Karen Hsieh, Laurence Chen, Allen Wang

🇺🇸 Philadelphia dbt Labs on dbt Meetup | Thursday, June 20th, organized by dbt Labs (from the ‘dbt Labs on dbt’ Meetup series)

🇪🇸 Barcelona | Thursday, June 20th, organized by Spaulding Ridge

🇧🇷 São Paulo | Monday, June 24th, organized by community members Bruno Souza de Lima and Thales Donizeti

🇨🇴 Bogotá | Wednesday, June 26th, organized by Factored

🇨🇦 Halifax | Wednesday, June 26th, organized by community member Esther Fraser

🇪🇸 Madrid | Thursday, June 27th, organized by Astrafy🇮🇪 Dublin | Thursday, June 27th, organized by dbt Labs (from the ‘dbt Labs on dbt’ Meetup series)

🇯🇵 Tokyo | Thursday, June 27th, organized by dbt Labs (from the ‘dbt Labs on dbt’ Meetup series)

There are so many exciting things going on in the dbt Community, and we can’t wait to see you all there! If you haven’t yet, join the community today.