We're extremely excited to announce our new feature in Secoda, which allows users to connect Secoda to dbt. This integration enhances our data catalog by automatically linking dbt model details, documentation, and run statuses with datasets in Secoda.

dbt is an amazing command-line tool that gives data analysts and engineers the ability to transform their data in their warehouse effectively. Chances are, if your team is a frequent user of dbt, your project has many models and people using those models to build dashboards. If this is the case, business users and frequent dashboard users are likely coming to the data team with questions about what models they should use, how to use them and if there’s any other information they should know. In many cases, there’s also an issue of people using the wrong models to make decisions. These kinds of things can keep data teams up at night. 

About Secoda

We built Secoda to solve these problems for data teams. Our goal is to increase the truth and transparency around company data. Our name comes from the first two letters from the term “searchable company data”. Because of this vision for our tool, building an integration to dbt quickly became one of our most requested features. We want to make sure business users are able to supplement their self service goals with information about what tables are most relevant and trustworthy. With the Secoda and dbt integration, teams can finally get simple data discovery for every employee at an affordable price. Teams can code their documentation and make it available to all employees through a simple search dashboard. With the dbt integration, we will automatically associate "dbt run" information with datasets in Secoda. The following information will be extracted from using the dbt cloud API and associated with the relevant dataset:

• Link to dbt model code
• dbt docs (will be put on the respective column/table description)
• dbt run status
• dbt run start/finish time
• Any downstream/upstream sources*
• Dataset last updated time*
• Dataset created at a time*

* Indicates that we already pull this information, but if the information from dbt is different it will provide a more holistic picture. For example, the owner that we pull from a Redshift table doesn't typically have an email associated with it, whereas dbt it will so it will be more accurate information.

As we continue to build integrations into Secoda, our goal is to create a holistic picture of the data stack and make it available to anyone looking for information. Instead of having multiple apps and tools open while trying to find the right information, Secoda can become a central source of truth about your team's data. Today, we integrate with most warehouses and BI tools. In the future, we plan on building integrations to Airflow, great_expectations, Amazon S3 and many other critical pieces of the data stack.

How the dbt Integration Works

With Secoda and dbt, you can verify modelled tables and manage them if they involve PII information. You can also enhance your dbt documentation with additional details like tags, owners and related tables. All details from your dbt docs are automatically transferred to Secoda, allowing you to document your data in code and allowing your business users to search for it through a simple, intuitive UI.

Creating a Holistic Data Stack

Additionally, Secoda allows you to connect to Slack, which dbt users can use to stay updated about new models, model changes and new documentation that other members are creating in dbt. This feature can help teams stay informed about the changes that are made in their infrastructure.

The integration with dbt enhances our ability to provide information on how the data should be interpreted, information on how the data is created and used and information on the frequency and types of updates to the data. This will help us make a more intuitive and relevant search experience for everyone who integrates dbt to Secoda.

Get Started Today

Our goal is to continue to simplify data discovery while trying to make the most intuitive data discovery platform for all users. To make things even more exciting, we're offering teams that connects dbt to Secoda a 20% off discount to the product. We're super excited to help data teams find clarity in the sea of data and are continuing to work towards integrations and features that can make data discovery simple.