How to Manage Multiple Projects in dbt Data Teams

Published
May 22, 2024
Author

What are multiple projects in dbt data teams?

Multiple projects in dbt data teams refer to different data models and resources that are defined and used across an account. These projects can be viewed in the account, providing a better understanding of cross-project resources. The resource-level lineage graph for a given project displays the cross-project relationships in the Directed Acyclic Graph (DAG).

  • Multiple projects: These are different data models and resources used across an account in dbt data teams.
  • Resource-level lineage graph: This is a visual representation of the cross-project relationships in the DAG.
  • Directed Acyclic Graph (DAG): It is a concept in mathematics and computer science that represents a set of directed edges with no directed cycles.

What is the significance of upstream and downstream projects?

Upstream projects are producer projects that are depended upon by downstream or consumer projects. Public models in an upstream project display a counter icon indicating how many downstream projects depend on them. When viewing a downstream project, public models from upstream projects show up in the lineage graph, indicating the relationship to a model from another project.

  • Upstream projects: These are producer projects that are depended upon by downstream projects.
  • Downstream projects: These are consumer projects that depend on upstream projects.
  • Public models: These are data models that are available for use across different projects.

How can you explore the project-level lineage graph?

For cross-project collaboration, the DAG can be interacted with at the project level. A list view of all the projects can be obtained by selecting the account name at the top of the Explore page. The project-level lineage graph can be viewed by clicking the Lineage view icon. If you have permissions for a project, you can view all public models used across the entire account.

  • Project-level lineage graph: This is a visual representation of the relationships between different projects.
  • Lineage view icon: This is an icon that, when clicked, displays the project-level lineage graph.
  • Public models: These are data models that are available for use across different projects.

What functionalities are available from the project-level lineage graph?

From the project-level lineage graph, you can view the cross-project lineage graph, view the project list, switch to a project’s main Explore page, view a model’s details page, perform searches on your projects, and switch to a particular project’s lineage graph. When a project node in the graph is selected, a project details panel opens where you can view counts of the resources defined in the project, view a list of its public models, and view a list of other projects that use the project.

  • Cross-project lineage graph: This is a visual representation of the relationships between different projects across an account.
  • Project list: This is a list of all the projects defined in an account.
  • Public models: These are data models that are available for use across different projects.

How can you share the project panel link?

You can share the project panel link by clicking the Share icon. This copies the project panel link to your clipboard, allowing you to share the graph with someone.

  • Share icon: This is an icon that, when clicked, copies the project panel link to your clipboard.
  • Project panel link: This is a link that directs to the project panel of a specific project.
  • Clipboard: This is a temporary storage area for data that the user wants to copy from one place to another.

Keep reading

See all