In this tutorial, we will explore the Run Visibility feature in dbt Cloud, which provides a dashboard for monitoring your project's job runs and allows you to rerun jobs as needed. This feature can help you identify areas for improvement in your scheduled jobs and keep track of your project's health.
What is Run Visibility in dbt Cloud?
Run Visibility is a dashboard in dbt Cloud that displays a project's job run history, including job name, status, environment, trigger, commit SHA, schema, and timing information. It allows users to review recent runs, find errored runs, track the progress of runs in progress, retry jobs from the start or failure point, and receive notifications about job statuses. It can be accessed by clicking Deploy and then Run history in the top navigation menu.
How To Access the Run History Dashboard
To access the Run History Dashboard in dbt Cloud, follow these steps:
1. Navigate to the Run History Dashboard
Click on the "Deploy" tab in the top navigation menu, and then click on "Run history." This will take you to the Run History Dashboard, where you can view your project's job runs.
2. Review and Filter Job Runs
On the Run History Dashboard, you can see a list of job runs with information such as job name, status, associated environment, job trigger, commit SHA, schema, and timing info. Use the filters at the top of the dashboard to focus on specific aspects of your project, such as runs with a particular status or runs within a certain time frame.
3. Monitor and Track Job Progress
Keep an eye on the progress of your job runs by checking their status on the dashboard. You can see if a run is in progress, has completed successfully, or has encountered an error. Secoda's dbt integration and Slack notifications can also help you stay updated on the status of your jobs.
How To Rerun a Job in dbt Cloud
If you need to rerun a job, follow these steps:
1. Go to the Job's Run Page
From the Run History Dashboard, click on the job run you want to rerun. This will take you to the job's Run page.
2. Click Rerun
In the upper right corner of the Run page, click the "Rerun" button. A dropdown menu will appear, allowing you to choose between "Rerun from start" and "Rerun from failure."
3. Choose Rerun Option
Select either "Rerun from start" to rerun the entire job or "Rerun from failure" to rerun only the failed steps. If you choose "Rerun from failure," a Rerun failed steps modal will open, listing the run steps that will be invoked, including the failed step and any skipped steps.
Common Challenges and Solutions
While using Run Visibility in dbt Cloud, you may encounter some common challenges:
- Identifying errored runs: Use the filters on the Run History Dashboard to quickly find runs with a "Failed" status.
- Understanding the cause of a failure: Check the error messages and logs on the job's Run page to help diagnose the issue.
- Receiving too many notifications: Adjust your notification settings in dbt Cloud or Slack to reduce the frequency of updates.
Best Practices for Monitoring and Rerunning Jobs
To make the most of Run Visibility in dbt Cloud, follow these best practices:
- Regularly review your job runs to identify patterns and areas for improvement.
- Use filters on the Run History Dashboard to focus on specific aspects of your project.
- Monitor your project's health with Secoda's dbt integration and Slack notifications.
- Address failures promptly by diagnosing issues and rerunning jobs as needed.
- Optimize your notification settings to stay informed without being overwhelmed.
