How to use this checklist
Answer each check with evidence, not memory. Most of the evidence is already in your warehouse’s query history, your orchestrator’s logs and your BI tool’s usage statistics; collecting it is most of the work, and the answer usually follows.
Download the worksheet to record a finding, an owner and a priority against each check, or print this page and write on it. Start with the metric reconciliation checks if you only have a day: they touch every other area, and they are the ones the business feels.
There is no score at the end. A count of worrying answers does not tell you what any of them costs; the worked example after the checks shows how to turn findings into an ordered backlog instead.
25 checks in 5 areas. The worksheet has the same checks with columns for your finding, owner, priority and next action, and opens in Sheets or Excel. No sign-up needed.
Sources and pipelines
Everything downstream inherits whatever arrives here. These checks establish whether you know what lands, when, and what breaks when it does not.
1. Every source feeding the warehouse is listed, with a named owner.
- Evidence to collect
- The connectors, ingestion jobs and scheduled loads from the orchestrator and the ingestion tool, each with the person or team who answers for it.
- A worrying answer
- Pipelines found only by reading the scheduler, or sources nobody will put their name to. When one breaks, the first hour goes on finding out whose it is.
- Next action
- Write the source inventory: source, how it lands, schedule, owner, and the upstream team to call. Mark orphans explicitly rather than leaving them blank.
Finding: ______________________ Owner: __________ Priority: ______
2. Staleness on critical tables is detected by an alert, not by a stakeholder.
- Evidence to collect
- The latest load timestamp of each table behind a business-critical metric, against its expected schedule, and how the last three late loads were noticed.
- A worrying answer
- The data team hears about stale data from the people reading the dashboard. Every decision made in between was made on yesterday’s, or last week’s, numbers.
- Next action
- Add freshness checks with a threshold per critical model, routed to the owner from the inventory.
From: Data quality foundation
Finding: ______________________ Owner: __________ Priority: ______
3. Row volumes on critical sources are monitored and explained.
- Evidence to collect
- Daily row counts for each critical source over the last 90 days, and one day reconciled against the source system’s own count.
- A worrying answer
- Drops, plateaus or step changes nobody can explain. Events can keep arriving while losing an attribute, so totals look fine while a segment quietly disappears.
- Next action
- Add volume monitoring with thresholds on the critical sources, and reconcile one day end to end against the source system.
From: Data completeness incident
Finding: ______________________ Owner: __________ Priority: ______
4. A schema change at a source is caught before it reaches a report.
- Evidence to collect
- What happened the last time a source column was renamed, retyped or dropped: when it was noticed, by whom, and what it broke.
- A worrying answer
- Pipelines stayed green while a column arrived empty or under a new name. Silent nulls are worse than a failed job, because nothing tells anyone to look.
- Next action
- Add schema monitoring on the critical sources and agree with each source owner how changes are announced.
Finding: ______________________ Owner: __________ Priority: ______
5. For any pipeline failure, you can say what it broke downstream.
- Evidence to collect
- The last three failures from the orchestrator’s logs and, for each, the models and reports it affected, traced through lineage rather than recalled.
- A worrying answer
- Nobody can say what a failure broke, so either everything is re-run and checked by hand or nothing is and a report is quietly wrong.
- Next action
- Build lineage from source to report for the critical metrics, and include the affected reports in the failure alert.
Finding: ______________________ Owner: __________ Priority: ______
Warehouse cost and access
Cost and access answer the same underlying question — who is doing what in the warehouse — and both are hard to reconstruct after the fact.
6. Warehouse spend can be attributed to the jobs, people and tools that cause it.
- Evidence to collect
- A full month of query history with bytes or credits per query, grouped by identity and by destination table or report.
- A worrying answer
- Most of the spend sits under one shared service account or an unlabelled scheduler, so nobody can say which workload grew the bill.
- Next action
- Give each application its own service account or role, and label or tag jobs where they are defined.
From: Warehouse cost governance
Finding: ______________________ Owner: __________ Priority: ______
7. The largest scheduled jobs process only what changed.
- Evidence to collect
- The top ten scheduled jobs by bytes scanned or credits, with whether each is a full rebuild or incremental, and how often it runs.
- A worrying answer
- Large tables rebuilt in full on a frequent schedule. This is usually the biggest recurring line in the bill, and it grows with the data rather than with the business.
- Next action
- Convert the most expensive full rebuilds to incremental models, starting at the top of the list.
From: Where your warehouse spend actually goes
Finding: ______________________ Owner: __________ Priority: ______
8. What each report costs to refresh is known next to how often it is read.
- Evidence to collect
- Refresh cost per report from query history, joined to view counts per report from the BI tool for the last 90 days.
- A worrying answer
- Expensive reports at the bottom of the usage list, refreshing on a timer whether anyone opens them or not.
- Next action
- Retire what is unused and slow the refresh of what is read rarely, before tuning anything.
From: Warehouse cost per query: SQL for your own history
Finding: ______________________ Owner: __________ Priority: ______
9. Someone sees warehouse cost daily, and a spike raises an alert.
- Evidence to collect
- Whether a daily cost table, dashboard or budget alert exists, whether quotas are set, and how the last cost surprise was discovered.
- A worrying answer
- The invoice is the alert, or a hit quota stops reporting before anyone knew consumption had grown.
- Next action
- Write daily cost by consumer into a table, put one chart on top, and alert above a threshold.
From: Warehouse cost governance
Finding: ______________________ Owner: __________ Priority: ______
10. Write and admin access to the warehouse is limited, reviewed and not tied to people.
- Evidence to collect
- Every user and service account with write or admin rights, when the list was last reviewed, and which scheduled jobs run on a person’s credentials.
- A worrying answer
- Jobs running as an individual, former staff still holding access, or broad write rights granted once for convenience. The first breaks when that person leaves; the others are a risk nobody has priced.
- Next action
- Move jobs to service accounts, reduce write access to the roles that need it, and put a recurring access review in the calendar.
Finding: ______________________ Owner: __________ Priority: ______
Transformation and testing
This is where a number becomes a definition. The checks ask whether that logic is visible, tested and changed safely.
11. All transformation logic is in version control and deployed one way.
- Evidence to collect
- The repository, how changes reach production, and a count of scheduled queries, views or warehouse procedures defined only in a console.
- A worrying answer
- Logic living in console-scheduled queries or one-off views, where nobody reviews a change and nobody can see its history.
- Next action
- Bring it into the repository, starting with whatever feeds a business-critical metric.
Finding: ______________________ Owner: __________ Priority: ______
12. Models are layered, and reports read from the final layer.
- Evidence to collect
- The lineage graph, and which reports query raw or staging tables directly instead of the models built for them.
- A worrying answer
- The same join or business rule written several times in several places, which guarantees the copies drift apart.
- Next action
- Agree a layering convention (staging, intermediate, marts or equivalent) and move the critical reports onto the final layer first.
From: Analytics engineering
Finding: ______________________ Owner: __________ Priority: ______
13. Models behind critical metrics have tests, and failures reach an owner.
- Evidence to collect
- Tests on each model in the critical path — unique and not-null keys, relationships, accepted values — and the failure history for the last month.
- A worrying answer
- No tests on the models that matter most, or tests that have been failing for weeks and are ignored. An ignored test is worse than none, because it trains everyone to skip the alert.
- Next action
- Add key and relationship tests along the critical path, fix or delete the permanently failing ones, and route failures to a named owner.
Finding: ______________________ Owner: __________ Priority: ______
14. Changes are compared against production before they ship.
- Evidence to collect
- How the last few model changes were tested, and whether development output was compared with production output for the models they touched.
- A worrying answer
- Changes go straight to production and are checked by whoever notices a report move. A fix that looks right can still change totals elsewhere.
- Next action
- Make a development-against-production comparison part of review for changes to critical models.
From: Seven-figure revenue attribution gap
Finding: ______________________ Owner: __________ Priority: ______
15. Critical models are documented well enough for someone new to use them.
- Evidence to collect
- The share of critical-path models with a description, column definitions and an owner, and how long the last new joiner took to answer a metric question unaided.
- A worrying answer
- The definitions live in one or two people’s heads. That is a single point of failure for every number the business uses.
- Next action
- Document the critical-path models first: what each row is, what each metric column means, and who owns it.
Finding: ______________________ Owner: __________ Priority: ______
Metric reconciliation
The audit’s core move: follow the numbers the business acts on from source to dashboard cell until every difference is proven rather than suspected.
16. The three to five numbers the business acts on are named, each with one definition.
- Evidence to collect
- The metrics leadership actually uses for decisions, the decision each one feeds, and every place each is defined.
- A worrying answer
- No agreement on which numbers matter, or one metric with two definitions in two tools. Every meeting then starts by arguing about whose number is right.
- Next action
- Write one definition per critical metric, with an owner, and say which report is the official one.
Finding: ______________________ Owner: __________ Priority: ______
17. Each critical metric reconciles from source system to model to report.
- Evidence to collect
- The same metric for the same period at three points: the source system, the warehouse model, and the dashboard.
- A worrying answer
- Differences nobody can explain, or that everyone has stopped noticing.
- Next action
- Follow the metric layer by layer until the exact divergence point is located, and prove which layers are clean.
From: Reconciliation under pressure
Finding: ______________________ Owner: __________ Priority: ______
18. Every divergence is classified as a real business difference or a bug.
- Evidence to collect
- For each known difference: its cause (timing, refunds, currency, cut-off, or an error) and whether it is written down anywhere.
- A worrying answer
- Gaps closed with a filter until the number matches. A filter that makes a report agree can be the thing that makes it wrong.
- Next action
- Document the accepted differences next to the metric definition, and raise every bug with the team that owns the layer where it happens.
From: Reconciliation under pressure
Finding: ______________________ Owner: __________ Priority: ______
19. Unattributed values are visible and shrinking, not absorbed.
- Evidence to collect
- The share of revenue, orders or events with a null, unknown or other value in each key dimension, month by month.
- A worrying answer
- A growing unknown bucket, or one quietly folded into a real category so the report looks complete.
- Next action
- Label unattributed values honestly so they stay visible, and trace the largest bucket back to where the attribute is lost.
From: Seven-figure revenue attribution gap
Finding: ______________________ Owner: __________ Priority: ______
20. When a metric moves, you can tell a business change from a data change.
- Evidence to collect
- The last two unexpected movements in a critical metric, and how long it took to establish whether the business or the data had changed.
- A worrying answer
- Reactions to metric movements before anyone checks the data, or a real problem dismissed as a data issue. A metric falling is not the same as the thing it measures falling.
- Next action
- Keep a short runbook for an unexpected movement: check freshness, volume and recent model changes first, then the business explanation.
From: Measuring what a bug actually cost
Finding: ______________________ Owner: __________ Priority: ______
BI ownership and usage
The reporting layer is where trust is won or lost. These checks ask what exists, who owns it, and whether anyone uses it.
21. You know which dashboards are used.
- Evidence to collect
- An export of every dashboard and report with views and distinct viewers over the last 90 days.
- A worrying answer
- Most of the estate unviewed. Every unused report still costs refreshes, maintenance and attention when its logic changes.
- Next action
- Agree a retirement process: archive what has no viewers, with a notice period and a way back.
From: BI and reporting
Finding: ______________________ Owner: __________ Priority: ______
22. Every used dashboard has a named owner.
- Evidence to collect
- The owner of each report viewed in the last 90 days, and whether that person still works there.
- A worrying answer
- Reports owned by people who have left, or by “the data team”. Nobody decides when such a report is wrong, changed or retired.
- Next action
- Assign an owner to every used report, or archive it.
Finding: ______________________ Owner: __________ Priority: ______
23. Business metric logic lives in models, not in dashboards.
- Evidence to collect
- Reports that compute a business metric in custom SQL or calculated fields rather than reading it from a model.
- A worrying answer
- The same metric calculated differently in two dashboards, which is how two reports end up answering the same question with different numbers.
- Next action
- Move metric logic down into the models and have reports read the result.
Finding: ______________________ Owner: __________ Priority: ______
24. People know which reports to trust.
- Evidence to collect
- Which reports are marked official or certified, whether stakeholders can name them, and how often numbers are exported to spreadsheets to be fixed.
- A worrying answer
- Stakeholders keeping their own spreadsheet copies because the dashboard is not believed. That is the cost of lost trust made visible.
- Next action
- Certify a small set of reports built on the final model layer, and say so in the tool where people look.
From: BI migration programme
Finding: ______________________ Owner: __________ Priority: ______
25. Stakeholders hear about data quality problems from the data team first.
- Evidence to collect
- How the last three data incidents were discovered and communicated, and whether any quality measure is published regularly.
- A worrying answer
- Stakeholders find problems before the data team does. Each one costs more trust than the incident itself.
- Next action
- Track a small set of quality KPIs on the critical models and agree how incidents are announced.
From: Data quality foundation
Finding: ______________________ Owner: __________ Priority: ______
Worked example: from findings to a backlog
Illustrative, not a client. An invented team of four analysts on a cloud warehouse with dbt and a self-serve BI tool, working through the checklist in a week. The findings below are invented to show the method; none of them is a measurement from a real engagement.
Each finding gets an impact and an effort, judged by the team, not computed. High impact and low effort goes first; anything that depends on another fix waits for it; large projects nothing in the findings points at are deferred. There is deliberately no score: a count of worrying answers does not tell you what any of them costs.
| # | Finding | Checks | Impact | Effort | Next action |
|---|---|---|---|---|---|
| 1 | Revenue on the finance dashboard differs from the billing system every month by roughly the refunds issued after month end. | 17, 18 | High | Low | Classify it as a real timing difference, write it next to the revenue definition, and stop re-investigating it monthly. |
| 2 | The nightly orders load and two other schedules run on the credentials of an engineer who has left. | 10 | High | Low | Move them to a service account this week, before the account is closed and the loads stop. |
| 3 | The orders table was stale twice last quarter and both times a sales manager noticed first. | 2, 25 | High | Low | Add freshness and volume checks on the orders models, alerting the pipeline owner. |
| 4 | Revenue models have no tests, and two dashboards compute revenue in their own custom SQL. | 13, 23 | High | Medium | Add key and relationship tests, then point both dashboards at the revenue model. |
| 5 | Three full rebuilds account for most scheduled scan volume. | 7 | Medium | Medium | Make the largest one incremental first and measure the change before doing the other two. |
| 6 | Most dashboards had no viewers in 90 days. | 8, 21 | Medium | Low | Announce and archive them, with a month’s notice and a way to restore. |
| 7 | The team had proposed moving ingestion to a new tool. | 1, 5 | Low | High | Defer. Nothing found points at the ingestion tool; revisit after the items above. |
The first three take days, not weeks, and remove the most risk. The proposed ingestion migration drops to the bottom: nothing in the findings says the ingestion tool is the problem, which is the kind of conclusion that stops a rebuild nobody needed.
Email me the checklist
Optional. You get a link to this page and the worksheet, then the monthly newsletter you can leave at any time. No sequence, no drip campaign.
When an outside review is useful
Most teams can run this checklist themselves, and should. An outside review earns its cost in a narrower set of situations: when the people who would run it built the stack, when the answers are politically awkward, or when a decision as large as a rebuild depends on them.
- You have inherited the stack and nobody who built it is left to ask.
- A rebuild has been proposed and you want an independent read before committing to it.
- Two teams disagree about where a number goes wrong, and each is sure it is the other’s layer.
- You have the findings but not the time to reconcile the critical metrics end to end.
- The data stack audit: this checklist, run for you in two weeks on read-only access, ending in a ranked fix backlog