Operational data warehouse sync

Reporting disagrees with itself because each dashboard queries a different system with a different definition of a customer, a sale, and a month.

1. 01

   Extract incrementally

   Watermark based extraction with change data capture where available, so a nightly run does not re-pull five years of history.

2. 02

   Land raw and immutable

   Source data lands untransformed first. Every downstream model can be rebuilt from raw without re-extracting.

3. 03

   Model in layers

   Staging, intermediate, and mart layers with tests at each boundary, so a broken assumption fails loudly at build rather than quietly in a dashboard.

4. 04

   Define metrics once

   Revenue, active customer, and churn defined in one place and consumed everywhere, which is the entire point of the exercise.

5. 05

   Test every build

   Uniqueness, not null, referential integrity, and freshness tests gate publication. A failed test blocks the mart rather than publishing bad numbers.

6. 06

   Alert on freshness

   Any source that stops arriving raises an alert before the morning dashboard shows a flat line.

• Raw layer is append only, so a bad transformation is never destructive
• Full lineage from dashboard number back to source row
• Personally identifying fields hashed or excluded at ingestion per your data policy
• Cost monitoring per model, because an unwatched warehouse bill is its own incident