Data Sources
Azure Blob Storage
Australia East
Python Script
SQLAlchemy + pandas
Azure Data Factory
No-code pipeline
Medallion Architecture — Azure SQL Database (Serverless)
🥉 Bronze Layer
Raw Ingestion Layer
Raw Tables (raw schema)
- Exact copy of source data
- No transformations applied
- loaded_at timestamp added
- Full reload on every run
Key Features
- Source of truth preserved
- 9 tables ingested
- BOM character handled
- latin-1 encoding fix
🥈 Silver Layer
Cleansed & Standardised Layer
dbt Staging Models (9 tables)
- Clean & rename columns
- Cast data types correctly
- transformed_at timestamp
- Geolocation deduplication
Key Features
- unique + not_null tests
- Composite key validation
- dbt_utils test framework
- schema.yml documentation
🥇 Gold Layer
Business Ready Layer
Kimball Star Schema
- fact_orders (factless fact)
- fact_order_items
- fact_order_payments
- fact_reviews
Key Features
- Incremental materialisation
- Upsert via MERGE statement
- dim_customers + geolocation
- Referential integrity tests
GitHub Actions — dbt run + dbt test on every push, PR & manual trigger 23+ workflow runs · Branch protection enforced 46 Tests Passing · Across Staging and Mart Layer Consumers
BI Dashboards
Tableau / Power BI
Analytics Teams
Self-service SQL
AI / ML Use Cases
Clean training data
Reporting & Alerts
Automated insights
Low Cost
Entire project build