Projects

Built a production-grade ELT pipeline on Microsoft Azure, transforming the Olist Brazilian E-Commerce dataset (~100K orders, 9 relational tables) from raw CSV files into an analytics-ready Kimball star schema using the Medallion Architecture.

• Ingested 9 CSV files (125MB) from Azure Blob Storage into Azure SQL Database via two methods — a Python script using SQLAlchemy with Microsoft Entra DefaultAzureCredential (zero hardcoded credentials), and an Azure Data Factory pipeline with ForEach, Filter, and Copy Data activities using Managed Identity authentication.
• Transformed raw data through a Medallion Architecture (Bronze → Silver → Gold) using dbt Core with dbt-sqlserver adapter — staging models for cleaning and casting, mart models implementing Kimball dimensional modelling with incremental materialisation and upsert logic via MERGE statements.
• Implemented 46 automated dbt tests (unique, not_null, relationships, composite key validation via dbt_utils) with GitHub Actions CI/CD running dbt run and dbt test on every push, pull request, and manual trigger — branch protection rules enforce only tested code reaches production.

Built an end-to-end churn modelling workflow to help subscription-style businesses understand and predict which customers are likely to leave, using a Telco-style dataset.

• Cleaned and preprocessed 7,000+ customer records, handling missing values, encoding categorical fields and scaling numerical features.
• Performed EDA to uncover relationships between tenure, contract types, billing patterns and churn behaviour.
• Improved model performance from a baseline of ~70% to ~82% through feature engineering, tuning and cross-validation, with a focus on business-relevant metrics.

Designed a command-line–driven ETL engine to automate payroll data workflows, integrating CSV and SQL sources into a single, reproducible pipeline.

• Implemented a structured flow covering extract → DDL creation → transform → load, reducing manual preparation time from hours to minutes.
• Built repeatable, auditable runs with a scripted pipeline design (configurable and reproducible).

I build interactive Tableau dashboards that combine clear KPIs, drill-down exploration, and a short “so what” insight layer. Recent work includes a Tour De France dashboard (real data), Victoria Road Crash analytics dashboard (real government open data) and a Global Graduate Employability dashboard (synthetic dataset for demonstration).

How I approach projects

• Start from a clear business question — for data pipelines, identify the grain, entities, and measures before writing code; for ML models, define the prediction target and success metric before touching the data.
• Use targeted EDA to shape decisions — not just to generate charts, but to understand data quality issues, feature relationships, and what the model or pipeline actually needs to handle.
• Design for the full stack from day one — raw ingestion, transformation layers, data quality tests, and CI/CD for pipelines; preprocessing, feature engineering, cross-validation, and business-relevant evaluation metrics for ML.
• Preserve source data as an immutable truth layer and own all transformation logic in version-controlled code — not in ad-hoc scripts or database procedures.
• Enforce quality automatically — schema tests and referential integrity for data pipelines; confusion matrices, precision/recall tradeoffs, and cross-validation for ML models — evaluated against what matters to the business, not just a single accuracy score.
• Make deliberate architectural tradeoffs and choose the right tool for each layer — document the reasoning so the next engineer understands why decisions were made, not just what was built.
• Treat security as a first-class concern — zero hardcoded credentials, Managed Identity where possible, and branch protection rules so only tested code reaches production.
• Document assumptions, known limitations, and what would change at scale — for future iterations, deployment, or the next person picking up the work.