Senior AI & Data Engineer
Well Experienced Data Warehouse Craftsman
I design, build, and operate end-to-end data pipeline (Ingestion, Transformation, and Serving) at scale.
Confluent Certified Data Streaming Engineer · GCP Certified Data Engineer
Projects
All projects →01
Crypto Tracker - Local Data Pipeline
An end-to-end local data pipeline: crypto and FX data ingested with dlt, warehoused in DuckDB, modelled with dbt into a medallion + star schema, orchestrated by Airflow 3.
dlt ingests CoinGecko and Frankfurter APIs into a DuckDB landing zone; dbt builds bronze trust-filter views, silver cleaned staging, and a gold star schema (dims, facts, marts). One Airflow 3 DAG runs ingest-then-build daily through a single-slot pool that serializes DuckDB writes. No API keys, no cloud account.
- data-engineering
- python
- SQL
- Airflow
- dbt
- DuckDB
- dlt
Writing
Everything →January 2025
Tired of stacking UNION ALL, have you tried BigQuery GROUPING SETS?
This article explores BigQuery's GROUPING SETS clause as a more concise and readable alternative to complex UNION ALL queries for generating aggregations at multiple levels of granularity.
- bigquery
- sql
- data-engineer
- data-analysis
January 2025
BigQuery “NOT IN” vs “NOT EXISTS” vs “EXCEPT DISTINCT” understand the differences in 3 minutes!
This article compares three BigQuery operators—NOT IN, NOT EXISTS, and EXCEPT DISTINCT—for excluding data from query results, using a real-world debugging scenario involving bot traffic exclusion.
- bigquery
- sql
- data-engineer