Graduate Courses

Workload

32 h per course
Lectures
20 h

5 modules × 4 h

  • Theoretical foundations and core concepts
  • Distributed architecture design principles
  • Guided technical exercises
Labs
8 h

1 hands-on lab per module (1.5-2 h)

  • Practical engineering assignments
  • Collaborative code and architecture reviews
Final Project
4 h

1 capstone workshop session

  • End-to-end production data pipeline deployment
  • Technical project guidance and review

Language & Tooling

Bilingual
Instruction
French & English

Teaching delivery

French Lectures English Technical Terms Bilingual Q&A
Materials
English

Course deliverables

Slides Code Repositories Assignments Exams

Assessment

4 Components
Final Exam
50%
Module Homework
20%
Final Project
15%
Attendance & Participation
15%
  • Data Engineering 2

    Fall 2025-2026

    • Master’s Year 2
    • 12 students

    The second-year sequel, moving from building a pipeline to running one that survives contact with production failure modes.

    Module 1: Synchronous & Asynchronous Data Flows

    • Flow model trade-offs: setting blocking request-response side by side with message queues to evaluate coupling vs. latency tolerance.
    • Downstream failure propagation: tracing how backpressure and queue growth prevent record loss under heavy load.
    • Lab: the same ingestion pipeline implemented twice (blocking HTTP vs. decoupled queue), then stalled under synthetic load to compare record loss against processing latency.

    Module 2: Resilient Pipeline Design & Idempotency

    • The core idempotency principle: Airflow guarantees that a task can be retried; your data model and transaction strategy must guarantee that the retry is safe.
    • Safe retry mechanics: enforcing unique business key UPSERTs, staging-table MERGE patterns, and transactional delete-before-write over date partitions.
    • Orchestration vs. Idempotency: separating Airflow DAG retry configurations from database-level transactional guarantees.
    • Lab: a multi-task Airflow DAG authored with task-level retries, exponential backoff, and idempotent UPSERTs, then made to fail mid-run and repaired with a dated backfill over the missing window.

    Module 3: Processing at Volume with Apache Spark

    • Distributed processing mechanics: running Apache Spark transformations against PostgreSQL operational sources.
    • Query plan inspection: reading Spark execution plans via EXPLAIN to identify shuffle stages and disk spill bottlenecks.
    • Lab: a join-and-aggregate transformation written in Spark against PostgreSQL source tables, refactored once the execution plan revealed shuffle bottlenecks, and the execution times compared.

    Module 4: Logging, Monitoring & Observability

    • Structured error tracking: embedding contextual task identifiers and execution IDs using Loguru.
    • Operational monitoring: building real-time dashboards and threshold alert rules in Datadog.
    • Lab: a pipeline instrumented with structured Loguru records shipped to Datadog, with an automated failure alert triggered and verified under synthetic error injection.

    Module 5: Packaging & Delivery

    • Environment containerization: packaging Python data pipelines into reproducible Docker images.
    • Data & artifact versioning: tracking datasets and intermediate models using DVC.
    • CI/CD automation: building automated workflows with GitHub Actions to push images into the GitHub Container Registry.
    • Lab: a pipeline packaged into a Docker image with DVC data tracking and a GitHub Actions workflow that builds, tests, and publishes containers on main-branch commits.

    Final Project: Fault-Tolerant Ingestion Pipeline

    • Technical Architecture & Data Flow: end-to-end ingestion pipeline carrying data from a PostgreSQL operational source through a Spark transformation into an analytical target, scheduled as an Airflow DAG with retries and a working backfill path.
    • Operability, Containerization & Monitoring: structured Loguru logging surfaced on a Datadog dashboard, the complete pipeline containerized with Docker, and the input dataset versioned under DVC.
    • Written Report & Oral Defense: single-command execution repository accompanied by an architecture diagram and oral defense of retry, timeout, and alert threshold choices.
  • Data Engineering 1

    Spring 2024-2025

    • Master’s Year 1
    • 16 students

    The first-year data engineering course: how analytical data is stored, moved, and processed at a volume that no longer fits one machine.

    Module 1: Big Data Fundamentals & Memory Limits

    • Single-machine boundaries: identifying the memory limit where single-machine processing fails and distributed tooling becomes necessary.
    • In-memory vs. chunked execution: comparing whole-dataset memory loading against partitioned chunk processing.
    • Lab: the same aggregation run over a dataset held whole in memory versus one read and partitioned in chunks, timed on both to locate the single-machine memory threshold.

    Module 2: Data Warehousing & Dimensional Modeling

    • OLTP vs. OLAP architectures: why analytical storage is structured differently from transactional operational databases.
    • Star schema modeling: defining facts, conformed dimensions, surrogate keys, and explicit grain in PostgreSQL.
    • Lab: a star schema designed from a normalised operational source and created in PostgreSQL, then queried to measure analytical query performance gains.

    Module 3: ETL Architectures & Safe Transformations

    • Pipeline stages: extracting, cleaning, conforming, and loading data from operational systems to warehouses.
    • Transformation logic: authoring robust ETL routines in Python with atomic database operations.
    • Lab: a Python ETL job extracting from an operational database, conforming records, and loading them into a PostgreSQL star schema without introducing duplicate rows.

    Module 4: Processing at Volume with Apache Spark

    • Distributed compute paradigms: distributed data processing using Apache Spark DataFrame abstractions.
    • Scalability comparison: evaluating distributed Spark compute against single-machine Python execution.
    • Lab: rewriting Python ETL transformations into Spark DataFrame operations against larger extracts to identify the crossover point where distributed compute pays for itself.

    Module 5: Analytics & Reporting

    • Analytical serving: connecting operational warehouses to BI tools for business reporting.
    • Semantic measure modeling: building Power BI reports over conformed star schemas using modeled measures rather than visual calculations.
    • Lab: Power BI connected to a PostgreSQL star schema with a dashboard constructed over conformed dimensions and explicit semantic measures.

    Final Project: Source-to-Dashboard Warehouse

    • Technical Architecture & Data Flow: end-to-end data pipeline from an operational source to a published Power BI dashboard, featuring Python extraction, Spark transformations, and PostgreSQL serving.
    • Operability & Data Modeling: star schema implementation with explicit grain definitions, surrogate keys, and atomic transformation pipelines.
    • Written Report & Oral Defense: delivery of schema definitions, ETL code, and oral defense of dimensional modeling decisions (grain, keys, and normalization trade-offs).
  • Machine Learning

    Fall 2024-2025

    • Master’s Year 2
    • 12 students

    The programme’s applied machine learning course, taken from stating a deterministic domain problem as a learnable one to deciding whether the result means anything or the experiment has to be redesigned.

    Module 1: Supervised Learning & Imbalanced Classification

    • The frame, and what makes it hard: the ULB card fraud frame carrying 284,807 transactions with 492 frauds (0.17% positive class).
    • The accuracy trap: demonstrating why a naive baseline predicting zero fraud reaches 99.83% accuracy, requiring evaluation via F1 score.
    • Feature engineering under constraint: working with 28 principal component columns where Time and Amount are the primary features available for transformation.
    • Estimator trade-offs: evaluating Decision Trees, Random Forests, and Logistic Regression in Scikit-learn against data assumptions.
    • Lab: twelve students in three teams, fitting one estimator per team (Decision Tree, Random Forest, Logistic Regression) over shared Pandas preparation in Google Colab, scoring accuracy vs. F1 across the room.

    Module 1 Homework: Fraud Detection Benchmark

    • Individual Colab Challenge: an individual assignment in Google Colab where dataset and evaluation metrics are fixed while feature selection, estimator choice, and hyperparameter tuning are free.
    • The three levers: feature selection/preparation, estimator selection, and hyperparameter optimization.
    • Submitted as: a commented notebook carrying the best accuracy and F1 score reached, reproducible directly from the notebook.
    • Oral argument: defending which levers moved the score and which parameters had negligible effect.

    Module 2: Gradient Descent & Optimization (From Scratch)

    • First-principles optimization: learning gradient descent as the underlying mechanism of model fitting rather than a library call.
    • Convergence parameters: analyzing learning rates, loss surfaces, and parameter update steps.
    • Lab: gradient descent implemented from the derivative upward in pure NumPy, tracing loss curves across learning rates until observing stall or divergence.

    Module 3: Unsupervised Learning & Clustering (From Scratch)

    • Unsupervised representation: discovering latent structure without ground-truth labels.
    • K-Means mechanics: assignment step, centroid update rule, and selecting optimal k via inertia and silhouette analysis.
    • Lab: K-Means implemented by hand in NumPy (assignment, centroid update, convergence test) and validated against Scikit-learn.

    Module 4: Model Evaluation & Diagnostic Curves

    • Generalization validation: cross-validation techniques that separate training fit from generalization performance.
    • Diagnostic visualization: evaluating confusion matrices, precision-recall trade-offs, and Matplotlib learning curves.
    • Lab: a classifier cross-validated on an imbalanced target, with confusion matrices and Matplotlib learning curves used to diagnose underfitting vs. overfitting.

    Module 5: The Practical Workflow & Model Export

    • End-to-end integration: pulling data from PostgreSQL, preparing features in Pandas, fitting Scikit-learn pipelines, and evaluating metrics.
    • Reproducibility & persistence: exporting fitted pipeline artifacts alongside evaluation metadata.
    • Lab: an end-to-end Python workflow script (query ➔ prep ➔ fit ➔ evaluate) with model artifacts and metrics persisted together.

    Final Project: Applied Prediction Study

    • Technical Architecture & Data Flow: supervised prediction study carried from a PostgreSQL extract through Pandas feature preparation into a tuned Scikit-learn model beating a naive baseline.
    • Operability & Pipeline Tuning: cross-validated hyperparameter search over scikit-learn pipelines with persisted model metrics.
    • Written Report & Oral Defense: notebook delivery featuring Matplotlib evaluation figures, defended on whether empirical results support the final conclusion.