0.1 Coming soon Expected Summer 2026
Probability You Can See: Densities, Sampling & Expectation
Random variables, densities, and the idea that an expectation is just an average over samples โ the Monte-Carlo lens we use everywhere.
Generative AI: Diffusion & Flow Models from First Principles
๐ // generative ai series ยท 2 of 20 modules live
A build-from-scratch journey through modern generative modeling which includes diffusion, score matching, and flow matching, derived from first principles, implemented in code, and made interactive so intuition sticks. This is a deliberately learner-first reordering: we build the probability you need first, go DDPM-first before any SDEs, live in 1-D/2-D where you can see the math, and arrive at flow matching as the payoff. It doubles as the backbone of a lecture I'm giving in Fall 2026.
// part 0 ยท Foundations you can see
0.2 Gen-AI Jun 13, 2026 ยท 2 min read
Gaussians & the Reparameterization Trick
The Gaussian is the workhorse of generative modeling. Here's everything you need โ mean, covariance, and the one trick (x = ฮผ + ฯฮต) that makes diffusion trainable.
0.3 Coming soon Expected Summer 2026
Conditioning, Joints & Marginals
How distributions combine and decompose: joint densities, conditioning (slicing) and marginalizing (averaging) โ the grammar of probabilistic models.
0.4 Coming soon Expected Summer 2026
Measuring Distance Between Distributions: Likelihood & KL
Likelihood, cross-entropy, and KL divergence โ why fitting a distribution means minimizing KL, and what that buys us.
0.5 Coming soon Expected Summer 2026
Change of Variables & the Score
Transforming densities (the Jacobian and pushforwards) and a first, geometric look at the score gradient of log-density โ the compass diffusion follows.
0.6 Coming soon Expected Summer 2026
A Little Dynamics: ODEs, SDEs & Flows
Velocity fields and Euler integration, then an SDE is an ODE plus noise โ the minimal dynamics needed to flow a density from noise to data.
// part 1 ยท The core idea โ discrete & concrete
1.1 Gen-AI Jun 13, 2026 ยท 2 min read
Diffusion from First Principles: From Data to Noise and Back
The one idea behind every diffusion model โ gradually destroy structure with noise, then learn to undo it โ with an interactive forward-diffusion explorable to build intuition.
1.2 Coming soon Expected Fall 2026
The Forward Process & Noise Schedules
The Gaussian Markov chain, the closed-form forward marginal, and how the noise schedule controls how fast structure is destroyed.
1.3 Coming soon Expected Fall 2026
Learning to Denoise: the DDPM Objective
The reverse Gaussian, predicting the noise epsilon, and the simplified training loss โ why a denoiser is all you need, from scratch.
1.4 Coming soon Expected Fall 2026
Sampling: Your First Generations
Ancestral sampling from noise to data, your first generated samples on 2-D toys (and MNIST), and what the reverse trajectory looks like.
// part 2 ยท The continuous view โ score & SDEs
2.1 Coming soon Expected Fall 2026
Score Matching
Denoising score matching and Tweedie's formula โ why predicting noise is the same as estimating the score, with a visual of the learned score field.
2.2 Coming soon Expected Fall 2026
From Markov Chains to SDEs
The forward VP/VE SDE, Anderson's reverse-time SDE, and how DDPM is just a discretization of a continuous process.
2.3 Coming soon Expected Fall 2026
The Probability-Flow ODE & Fast Samplers
The deterministic ODE with the same marginals as the SDE โ DDIM, sampler choices, and the step-count vs quality trade-off.
// part 3 ยท Flow matching โ the unifying view
3.1 Coming soon Expected Fall 2026
From CNFs to Flow Matching
Continuous normalizing flows and the likelihood bottleneck, then the flow-matching idea: just regress a velocity field.
3.2 Coming soon Expected Fall 2026
Conditional Flow Matching
Conditional probability paths and vector fields, and the marginalization trick that makes flow matching trainable without simulating the flow.
3.3 Coming soon Expected Fall 2026
Diffusion as a Special Case; Rectified Flows
Recovering diffusion from the flow-matching lens, and straightening paths (rectified flow / OT) for faster, cleaner sampling.
// part 4 ยท Making it real
4.1 Coming soon Expected Fall 2026
Conditioning & Classifier-Free Guidance
Conditioning on prompts and labels and the classifier-free guidance trick โ plus the quality-vs-diversity trade-off you can dial.
4.2 Coming soon Expected Fall 2026
Architectures & Latent Diffusion
Time embeddings, U-Nets and Diffusion Transformers, and why moving to a VAE latent space made high-resolution generation practical.
4.3 Coming soon Expected Fall 2026
Numerics, Evaluation & Failure Modes
Samplers, EMA, numerical stability, and how we actually measure generative models (FID, likelihood) โ plus the failure modes to watch for.
4.4 Coming soon Expected Fall 2026
Discrete Diffusion & Scientific Applications
Diffusion beyond pixels: continuous-time Markov chains for language, and diffusion for molecules and protein structure.