Skip to content

Subham Sahoo

Simple and Effective Masked Diffusion Language Models

Ph.D. Candidate at Cornell Tech
November 24, 2025 (Mon), 10:30 a.m. KST
Online (Zoom).

Guest Lecture at CS492(C): Diffusion and Flow Models
Minhyuk Sung, KAIST, Fall 2025


Zoom Link Google Calendar Link

Teaser1

Abstract

We are at a pivotal moment in language modeling: autoregressive (AR) LLMs now face serious competition from diffusion-based approaches. Unlike AR models, diffusion models are not bound to sequential generation, enabling faster sampling, greater controllability, and improved long-term planning. In this talk, I'll introduce a simplified discrete diffusion framework—MDLM—that leverages Rao-Blackwellized likelihood bounds, which are provably tighter and exhibit lower variance than prior formulations. MDLM achieves perplexities close to those of AR models on standard datasets and, for the first time, surpasses AR models on several downstream language generation benchmarks. The impact is already visible: ByteDance’s Seed-Diffusion LLM, built on our MDLM framework, is now the fastest diffusion LLM to date—surpassing Google’s Gemini diffusion.

Bio

Subham Sahoo is a Ph.D. candidate at Cornell Tech, advised by Prof. John Thickstun. His research focuses on diffusion language models and discrete generative modeling. He has co-authored several papers at ICML, NeurIPS, and ICLR, including The Diffusion Duality, Simple and Effective Masked Diffusion Language Models, and Block Diffusion. Prior to his Ph.D., he worked at Google Research and received his B.Tech. from IIT Kharagpur.


  1. Image from https://s-sahoo.com/duo/. ↩