TL;DR: A diffusion model learns a vector field over image space – for every point in a million-dimensional void, it predicts the direction toward the nearest cluster of plausible images. Training r...

TL;DR: Yoshua Bengio’s 2003 paper “A Neural Probabilistic Language Model” is the Genesis of modern NLP. Before this paper, language models were statistical counting machines. After it, they became ...

Review of all the books I read in 2024.

TL;DR: A language model knows only what it memorised during training — no private documents, no events after the cutoff, and no reliable restraint about making things up. Retrieval-Augmented Genera...

ICVGIP 2024 • DOI Video Moment Retrieval (VMR) is the task of finding the specific timestamp in a video that corresponds to a natural language query. You search “the part where they explain the ...

TL;DR: LLaMA is not a new architecture — it is a GPT-style decoder-only Transformer with five deliberate changes: pre-normalization, RMS Norm instead of Layer Norm, Rotary Positional Embeddings on ...

TL;DR: Every language model before BERT read in one direction — either left-to-right or, if “bidirectional,” left-to-right and right-to-left separately then stitched together. BERT asked a simple q...

TL;DR: The original Transformer was two halves — an encoder that reads and a decoder that generates. GPT threw away the left half. Decoder-only models use a single stack of masked self-attention la...

TL;DR: Standard attention is slow not because of the math, but because of memory traffic. The $N \times N$ attention matrix gets written to and read from GPU HBM multiple times, and that data movem...

TL;DR: During autoregressive generation, a Transformer recomputes the Key and Value vectors for every previous token at every single step. This is pure waste – those vectors never change. The KV ca...