TL;DR: VAEs could generate images, but they were blurry. GANs fixed that by replacing the reconstruction loss with something smarter — a second neural network whose entire job is to call out fakes....

TL;DR: In 2014, Goodfellow et al. showed that neural networks — no matter how accurate — can be fooled by adding tiny, invisible perturbations to their inputs. A panda becomes a gibbon. A stop sign...

Reading Nietzsche requires a kind of uncoulturing — actively unlearning what you think art is before his framework can land. My default assumption, before this book, was that “art” meant Shakespear...

TL;DR: Hinton’s autoencoder taught networks to compress data into a small latent vector and reconstruct it back. It worked beautifully — but the latent space was a mess you couldn’t generate from. ...

TL;DR: RNNs were the first serious attempt at giving neural networks memory. The idea was elegant — feed the past into the present. But they collapsed under their own weight, literally. Gradients v...

Review of all the books I read in 2023.

TL;DR: Before normalization, training deep networks was like trying to stack cards in a hurricane—one small change would topple everything. BatchNorm (2015) changed the game by stabilizing training...

TL;DR: Dropout started as a simple trick to prevent overfitting—randomly turn off neurons during training. But it evolved into something profound: a gateway to understanding uncertainty in deep lea...

NCVPRIPG 2023 • DOI Given two images of handwritten text, were they written by the same person? This is the writer verification problem — a form of biometric identification without any dynamic ...

TL;DR: Every time you train a neural network, you’re solving an optimization problem in a space with millions of dimensions. This is the story of how we went from basic SGD taking baby steps to Ada...