The Physics of Language Models
And What We Actually Know About Knowledge
Ceyblogs
The Paper That Started the Neural NLP Revolution: Bengio's Neural Probabilistic Language Model
TL;DR: Yoshua Bengio’s 2003 paper “A Neural Probabilistic Language Model” is the Genesis of modern NLP. Before this paper, language models were statistical counting machines. After it, they became ...
2024 in Books
Review of all the books I read in 2024.
Talk Trois
Talk Trois Log
2023 in Books
Review of all the books I read in 2023.
ML Normalization; A Primer
TL;DR: Before normalization, training deep networks was like trying to stack cards in a hurricane—one small change would topple everything. BatchNorm (2015) changed the game by stabilizing training...
ML Regularization; A Primer
TL;DR: Dropout started as a simple trick to prevent overfitting—randomly turn off neurons during training. But it evolved into something profound: a gateway to understanding uncertainty in deep lea...
ML Optimization; A Primer
TL;DR: Every time you train a neural network, you’re solving an optimization problem in a space with millions of dimensions. This is the story of how we went from basic SGD taking baby steps to Ada...
2022 in Books
Review of all the books I read in 2022.
Detroit; Become Human & the Inheritance of Humanity
Detroit; Become Human is a 2018 thrd-person game, built around three "deviant" androids...