Transformers: More Than Meets the Eye
TL;DR: RNNs processed language one word at a time. LSTMs added memory gates but stayed sequential. The Transformer threw all of that out — no recurrence, no convolutions — and replaced it with a si...
Ceyblogs
SIREN: Teaching Networks to Think in Sine Waves
TL;DR: ReLU networks can’t represent fine detail in continuous signals because their derivatives are piecewise constant and their second derivatives are zero. SIREN replaces ReLU with sine activati...
Semantics and Pragmatics: Where Does Meaning Live?
The standard introductory distinction: semantics is what a sentence means; pragmatics is what a speaker means by saying it. “I’m cold” means there is a person and they are experiencing low tempera...
The JEE Aspirant Experience: Power, Bare Life, and Hegemony
What does it mean to be a JEE dropper? You are 18 or 19 years old. You are not a student — no institution claims you. You are not working. You are preparing for an exam that will, in theory, determ...
Nothing is True; Nothing is Permitted: A Cultural Analysis of Assassin's Creed: India
Assassin’s Creed’s central philosophical claim — nothing is true, everything is permitted — is one of the more genuinely interesting premises in mainstream gaming. It’s a denial of natural social o...
GANs: From a Thought Experiment to Photorealistic Faces
TL;DR: VAEs could generate images, but they were blurry. GANs fixed that by replacing the reconstruction loss with something smarter — a second neural network whose entire job is to call out fakes....
When Neural Networks Lie: Adversarial Examples and the Art of Fooling AI
TL;DR: In 2014, Goodfellow et al. showed that neural networks — no matter how accurate — can be fooled by adding tiny, invisible perturbations to their inputs. A panda becomes a gibbon. A stop sign...
Birth of Tragedy: Appreciation and Criticism
Reading Nietzsche requires a kind of uncoulturing — actively unlearning what you think art is before his framework can land. My default assumption, before this book, was that “art” meant Shakespear...
From Autoencoders to VAEs: Learning to Generate, Not Just Compress
TL;DR: Hinton’s autoencoder taught networks to compress data into a small latent vector and reconstruct it back. It worked beautifully — but the latent space was a mess you couldn’t generate from. ...
LSTMs: How We Taught Neural Networks to Remember
TL;DR: RNNs were the first serious attempt at giving neural networks memory. The idea was elegant — feed the past into the present. But they collapsed under their own weight, literally. Gradients v...